8000 feat: add support for browser rendering by atinux · Pull Request #271 · nuxt-hub/core · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

feat: add support for browser rendering #271

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Sep 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/content/0.index.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ hero:
light: '/images/landing/hero-light.svg'
dark: '/images/landing/hero-dark.svg'
headline:
label: Using AI for User Experience
to: /blog/cloudflare-ai-for-user-experience
label: Browser rendering is available
to: /changelog/hub-browser
icon: i-ph-arrow-right
links:
- label: Start reading docs
Expand Down
4 changes: 2 additions & 2 deletions docs/content/1.docs/2.features/ai.md
Original file line number Diff line number Diff line change
Expand Up @@ -169,11 +169,11 @@ NuxtHub AI is compatible with some functions of the [Vercel AI SDK](https://sdk
Make sure to install the Vercel AI SDK in your project.

```[Terminal]
npx nypm add ai @ai-sdk/vue
npx ni ai @ai-sdk/vue
```

::note
[`nypm`](https://github.com/unjs/nypm) will detect your package manager and install the dependencies with it.
[`ni`](https://github.com/antfu/ni) will detect your package manager and install the dependencies with it.
::

### `useChat()`
Expand Down
230 changes: 230 additions & 0 deletions docs/content/1.docs/2.features/browser.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,230 @@
---
title: Browser Rendering
navigation.title: Browser
description: Control and interact with a headless browser instance in your Nuxt application using Puppeteer.
---

## Getting Started

Enable browser rendering in your Nuxt project by enabling the `hub.browser` option:

```ts [nuxt.config.ts]
export default defineNuxtConfig({
hub: {
browser: true
},
})
```

Lastly, install the required dependencies by running the following command:

```bash [Terminal]
npx ni @cloudflare/puppeteer puppeteer
```

::note
[ni](https://github.com/antfu/ni) will automatically detect the package manager you are using and install the dependencies.
::

## Usage

In your server API routes, you can use the `hubBrowser` function to get a [Puppeteer browser instance](https://github.com/puppeteer/puppeteer):

```ts
const { page, browser } = await hubBrowser()
```

In production, the instance will be from [`@cloudflare/puppeteer`](https://developers.cloudflare.com/browser-rendering/platform/puppeteer/) which is a fork of Puppeteer with version specialized for working within Cloudflare workers.

::tip
NuxtHub will automatically close the `page` instance when the response is sent as well as closing or disconnecting the `browser` instance when needed.
::

## Use Cases

Here are some use cases for using a headless browser like Puppeteer in your Nuxt application:
- **Web scraping:** Extract data from websites, especially those with dynamic content that requires JavaScript execution.
- **Generating PDFs or screenshots:** Create snapshots or PDF versions of web pages.
- **Performance monitoring:** Measure load times, resource usage, and other performance metrics of web applications.
- **Automating interactions or testing:** Simulating user actions on websites for tasks like form filling, clicking buttons, or navigating through multi-step processes.

## Limits

::important
Browser rendering is only available on the [Workers Paid](https://www.cloudflare.com/plans/developer-platform/) plan for now.
::

To improve the performance in production, NuxtHub will reuse browser sessions. This means that the browser will stay open after each request (for 60 seconds), a new request will reuse the same browser session if available or open a new one.

The Cloudflare limits are:
- 2 new browsers per minute per Cloudflare account
- 2 concurrent browser sessions per account
- a browser instance gets killed if no activity is detected for 60 seconds (idle timeout)

You can extend the idle timeout by giving the `keepAlive` option when creating the browser instance:

```ts
// keep the browser instance alive for 120 seconds
const { page, browser } = await hubBrowser({ keepAlive: 120 })
```

The maximum idle timeout is 600 seconds (10 minutes).

::tip
Once NuxtHub supports [Durable Objects](https://github.com/nuxt-hub/core/issues/50), you will be able to create a single browser instance that will stay open for a long time, and you will be able to reuse it across requests.
::

## Screenshot Capture

Taking a screenshot of a website is a common use case for a headless browser. Let's create an API route to capture a screenshot of a website:

```ts [server/api/screenshot.ts]
import { z } from 'zod'

export default eventHandler(async (event) => {
// Get the URL and theme from the query parameters
const { url, theme } = await getValidatedQuery(event, z.object({
url: z.string().url(),
theme: z.enum(['light', 'dark']).optional().default('light')
}).parse)

// Get a browser session and open a new page
const { page } = await hubBrowser()

// Set the viewport to full HD & set the color-scheme
await page.setViewport({ width: 1920, height: 1080 })
await page.emulateMediaFeatures([{
name: 'prefers-color-scheme',
value: theme
}])

// Go to the URL and wait for the page to load
await page.goto(url, { waitUntil: 'domcontentloaded' })

// Return the screenshot as response
setHeader(event, 'content-type', 'image/jpeg')
return page.screenshot()
})
```

On the application side, we can create a simple form to call our API endpoint:

```vue [pages/capture.vue]
<script setup>
const url = ref('https://hub.nuxt.com')
const image = ref('')
const theme = ref('light')
const loading = ref(false)

async function capture {
if (loading.value) return
loading.value = true
const blob = await $fetch('/api/browser/capture', {
query: {
url: url.value,
theme: theme.value
}
})
image.value = URL.createObjectURL(blob)
loading.value = false
}
</script>

<template>
<form @submit.prevent="capture">
<input v-model="url" type="url" />
<select v-model="theme">
<option value="light">Light</option>
<option value="dark">Dark</option>
</select>
<button type="submit" :disabled="loading">
{{ loading ? 'Capturing...' : 'Capture' }}
</button>
<img v-if="image && !loading" :src="image" style="aspect-ratio: 16/9;" />
</form>
</template>
```

That's it! You can now capture screenshots of websites using Puppeteer in your Nuxt application.

### Storing the screenshots

You can store the screenshots in the Blob storage:

```ts
const screenshot = await page.screenshot()

// Upload the screenshot to the Blob storage
const filename = `screenshots/${url.value.replace(/[^a-zA-Z0-9]/g, '-')}.jpg`
const blob = await hubBlob().put(filename, screenshot)
```

::note{to="/docs/features/blob"}
Learn more about the Blob storage.
::

## Metadata Extraction

Another common use case is to extract metadata from a website.

```ts [server/api/metadata.ts]
import { z } from 'zod'

export default eventHandler(async (event) => {
// Get the URL from the query parameters
const { url } = await getValidatedQuery(event, z.object({
url: z.string().url()
}).parse)

// Get a browser instance and navigate to the url
const { page } = await hubBrowser()
await page.goto(url, { waitUntil: 'networkidle0' })

// Extract metadata from the page
const metadata = await page.evaluate(() => {
const getMetaContent = (name) => {
const element = document.querySelector(`meta[name="${name}"], meta[property="${name}"]`)
return element ? element.getAttribute('content') : null
}

return {
title: document.title,
description: getMetaContent('description') || getMetaContent('og:description'),
favicon: document.querySelector('link[rel="shortcut icon"]')?.href
|| document.querySelector('link[rel="icon"]')?.href,
ogImage: getMetaContent('og:image'),
origin: document.location.origin
}
})

return metadata
})
```

Visiting `/api/metadata?url=https://cloudflare.com` will return the metadata of the website:

```json
{
"title": "Connect, Protect and Build Everywhere | Cloudflare",
"description": "Make employees, applications and networks faster and more secure everywhere, while reducing complexity and cost.",
"favicon": "https://www.cloudflare.com/favicon.ico",
"ogImage": "https://cf-assets.www.cloudflare.com/slt3lc6tev37/2FNnxFZOBEha1W2MhF44EN/e9438de558c983ccce8129ddc20e1b8b/CF_MetaImage_1200x628.png",
"origin": "https://www.cloudflare.com"
}
```

To store the metadata of a website, you can use the [Key Value Storage](/docs/features/kv).

Or directly leverage [Caching](/docs/features/cache) on this API route:

```ts [server/api/metadata.ts]
export default cachedEventHandler(async (event) => {
// ...
}, {
maxAge: 60 * 60 * 24 * 7, // 1 week
swr: true,
// Use the URL as key to invalidate the cache when the URL changes
// We use btoa to transform the URL to a base64 string
getKey: (event) => btoa(getQuery(event).url),
})
```
2 changes: 1 addition & 1 deletion docs/content/1.docs/3.recipes/5.postgres.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ The module ensures that you can connect to your PostgreSQL database using [Cloud
2. Install the [`postgres`](https://www.npmjs.com/package/postgres) NPM package in your project.

```bash
npx nypm add postgres
npx ni postgres
```

::tip{icon="i-ph-rocket-launch"}
Expand Down
66 changes: 66 additions & 0 deletions docs/content/4.changelog/hub-browser.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
---
title: Browser Rendering is available
description: "Taking screenshots, crawling websites, extracting information has never been easier with `hubBrowser()`."
date: 2024-08-28
image: '/images/changelog/nuxthub-browser.jpg'
authors:
- name: Sebastien Chopin
avatar:
src: https://avatars.githubusercontent.com/u/904724?v=4
to: https://x.com/atinux
username: atinux
---

::tip
This feature is available on both [free and pro plans](/pricing) of NuxtHub but on the [Workers Paid plan](https://www.cloudflare.com/plans/developer-platform/) for your Cloudflare account.
::

We are excited to introduce [`hubBrowser()`](/docs/features/browser). This new method allows you to run a headless browser directly in your Nuxt application using [Puppeteer](https://github.com/puppeteer/puppeteer).

::video{poster="https://res.cloudinary.com/nuxt/video/upload/v1725901706/nuxthub/nuxthub-browser_dsn1m1.jpg" controls class="w-full h-auto rounded"}
:source{src="https://res.cloudinary.com/nuxt/video/upload/v1725901706/nuxthub/nuxthub-browser_dsn1m1.webm" type="video/webm"}
:source{src="https://res.cloudinary.com/nuxt/video/upload/v1725901706/nuxthub/nuxthub-browser_dsn1m1.mov" type="video/mp4"}
:source{src="https://res.cloudinary.com/nuxt/video/upload/v1725901706/nuxthub/nuxthub-browser_dsn1m1.ogg" type="video/ogg"}
::

## How to use hubBrowser()

1. Update `@nuxthub/core` to the latest version (`v0.7.11` or later)

2. Enable `hub.browser` in your `nuxt.config.ts`

```ts [nuxt.config.ts]
export default defineNuxtConfig({
hub: {
browser: true
}
})
```

3. Install the required dependencies

```bash [Terminal]
npx ni @cloudflare/puppeteer puppeteer
```

4. Start using [`hubBrowser()`](/docs/features/browser) in your server routes

```ts [server/api/screenshot.ts]
export default eventHandler(async (event) => {
const { page } = await hubBrowser()

await page.setViewport({ width: 1920, height: 1080 })
await page.goto('https://cloudflare.com')

setHeader(event, 'content-type', 'image/jpeg')
return page.screenshot()
})
```

5. Before deploying, make sure you are subscribed to the [Workers Paid plan](https://www.cloudflare.com/plans/developer-platform/)

6. [Deploy your project with NuxtHub](/docs/getting-started/deploy)

::note{to="/docs/features/browser"}
Read the documentation about `hubBrowser()` with more examples.
::
2 changes: 1 addition & 1 deletion docs/pages/index.vue
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ onMounted(() => {
</template>

<template #title>
<MDC :value="page?.hero.title" />
<span v-html="page?.hero.title" />
</template>

<template #description>
Expand Down
Binary file added docs/public/images/changelog/nuxthub-browser.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
5 changes: 3 additions & 2 deletions playground/app/app.vue
7180
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,10 @@ useSeoMeta({
const links = [
{ label: 'Home', to: '/' },
{ label: 'AI', to: '/ai' },
{ label: 'Browser', to: '/browser' },
{ label: 'Blob', to: '/blob' },
{ label: 'Database', to: '/database' },
{ label: 'KV', to: '/kv' },
{ label: 'Blob', to: '/blob' }
{ label: 'KV', to: '/kv' }
]
</script>

Expand Down
Loading
0