8000 GitHub - altsang/browser-use: Make websites accessible for AI agents
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

altsang/browser-use

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Shows a black Browser Use Logo in light color mode and a white one in dark color mode.

Enable AI to control your browser 🤖

GitHub stars Discord Cloud Documentation Twitter Follow Twitter Follow Weave Badge

🌐 Browser-use is the easiest way to connect your AI agents with the browser.

💡 See what others are building and share your projects in our Discord! Want Swag? Check out our Merch store.

🌤️ Skip the setup - try our hosted version for instant browser automation! Try the cloud ☁︎.

Quick start

With pip (Python>=3.11):

pip install browser-use

For memory functionality (requires Python<3.13 due to PyTorch compatibility):

pip install "browser-use[memory]"

Install Playwright:

playwright install chromium

Spin up your agent:

from langchain_openai import ChatOpenAI
from browser_use import Agent
import asyncio
from dotenv import load_dotenv
load_dotenv()

async def main():
    agent = Agent(
        task="Compare the price of gpt-4o and DeepSeek-V3",
        llm=ChatOpenAI(model="gpt-4o"),
    )
    await agent.run()

asyncio.run(main())

Advanced Mode

For complex browser automation tasks requiring full Playwright capabilities:

from langchain_openai import ChatOpenAI
from browser_use import Agent
from browser_use.browser.browser import Browser, BrowserConfig
import asyncio
from dotenv import load_dotenv
load_dotenv()

async def main():
    # Enable advanced mode with full Playwright capabilities
    browser_config = BrowserConfig(
        advanced_mode=True,  # Enable JavaScript execution, iframe support, etc.
        headless=False       # Set to True for headless operation
    )
    
    browser = Browser(config=browser_config)
    
    agent = Agent(
        task="Navigate complex web interfaces with iframes and dynamic content",
        llm=ChatOpenAI(model="gpt-4o"),
        browser=browser
    )
    await agent.run()

asyncio.run(main())

Add your API keys for the provider you want to use to your .env file.

OPENAI_API_KEY=
ANTHROPIC_API_KEY=
AZURE_OPENAI_ENDPOINT=
AZURE_OPENAI_KEY=
AZURE_OPENAI_API_VERSION=
GEMINI_API_KEY=
DEEPSEEK_API_KEY=
GROK_API_KEY=
NOVITA_API_KEY=

Command-line Flags

The library supports command-line flags for flexible configuration:

# Basic usage
python examples/advance.py

# With command-line flags
python examples/advance.py --cdp-port 9222 --no-headless --model gpt-4o --task "Your custom task"

# Using Azure OpenAI
python examples/advance.py --use-azure --model gpt-4o

# With screenshots enabled
python examples/advance.py --screenshots --no-headless --advanced-mode

Available flags:

  • --cdp-port PORT: CDP port for browser connection (for connecting to existing browser)
  • --headless/--no-headless: Run browser in headless/visible mode
  • --advanced-mode/--no-advanced-mode: Enable/disable advanced Playwright capabilities
  • --model MODEL: Model to use (default: gpt-4o)
  • --use-azure: Use Azure OpenAI instead of OpenAI
  • --task "TASK": Custom task to perform
  • --screenshots: Enable taking screenshots during automation
  • --debug: Enable debug logging

Complete Command-line Reference

Example scripts support various command-line arguments:

advance.py

python examples/advance.py [options]

Options:

  • --use-azure: Use Azure OpenAI instead of OpenAI
  • --model MODEL: Specify the model name (default: gpt-4o)
  • --no-headless: Run in visible browser mode
  • --advanced-mode: Enable advanced Playwright capabilities
  • --cdp-port PORT: Specify CDP port for browser connection
  • --screenshots: Enable taking screenshots during navigation
  • --debug: Enable debug logging

Example with all options:

python examples/advance.py --no-headless --advanced-mode --use-azure --model gpt-4o --cdp-port 9222 --screenshots --debug

Screenshot Functionality

The browser-use library provides screenshot capabilities in two ways:

  • Base64 encoded screenshots: The BrowserContext.take_screenshot() method returns a base64 encoded screenshot (doesn't save to file)
  • File-based screenshots: Example scripts like advance.py include custom functions to save screenshots to files

Important: Screenshots are not saved to files by default. When running examples that support screenshots (like advance.py), you must explicitly enable screenshots with the --screenshots flag:

python examples/advance.py --screenshots --no-headless --advanced-mode

Screenshots will be saved to ~/screenshots/ directory using platform-independent paths. The directory will be created automatically if it doesn't exist on both Linux and macOS systems.

Note: The screenshot functionality has no dependencies on any specific environment and works across all platforms using standard Python and Playwright functionality.

Environment-Specific Usage

Local Mac Usage

# Extract CDP port
CDP_PORT=$(ps -ax | grep -o '\-\-remote-debugging-port=[0-9]\+' | awk -F= '{print $2}' | head -1)

# Run with visible browser window
python examples/advance.py --no-headless --task "Your custom task"

# Run with advanced mode and custom model
python examples/advance.py --no-headless --advanced-mode --model gpt-4o

# For Naver Maps photo navigation on Mac
python examples/advance.py --cdp-port $CDP_PORT --no-headless --advanced-mode --model gpt-4o

# With screenshots enabled
python examples/advance.py --no-headless --advanced-mode --screenshots

Mac-Specific Screenshot Notes

When running on macOS:

  • The screenshots directory is created at ~/screenshots/ using platform-independent paths
  • You must explicitly enable screenshots with the --screenshots flag
  • Screenshots are saved with timestamps and descriptive names
  • The directory will be created automatically if it doesn't exist

Devin/Remote Environment Usage

# Extract CDP port for connecting to existing browser
CDP_PORT=$(ps aux | grep -o '\-\-remote-debugging-port=[0-9]\+' | awk -F= '{print $2}' | head -1)

# Run with CDP port connection
python examples/advance.py --cdp-port $CDP_PORT --task "Your custom task"

# Run with Azure OpenAI
python examples/advance.py --cdp-port $CDP_PORT --use-azure --model gpt-4o

For other settings, models, and more, check out the documentation 📕.

Enhanced Features

Advanced Mode Capabilities

The advanced mode (advanced_mode=True) enables full Playwright capabilities:

  • JavaScript Execution: Run custom JavaScript via page.evaluate() for complex interactions
  • Iframe Support: Access and traverse nested iframes using page.frames()
  • Enhanced UI Interaction: Full locator strategy support and React component interaction
  • Korean Language Support: Improved handling of Korean text elements
  • Dynamic Content Handling: Better waiting for JavaScript page loads and state changes

Enhanced Korean Text Detection

Advanced mode includes specialized methods for Korean websites:

# Example: Using enhanced Korean text detection
await context.get_element_by_korean_text("외부")  # Find element by Korean text
await context.get_naver_photo_elements(search_in_frames=True)  # Find photo elements in Naver Maps
await context.get_element_by_photo_category("외부")  # Find photo category by name

Iframe Navigation Enhancements

Advanced mode provides improved iframe handling:

# Example: Working with iframes
frames = await context.get_frames()  # Get all frames on the page
place_frame = await context.get_frame_by_url_pattern("pcmap.place.naver.com")  # Find frame by URL pattern
await context.switch_to_frame(place_frame)  # Switch context to specific frame

Dynamic Element Selection

Advanced mode provides improved methods for handling dynamic elements:

# Example: Using dynamic element selection
# Find elements with retry logic
element = await context.get_element_with_retry(selector=".dynamic-class", 
                                              retry_count=5, 
                                              wait_time=1000)

# Find elements by text content with fuzzy matching
element = await context.get_element_by_text_content("Approximate text", 
                                                   fuzzy_match=True)

Timing and State Management

Advanced mode includes improved timing and state management:

# Example: Using enhanced timing and state management
# Wait for network to be idle
await context.wait_for_network_idle(timeout=10000)

# Wait for element to be visible with custom timeout
await context.wait_for_element_visible(".selector", timeout=5000)

# Wait for page to finish navigation
await context.wait_for_navigation_complete()

Screenshots

Advanced mode provides enhanced screenshot capabilities:

# Take a screenshot and get base64 encoded data
screenshot_b64 = await context.take_screenshot(full_page=True)

# In example scripts, enable screenshots with the --screenshots flag
# python examples/advance.py --screenshots --no-headless --advanced-mode [other options]

Screenshots will be saved to ~/screenshots/ directory with timestamps and descriptive names. The directory will be created if it doesn't exist.

Example screenshot usage in custom scripts:

# Save a screenshot to a file (note: this doesn't require the --screenshots flag)
page = await context.get_current_page()
# Use os.path.expanduser to ensure cross-platform compatibility
screenshot_path = os.path.join(os.path.expanduser("~"), "screenshots", "my_screenshot.png")
os.makedirs(os.path.dirname(screenshot_path), exist_ok=True)
await page.screenshot(path=screenshot_path)

# Process a base64 encoded screenshot (available in both normal and advanced mode)
import base64
import os
screenshot_b64 = await context.take_screenshot()
screenshot_bytes = base64.b64decode(screenshot_b64)
screenshot_path = os.path.join(os.path.expanduser("~"), "screenshots", "decoded_screenshot.png")
os.makedirs(os.path.dirname(screenshot_path), exist_ok=True)
with open(screenshot_path, "wb") as f:
    f.write(screenshot_bytes)

Test with UI

You can test browser-use with a UI repository

Or simply run the gradio example:

uv pip install gradio
python examples/ui/gradio_demo.py

Demos



Task: Add grocery items to cart, and checkout.

AI Did My Groceries



Prompt: Add my latest LinkedIn follower to my leads in Salesforce.

LinkedIn to Salesforce



Prompt: Read my CV & find ML jobs, save them to a file, and then start applying for them in new tabs, if you need help, ask me.'

apply.to.jobs.8x.mp4



Prompt: Write a letter in Google Docs to my Papa, thanking him for everything, and save the document as a PDF.

Letter to Papa



Prompt: Look up models with a license of cc-by-sa-4.0 and sort by most likes on Hugging face, save top 5 to file.

hugging_face_high_quality.mp4



More examples

For more examples see the examples folder or join the Discord and show off your project.

Vision

Tell your computer what to do, and it gets it done.

Roadmap

Agent

  • Improve agent memory (summarize, compress, RAG, etc.)
  • Enhance planning capabilities (load website specific context)
  • Reduce token consumption (system prompt, DOM state)

DOM Extraction

  • Improve extraction for datepickers, dropdowns, special elements
  • Improve state representation for UI elements

Rerunning tasks

  • LLM as fallback
  • Make it easy to define workflow templates where LLM fills in the details
  • Return playwright script from the agent

Datasets

  • Create datasets for complex tasks
  • Benchmark various models against each other
  • Fine-tuning models for specific tasks

User Experience

  • Human-in-the-loop execution
  • Improve the generated GIF quality
  • Create various demos for tutorial execution, job application, QA testing, social media, etc.

Contributing

We love contributions! Feel free to open issues for bugs or feature requests. To contribute to the docs, check out the /docs< 9E7A /code> folder.

Local Setup

To learn more about the library, check out the local setup 📕.

main is the primary development branch with frequent changes. For production use, install a stable versioned release instead.

Project Directory Structure

Core Directories

/browser_use/ - Main Package

The core package containing all the browser automation functionality.

/browser_use/agent/ - Agent Module

Contains the LLM agent implementation that processes tasks and controls browser automation.

  • service.py: Core agent service implementation
  • prompts/: System and agent prompts
  • views.py: Data structures for agent state
  • memory/: Agent memory management
  • message_manager/: Handles message formatting and processing

/browser_use/browser/ - Browser Control

Manages browser instances and interactions with Playwright.

  • browser.py: Browser configuration and initialization
  • context.py: Browser context management
  • chrome.py: Chrome-specific implementation
  • utils/: Browser utility functions

/browser_use/dom/ - DOM Manipulation

Handles DOM tree building, element selection, and interaction.

  • buildDomTree.js: JavaScript for DOM tree extraction
  • service.py: DOM service implementation
  • clickable_element_processor/: Processes clickable elements
  • history_tree_processor/: Manages navigation history

/browser_use/controller/ - Controller Logic

Manages the control flow between agent and browser.

  • registry/: Registry for controller components

/browser_use/telemetry/ - Telemetry

Handles usage tracking and analytics.

/browser_use/tools/ - Tools Integration

Contains tools that can be used by the agent.

  • registry.py: Central registry for tool registration
  • standee_detection/: Standee detection tool implementation

Supporting Directories

/docs/ - Documentation

Project documentation and architecture diagrams.

  • cloud/, customize/, development/: Documentation sections

/examples/ - Example Scripts

Example scripts demonstrating library usage.

  • simple.py: Basic usage example
  • advance.py: Advanced usage with full Playwright capabilities

/tests/ - Test Suite

Unit and integration tests.

  • tools/: Tests for tool implementations
  • mind2web_data/: Test data

/static/ - Static Assets

Static files used by the project.


Cooperations

We are forming a commission to define best practices for UI/UX design for browser agents. Together, we're exploring how software redesign improves the performance of AI agents and gives these companies a competitive advantage by designing their existing software to be at the forefront of the agent age.

Email Toby to apply for a seat on the committee.

Swag

Want to show off your Browser-use swag? Check out our Merch store. Good contributors will receive swag for free 👀.

Citation

If you use Browser Use in your research or project, please cite:

@software{browser_use2024,
  author = {Müller, Magnus and Žunič, Gregor},
  title = {Browser Use: Enable AI to control your browser},
  year = {2024},
  publisher = {GitHub},
  url = {https://github.com/browser-use/browser-use}
}

Twitter Follow Twitter Follow

Made with ❤️ in Zurich and San Francisco

About

Make websites accessible for AI agents

Resources

License

Security policy

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 92.8%
  • JavaScript 7.2%
0