Browser Use: Collect and Verify Information across Websites

A cross-site research task needs more than “browse and summarize.” Without explicit sources and failure rules, an agent can present remembered information as newly collected evidence.

This tutorial reads specified Crawlee, Playwright, and Reader documentation pages. It compares JavaScript-page capability and whether basic use requires a model, retaining source evidence. It uses a local browser and the reader's configured model rather than a cloud-browser plan or a benchmark.

Prepare the input addresses

Locate the following pages in your browser, confirm their contents, and copy their full addresses. Obtain service and repository addresses from account integration settings, official setup instructions, or the repository clone menu. Do not derive real addresses from illustrative output.

CRAWLEE_DOC_URL: the official Crawlee JavaScript quick-start address.
PLAYWRIGHT_DOC_URL: the official Playwright Python intro address.
READER_REPO_URL: the official vakra-dev/reader repository-page address.

Set these in the same Bash/WSL terminal used for subsequent commands:

export CRAWLEE_DOC_URL='<paste-the-actual-crawlee-doc-url>'
export PLAYWRIGHT_DOC_URL='<paste-the-actual-playwright-doc-url>'
export READER_REPO_URL='<paste-the-actual-reader-repo-url>'

Replace each <VARIABLE_NAME> in JSON or task parameters with the copied value before submission; JSON does not expand environment variables. Any example.16yun.cn source in an output illustration shows field structure, not a demo service deployed for this article.

Define the questions before browsing

The projects have different responsibilities, so this is not a strongest-tool ranking. Each row answers bounded questions from its own official page.

An inaccessible source is unavailable, not evidence that a product lacks a feature. Do not silently replace it with an unrelated summary.

Install the local runtime

Use uv with Python 3.12:

uv init --python 3.12 browser-comparison
cd browser-comparison
uv add browser-use==0.13.14 python-dotenv
uv run browser-use install

Browser installation requires network access and possibly system dependencies on Linux. See the official project.

Create a private environment file:

OPENAI_API_KEY=replace-with-your-key
MODEL_NAME=replace-with-an-available-model

Use an available model supported by the selected ChatOpenAI adapter. Another provider requires its matching adapter and settings, not merely a different model string. Running the example consumes the reader's configured model service.

Define explicit partial outcomes

Create compare_sites.py:

import asyncio
import json
import os
from pathlib import Path
from typing import Literal
from dotenv import load_dotenv
from pydantic import BaseModel
from browser_use import Agent, Browser, ChatOpenAI
 
load_dotenv()
sources = {
    "crawlee": os.environ["CRAWLEE_DOC_URL"],
    "playwright": os.environ["PLAYWRIGHT_DOC_URL"],
    "reader": os.environ["READER_REPO_URL"],
}
 
class ToolRow(BaseModel):
    id: Literal["crawlee", "playwright", "reader"]
    status: Literal["read", "unavailable"]
    javascript_support: str | None
    model_required_for_basic_use: bool | None
    evidence_quote: str | None
    source_url: str
 
class Comparison(BaseModel):
    rows: list[ToolRow]
 
async def main():
    browser = Browser(
        use_cloud=False,
        allowed_domains=["crawlee.dev", "playwright.dev", "github.com"],
    )
    agent = Agent(
        task=(
            "Open each official source below and answer only from its page; "
            "do not fill gaps from memory: "
            + json.dumps(sources)
            + ". Return one record per source. Describe JavaScript page support "
              "and whether basic use requires a model, with short verbatim evidence. "
              "Use the given source for source_url. If inaccessible, set "
              "status=unavailable and leave other unknown fields null. "
              "Do not register, purchase, log in, or modify any website."
        ),
        llm=ChatOpenAI(model=os.environ["MODEL_NAME"]),
        browser=browser,
        output_model_schema=Comparison,
        max_failures=3,
    )
    history = await agent.run(max_steps=24)
    history.save_to_file("history.json")
    raw = history.final_result()
    if not raw:
        raise RuntimeError("Agent 没有返回最终数据，请查看 history.json")
    result = Comparison.model_validate_json(raw)
    ids = [row.id for row in result.rows]
    if len(ids) != len(sources) or set(ids) != set(sources):
        raise ValueError("来源记录存在重复或遗漏")
    for row in result.rows:
        if row.source_url.rstrip("/") != sources[row.id].rstrip("/"):
            raise ValueError("结果来源不是指定页面")
        if row.status == "read" and not row.evidence_quote:
            raise ValueError("已读取记录缺少证据")
    Path("comparison.json").write_text(
        result.model_dump_json(indent=2), encoding="utf-8"
    )
    incomplete = not history.is_successful() or any(
        row.status != "read" for row in result.rows
    )
    print({"records": len(result.rows), "incomplete": incomplete})
    if incomplete:
        raise SystemExit(1)
 
asyncio.run(main())

Run:

uv run python compare_sites.py

The example uses the Agent's default cleanup after run. It does not enable a persistent keep-alive session. Debugging with retained browsers requires explicit ownership and shutdown.

Understand the limits of steps and schemas

The step limit bounds action rounds, not a strict elapsed deadline or guaranteed coverage. Inspect history for blocked pages, repeated navigation, or ambiguous tasks before raising it.

A schema checks output shape, not whether the page was read. The application also checks source identity, uniqueness, and evidence presence. Human review must still establish that quotes exist and support each claim.

Unavailable rows preserve partial failure. They must not be translated into negative capability claims.

Inspect the result

This is an example failure structure, not collected model output:

{
  "id": "reader",
  "status": "unavailable",
  "javascript_support": null,
  "model_required_for_basic_use": null,
  "evidence_quote": null,
  "source_url": "https://example.16yun.cn/reader"
}

For a successful row, compare its quote with the original page. Distinguish browser support from mandatory browser use, and optional model features from a basic model requirement.

Keep status and source in any spreadsheet. Null does not mean no: unknown, inaccessible, and established lack of support are different states.

History may contain source material and actions. Manage it as local task data rather than automatically publishing it. This public-document task does not require recording video.

Repair failures deliberately

Model authentication: check environment loading, adapter, and model availability independently of browser connectivity.

Browser startup: check installation, runtime account, binary availability, and Linux prerequisites. Local mode does not provide a cloud account fallback.

Blocked redirect domain: inspect the actual destination and allow only a justified official source, rather than removing all scope controls.

Only two source rows: inspect history and recollect the missing source with a smaller task. Preserve source identity when combining results.

Plausible but nonexistent quote: reject the claim and inspect the source directly. Structured output cannot certify factual accuracy.

Repeated steps: inspect stalled state and reduce unrelated demands. Keep this task to three sources and two questions before extending it.

The value of an agent-produced comparison comes from traceable evidence and explicit failure, not fluent wording alone.