Computer Use & Browser Automation

Browser automation gives your agent the ability to browse websites, interact with web applications, extract data, fill forms, and take screenshots — using deterministic browser tools and Computer Use (vision-based UI control) when needed.

---

What your agent can do

• Navigate to URLs and interact with pages
• Click, type, scroll, and hover via Computer Use (vision sub-agent)
• Extract structured data from web pages
• Take screenshots and describe what’s on screen
• Download and process files in the sandbox
• Run terminal commands in an isolated sandbox

---

How it works

Each agent shares one sandbox session per organization (keyed by agent). The session persists across runs so files and environment state carry over:

1. Agent acquires or reuses its sandbox session on first browser or shell tool call
2. Browser navigates to the target URL
3. A screenshot and page context are sent back to the agent
4. Agent chooses the next action (navigate, Computer Use delegation, extract data)
5. Steps repeat until the task is complete
6. The sandbox session stays available for the next run (not destroyed after each task)

Note

Sandboxes run in isolated GKE containers on GCP. They cannot access your local machine. Workspace files are restricted to /workspace.

---

Enabling browser automation

1. Open your agent in the Console
2. Go to Capabilities (or the agent wizard capabilities step)
3. Enable Drive a real browser (Chromium) — and Run code / shell in a sandbox if the agent needs terminal or file access
4. Save your changes

Internal agents include sandbox + browser capabilities by default. Customer-facing agents do not get browser or Computer Use tools.

---

Use cases

Web Data Extraction

Task: Go to https://news.ycombinator.com/front and extract the top 10 stories.
For each story, get:
- Title
- URL
- Number of points
- Number of comments
Return the results as a markdown table.

Form Automation

Task: Go to https://forms.example.com/onboarding and fill in the form:
- Company name: Acme Corp
- Contact: John Smith
- Email: john@acme.com
Submit the form and confirm it was successful.

UI Testing

Task: Test the login flow on https://app.example.com:
1. Navigate to the login page
2. Log in with the test credentials provided in my message
3. Verify you reach the dashboard
4. Take a screenshot of the dashboard
5. Report any errors encountered

Monitoring

Task: Take a screenshot of our status page at https://status.example.com.
If any service shows a non-green status, send an alert to #ops-alerts on Slack
with the screenshot attached.

---

Tips for reliable automation

Be explicit about UI elements

Instead of “click the login button”, say “click the button that says ‘Sign In’ near the top right of the page.”

Add error handling

“If the page takes more than 10 seconds to load, try refreshing once. If it still fails, report back with the URL and a screenshot.”

Test in the panel first

Always test browser tasks in the workspace Test panel before deploying to a Flow.

Never hardcode credentials

Pass credentials in your message or store them in integration connections — do not put secrets in agent instructions.

---

Compute usage

Sandbox and browser time is billed against your plan’s compute allowance (separate from LLM credits). Check usage in Console → Settings → Usage. See Credits & Billing → for plan limits.