How AI Browser Agents Work: Architecture Diagrams for Atlas, Comet, and Claude Cowork

AI browser agents went mainstream in the first quarter of 2026. Perplexity's Comet expanded from an invite-only beta to a cross-platform release. OpenAI shipped Atlas with an Operator-style "Agent Mode" built directly into the address bar. Anthropic released Claude Cowork on March 23, 2026 — an OS-level agent that controls entire applications, not just a browser tab. Google followed with a Gemini 3 auto-browse preview inside Chrome. Within a few months, "let the AI drive the browser" went from a research demo to a product category with four distinct architectures competing for the same job.

This is a narrower category than "AI agent" in general. An agentic browser architecture is specifically about an action-execution loop that perceives a browser or OS UI and manipulates it — clicking, typing, navigating, filling forms — as opposed to a background agent that calls APIs or an ambient agent that watches for triggers. The trust boundary is different, the failure modes are different, and the diagram needs to show different things: where the perception layer sits, what the action layer is allowed to touch, and where a human has to sign off before something irreversible happens. This guide breaks down the four architectural families in production today, the perception-decision-action loop that powers all of them, the guardrails that keep them from doing damage, and ready-to-use prompt templates for diagramming your own agentic browser system.

Four architectural families

Every product marketed as an "AI browser agent" falls into one of four architectural families. They share the same underlying perception-decision-action loop, but where that loop runs, what it can see, and what it can touch differ enough that they need to be diagrammed differently.

1. Dedicated agentic browser

Examples: Perplexity Comet, OpenAI Atlas. Here the browser itself is the agent runtime — not a browser with a plugin bolted on, but a Chromium fork purpose-built around an agent loop. The agent has native access to the browser's tab manager, cookie jar, extension APIs, and rendering engine, so it can read the DOM directly rather than going through a screenshot round-trip for every step. Because the agent and the browser share a process boundary, these systems can move fast and maintain rich context across tabs, but the security model is also the tightest coupling: a vulnerability in the agent loop has the same blast radius as a vulnerability in the browser itself.

2. OS-level agent

Example: Claude Cowork. This family generalizes past the browser entirely. Instead of controlling one application, an OS-level agent operates a full desktop session — it can drive a browser tab, then switch to a spreadsheet application, then open a terminal, then paste output back into an email client, all within the same task. The perception layer typically combines the accessibility tree exposed by the OS (or individual apps) with periodic screenshots, and the action layer sends OS-level input events (mouse movement, clicks, keystrokes) rather than browser-specific commands. This is architecturally the most powerful — and the highest-risk — family, because the sandbox boundary has to contain an entire desktop environment, not just a browser profile.

3. Embedded copilot

Examples: Copilot Mode in Microsoft Edge, Brave Leo. This family is assistive rather than autonomous: an LLM overlay runs alongside a normal browsing session and can summarize pages, answer questions about on-screen content, or take a small number of pre-approved actions (opening a tab, filling a search box) — but it does not run an unattended multi-step action loop the way the other three families do. Architecturally this is the simplest and lowest-risk pattern: the copilot reads page context on request and proposes actions, and the human executes or approves each one. Many diagrams in this category are really "LLM with page-context injection," not a full agent loop.

4. API-driven operator loop

Example: headless browser automation driven by an LLM via Playwright or the Chrome DevTools Protocol (CDP). This is the family most engineering teams actually build themselves. There is no dedicated browser product — instead, a backend service spins up a headless (or headed) Chromium instance, drives it with Playwright or a raw CDP connection, and an LLM decides what actions to take at each step based on DOM extraction or screenshots. This pattern powers most QA-testing agents, scraping agents, and headless shopping/booking bots, and it is the easiest of the four families to sandbox tightly because the entire runtime is a service you control end to end, with no consumer-facing browser UI to secure.

The perception → decision → action loop

Regardless of family, every agentic browser system runs the same core loop, and your architecture diagram should trace it explicitly as a cycle rather than a one-way pipeline:

1. Perceive: The system extracts a structured representation of the current page or screen — either the DOM / accessibility tree (fast, precise, and the preferred source for dedicated agentic browsers and API-driven operators) or a screenshot (necessary for OS-level agents and any UI that doesn't expose a clean accessibility tree, such as a canvas-rendered app or legacy desktop software). Many production systems combine both: the accessibility tree gives element coordinates and roles, the screenshot gives visual grounding.
2. Structure: Raw DOM or screenshot data is too large and unstructured to hand to an LLM every step. A structuring layer compresses it — pruning irrelevant DOM nodes, labeling interactive elements with stable IDs, resizing and compressing screenshots — into a compact page representation the model can reason over cheaply and consistently across turns.
3. Decide: The structured page representation, the task goal, and the action history are sent to the LLM, which returns a single next action — click element #14, type "wireless mouse" into the search field, scroll down, open a new tab, or declare the task complete. This is the only place in the loop where the model actually reasons; everything else is plumbing.
4. Act: An action executor translates the model's decision into real input. For browser-scoped agents this is a CDP command (click, type, navigate) sent through Playwright or a raw DevTools connection. For OS-level agents this is a native input event — synthetic mouse movement and keystrokes injected at the operating system level, since there is no DOM to target once the agent leaves the browser process.
5. Observe: The system re-perceives the resulting state and compares it to what the model expected. If the action didn't produce the expected change (a click missed, a page failed to load, an unexpected modal appeared), the loop can retry, ask the model to reassess, or escalate to a human rather than blindly continuing.

This loop repeats until the task goal is met, a hard step or time limit is hit, or a guardrail forces a stop. Show it in your diagram as an explicit cycle with a labeled exit condition — an agentic browser diagram without a visible loop termination is missing the detail that most often explains a production incident (a runaway agent that never stopped clicking).

Guardrails architecture

Because the action layer can do anything a human could do in a browser or OS session — including irreversible things — every production-grade agentic browser needs a guardrails layer sitting between the decision step and the action step. This is the part of the architecture that most demo-stage projects skip and that every production incident traces back to.

• Sandboxing: Every agentic browser task should run inside an isolated browser profile or container — a fresh session with no access to the user's other tabs, other logged-in accounts, or the host file system. OS-level agents need a stronger boundary still: a disposable VM or container that scopes the entire desktop session, not just the browser, since the agent can open arbitrary applications.
• Human-in-the-loop confirmation gates: High-risk actions — submitting a payment, confirming a purchase, changing an account password, sending an email, deleting data — should pause the loop and require explicit human approval before the action executor runs. This gate should be a distinct node in your diagram, not an implicit assumption; teams that skip drawing it tend to skip building it too.
• Permission scoping: The agent should operate against an explicit allowlist of domains and action types for a given task, not the open web. A shopping agent scoped to three retail domains cannot be tricked by a malicious page into navigating to an unrelated site and entering credentials there — a class of attack sometimes called prompt injection via page content.
• Rate limiting: Hard caps on actions per task, actions per minute, and total task duration stop a stuck loop from hammering a target site or racking up unbounded model spend. A task that has taken 150 actions is almost never still making progress — it should be forced to stop and escalate.

For the broader security patterns that guardrails like these sit inside — threat models for autonomous action, not just autonomous reasoning — see the agentic AI security architecture guide.

Session and state architecture

The second architectural surface that's easy to under-diagram is session and state management — where credentials live, how the agent handles multiple tabs and windows, and what (if anything) it remembers between tasks.

Credential and auth-token handling

Agentic browsers generally handle credentials one of two ways. Dedicated agentic browsers and embedded copilots typically reuse the user's existing signed-in browser session — cookies and auth tokens already present in the profile — so the agent inherits whatever access the human already has, scoped to that profile. API-driven operator loops and most OS-level automations instead inject credentials explicitly at task start, from a secrets manager or environment variables, into a fresh, isolated profile that is destroyed at task end. The second pattern is significantly easier to audit and revoke, because credential exposure is scoped to a single task rather than persisting across an entire user session. Your diagram should show explicitly which pattern is in use — it is the single most important detail for a security reviewer looking at an agentic browser system.

Multi-tab and multi-window context

Real tasks often span multiple tabs — comparing prices across three retailer tabs, or copying a value from one tab into a form on another. The architecture needs a tab/window manager that tracks which tab is currently in focus, maintains per-tab state (scroll position, form values already entered), and exposes a way for the LLM to explicitly switch context ("go back to tab 2") rather than losing track of where it left off. OS-level agents face the same problem one level up, across application windows rather than browser tabs.

Cross-session memory

Most production agentic browsers deliberately keep cross-session memory minimal or entirely absent — each task starts from a clean profile with no memory of prior tasks, which is also what makes the sandbox-and-destroy credential pattern above possible. Where cross-session memory does exist (a shopping agent that remembers a user's size and shipping address, an OS-level agent that remembers a recurring weekly report format), it should be stored in a separate memory store outside the browser profile itself — never inside cookies or local storage of the disposable session — so that memory persistence and credential isolation remain two independent, separately auditable concerns.

Prompt templates for agentic browser architecture diagrams

Dedicated agentic browser: shopping assistant

OS-level agent: Claude Cowork multi-app workflow

API-driven operator loop: headless QA-testing agent

Embedded copilot: page-context assistant

Comparing the four architectural families

What a good agentic browser architecture diagram must show

Given how much can go wrong in a system that autonomously clicks and types on real websites and applications, a diagram that just draws "LLM → browser" is not useful for a design review. At minimum, an agentic browser architecture diagram should make the following explicit:

• Perception layer: What the agent actually sees at each step — DOM, accessibility tree, screenshot, or a combination — and how that raw data is structured before it reaches the model.
• Action layer: The precise mechanism used to execute decisions — CDP commands via Playwright, or OS-level synthetic input events — and what scope of the system that mechanism can reach (one tab, one browser, or an entire desktop).
• Guardrail / approval gates: Every point where the loop pauses for human confirmation, drawn as a distinct node — not implied. If a diagram doesn't show a gate before payments, account changes, or data deletion, that almost always means the system doesn't have one.
• Credential storage: Whether the agent reuses an existing signed-in session or is injected with scoped, task-specific credentials from a secrets manager — and where those credentials live between tasks, if at all.
• Sandbox boundary: A clearly drawn isolation boundary — container, VM, or dedicated browser profile — separating the agent's execution environment from the host system, the user's other sessions, and production infrastructure.

Common mistakes in agentic browser architecture

• No human confirmation gate on destructive or irreversible actions — payments, account changes, deletions, and message sends should never execute without an explicit approval step in the loop
• Credentials shared across agent sessions — reusing the same logged-in profile or API key across unrelated tasks means a compromised task can access everything every other task could reach
• No action audit log — without a step-by-step record of what the agent clicked, typed, and navigated to, an unexpected outcome is nearly impossible to debug or explain to a user
• Unscoped domain access — letting the agent navigate anywhere on the open web instead of an explicit allowlist makes it far easier for a malicious page to hijack the agent's task via prompt injection embedded in page content
• No sandbox isolation — running an agentic browser task directly in the user's primary profile or on a production host, rather than a disposable container or VM, turns any agent bug into a host-level incident
• No hard step or time limit — a loop with no termination condition beyond "the model says it's done" can run indefinitely against a page it's misinterpreting, racking up cost and risk with every extra step
• Treating screenshots as the only source of truth — vision-only perception is slower and less precise than DOM/accessibility-tree extraction where the latter is available; use screenshots as a grounding supplement, not the primary channel, whenever the target UI exposes a tree

Frequently asked questions about agentic browser architecture

What is an agentic browser?

An agentic browser is a web browser — or a browser-controlling system — built so that an AI agent can autonomously perceive the page, decide on an action, and execute that action through the browser's own interfaces, running a multi-step loop toward a task goal with minimal human intervention. This is distinct from a generic AI agent architecture in that the action space is specifically UI manipulation — clicking, typing, navigating — rather than structured API or tool calls. Some agentic browsers (Comet, Atlas) are dedicated products where the browser itself is the runtime; others are built on top of a normal browser via headless automation frameworks like Playwright.

How is an agentic browser different from an MCP-based agent?

An MCP-based agent acts through structured tool calls exposed by MCP servers — a well-defined API surface with typed inputs and outputs, where a "browser automation MCP server" is just one of many tools available. An agentic browser instead perceives and manipulates a UI directly, with no structured API contract to rely on — it has to interpret visual or DOM state and decide on low-level actions like clicks and keystrokes. The two approaches are complementary rather than competing: a sophisticated agent might use MCP tools for structured tasks (querying a database, creating a ticket) and fall back to agentic browser control only when no API exists for the target system, such as a legacy internal tool or a site that blocks programmatic access.

How do I diagram an agentic browser system for a security review?

Start with the sandbox boundary — container, VM, or isolated browser profile — as the outermost box, since that boundary is the primary control a reviewer will want to verify. Inside it, show the perception layer (DOM/accessibility tree and/or screenshot capture), the decision step (the LLM call), and the action executor (CDP commands or OS input events) as a labeled cycle. Mark every human-in-the-loop confirmation gate as a distinct node on the path to any irreversible action. Show credential flow explicitly — where credentials come from, how they enter the sandbox, and whether they persist after the task ends. For systems where multiple agents or subagents coordinate browser tasks, layer in the patterns from the multi-agent orchestration guide. If your organization requires formal threat models for this kind of autonomous-action system, pair the architecture diagram with a threat model diagram and evaluate the sandbox boundary against zero trust architecture principles — treat every action the agent takes as untrusted until it passes an explicit policy check, not just because it originated inside the sandbox.

Related guides: AI agent architecture diagrams, computer use architecture, agentic AI security architecture, and multi-agent orchestration patterns.