Files
Obsidian/X-Articles/Hermes Has a Browser Escape Hatch Most Agents Don’t.md
T
2026-07-21 14:53:24 +08:00

8.9 KiB
Raw Blame History

title, source, author, published, created, description, tags
title source author published created description tags
Hermes Has a Browser Escape Hatch Most Agents Don’t https://x.com/NeoAIForecast/status/2047521653954089113
@NeoAIForecast
2026-04-24 2026-04-24 Most AI browser agents break when the page gets weird.A button hides inside an iframe. A native dialog blocks execution. A cookie issue ruin...
clippings

图像

Most AI browser agents break when the page gets weird.

A button hides inside an iframe. A native dialog blocks execution. A cookie issue ruins the session. A JS-heavy app does not show the state you need.

Hermes has a technical answer for this: browser_cdp.

It is not the normal browser tool. It is the escape hatch underneath it.

Most browser agents are built around high-level actions:

  • navigate to a page
  • click a button
  • type into a field
  • scroll
  • read the page
  • take a screenshot

That works until it doesn’t.

Modern websites are messy. They have shadow DOMs, iframes, native dialogs, hidden state, client-side routing, weird auth flows, cookie problems, dynamic rendering, and JavaScript errors that never appear in the visible page.

Hermes Agent handles normal browser work with high-level tools like:

browser_navigate
browser_snapshot
browser_click
browser_type
browser_console
browser_vision

But when those are not enough, Hermes exposes something much lower level:

browser_cdp

That stands for:

Chrome DevTools Protocol.

It lets Hermes send raw CDP commands directly to a connected Chrome instance.

In plain English:

Hermes can drop below “AI browser control” and talk to the browser like DevTools does. That is a serious power-user feature.

Why high-level browser tools are not enough

High-level browser tools are great for most agent workflows.

They give the model a clean accessibility-tree view of the page. Hermes can see interactive elements, assign refs like @e1 or @e2, click buttons, fill forms, scroll, and inspect page text.

That is exactly what you want for normal browsing. But the abstraction can hide important details.

For example:

  • the accessibility tree may not expose the element you need
  • an iframe may require target-specific inspection
  • a native JavaScript dialog may block the page
  • a frontend error may only appear in the console
  • a site may store critical state in cookies or local storage
  • a SPA may render data after network calls complete
  • visual state may differ from text state

Most agents either fail here or hallucinate around it.

Hermes gives the agent a lower-level path.

Takeaway: High-level browser tools are the default. browser_cdp is what you use when the default abstraction leaks.

What browser_cdp actually does

browser_cdp is a raw Chrome DevTools Protocol passthrough.

That means Hermes can call CDP methods such as:

Target.getTargets
Runtime.evaluate
Page.handleJavaScriptDialog
Network.getAllCookies
DOM.*
Emulation.*
Storage.*
Browser.*

The official Hermes docs describe it as an escape hatch for browser operations not covered by normal browser tools.

Common uses include:

  • native dialog handling
  • iframe-scoped evaluation
  • cookie inspection
  • network control
  • direct JavaScript evaluation
  • browser/tab target inspection
  • low-level DOM operations
  • debugging page state

Example:

browser_cdp(method="Target.getTargets")

That lists available browser targets/tabs.

Then a page-level command can target a specific tab:

browser_cdp(
  method="Runtime.evaluate",
  params={
    "expression": "document.title",
    "returnByValue": true
  },
  target_id="<tabId>"
)

This is not “click the button.”

This is “ask Chrome directly what the page is doing.”

Takeaway: browser_cdp turns Hermes from a browser user into a browser debugger.

The setup is intentionally explicit

browser_cdp is not always available.

Hermes only exposes it when a reachable CDP endpoint exists at session start.

The clean path is local Chrome via:

/browser connect

Inside the Hermes CLI, you can run:

/browser connect
/browser status
/browser disconnect

By default, Hermes connects to:

ws://localhost:9222

You can also provide a specific CDP endpoint:

/browser connect ws://host:port

If Chrome is not already running with remote debugging, Hermes can attempt to launch it.

The manual Linux path looks like this:

google-chrome \
  --remote-debugging-port=9222 \
  --user-data-dir=$HOME/.hermes/chrome-debug \
  --no-first-run \
  --no-default-browser-check &

Then launch Hermes from the terminal and run:

/browser connect

Important detail:

/browser connect is an interactive CLI slash command.

It is not dispatched by Telegram, Discord, WebUI, or other gateway chats. If you type it into a gateway chat, it will be treated like normal text instead of executing the slash command.

Takeaway: browser_cdp is a CLI/operator feature. Start Hermes locally when you want raw browser control.

The mental model: two browser layers

Hermes browser automation has two layers.

Layer 1: Agent-friendly browser tools

Use these for normal workflows:

browser_navigate
browser_snapshot
browser_click
browser_type
browser_scroll
browser_press
browser_console
browser_vision

This layer is optimized for LLMs.

Hermes converts the page into a text-based accessibility tree. The agent sees refs like @e3, clicks them, types into fields, and verifies results.

This is the default path.

Layer 2: Raw CDP passthrough

Use this when the first layer cannot see or control what you need:

browser_cdp

This layer is optimized for power users. It exposes the Chrome DevTools Protocol directly.

The agent can inspect targets, evaluate JavaScript, read cookies, handle dialogs, and interact with browser internals.

Takeaway: Hermes does not force every browser task through one abstraction. It gives you a clean path and a low-level fallback.

图像

When I would use browser_cdp

Good use cases

1. Native JavaScript dialogs

If a site opens a blocking alert, confirm, or prompt, the normal click/type loop may get stuck.

CDP can handle it directly:

Page.handleJavaScriptDialog

2. Debugging frontend state

If the page looks wrong, ask the browser directly:

Runtime.evaluate

Example expression:

document.title

Or:

window.location.href

Or app-specific state if exposed globally.

3. Cookie/session inspection

For auth issues, CDP can inspect cookies:

Network.getAllCookies

That is much more precise than guessing whether a login “worked.”

4. Iframe problems

Some pages hide the useful interaction inside nested frames.

CDP gives the agent a way to inspect targets and work closer to the browser’s real structure.

5. Frontend QA

For local web-app testing, Hermes can combine:

browser_console
browser_snapshot
browser_cdp

That means it can see the page, check the console, and query browser internals.

Takeaway: Use browser_cdp when the browser state matters more than the visible page text.

The workflow I’d teach operators

Here is the practical Hermes browser-debug workflow.

Step 1: Start Chrome with CDP

google-chrome \
  --remote-debugging-port=9222 \
  --user-data-dir=$HOME/.hermes/chrome-debug \
  --no-first-run \
  --no-default-browser-check &

Step 2: Open Hermes locally

hermes

Step 3: Connect the browser

/browser connect

Step 4: Verify the connection

/browser status

Step 5: Use normal browser tools first

Ask Hermes to navigate, inspect, click, type, and read the page normally.

Example:

Open my local app at http://localhost:3000, inspect the login flow, and check for JS errors.

Step 6: Drop to CDP only when needed

If the normal browser tools cannot reach the right state, ask Hermes to use browser_cdp.

Example:

Use browser_cdp to list targets, identify the active tab, and evaluate document.title on that tab.

Step 7: Disconnect when done

/browser disconnect

Takeaway: Treat CDP as the escalation path, not the first move.

Final takeaway

Most browser agents give you a steering wheel. Hermes also gives you access to the diagnostic port.

That is the difference.

For normal tasks, use the clean high-level browser tools. But when the page gets weird, Hermes can drop into Chrome DevTools Protocol and debug the browser directly.

That is why browser_cdp is one of the most technical and underrated Hermes features.

It is not flashy, It is better than flashy, It is the escape hatch you want when real web automation breaks.