Files
Obsidian/X-Articles/Hermes Has a Browser Escape Hatch Most Agents Don’t.md
2026-07-21 14:53:24 +08:00

376 lines
8.9 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
title: "Hermes Has a Browser Escape Hatch Most Agents Don’t"
source: "https://x.com/NeoAIForecast/status/2047521653954089113"
author:
- "[[@NeoAIForecast]]"
published: 2026-04-24
created: 2026-04-24
description: "Most AI browser agents break when the page gets weird.A button hides inside an iframe. A native dialog blocks execution. A cookie issue ruin..."
tags:
- "clippings"
---
![图像](https://pbs.twimg.com/media/HGo8CribEAA83HP?format=jpg&name=large)
Most AI browser agents break when the page gets weird.
A button hides inside an iframe. A native dialog blocks execution. A cookie issue ruins the session. A JS-heavy app does not show the state you need.
Hermes has a technical answer for this: **browser\_cdp.**
It is not the normal browser tool. It is the escape hatch underneath it.
Most browser agents are built around high-level actions:
- navigate to a page
- click a button
- type into a field
- scroll
- read the page
- take a screenshot
That works until it doesn’t.
Modern websites are messy. They have shadow DOMs, iframes, native dialogs, hidden state, client-side routing, weird auth flows, cookie problems, dynamic rendering, and JavaScript errors that never appear in the visible page.
Hermes Agent handles normal browser work with high-level tools like:
```plaintext
browser_navigate
browser_snapshot
browser_click
browser_type
browser_console
browser_vision
```
But when those are not enough, Hermes exposes something much lower level:
```plaintext
browser_cdp
```
That stands for:
**Chrome DevTools Protocol.**
It lets Hermes send raw CDP commands directly to a connected Chrome instance.
In plain English:
Hermes can drop below “AI browser control” and talk to the browser like DevTools does. That is a serious power-user feature.
## Why high-level browser tools are not enough
High-level browser tools are great for most agent workflows.
They give the model a clean accessibility-tree view of the page. Hermes can see interactive elements, assign refs like [@e1](https://x.com/@e1) or [@e2](https://x.com/@e2), click buttons, fill forms, scroll, and inspect page text.
That is exactly what you want for normal browsing. But the abstraction can hide important details.
For example:
- the accessibility tree may not expose the element you need
- an iframe may require target-specific inspection
- a native JavaScript dialog may block the page
- a frontend error may only appear in the console
- a site may store critical state in cookies or local storage
- a SPA may render data after network calls complete
- visual state may differ from text state
Most agents either fail here or hallucinate around it.
Hermes gives the agent a lower-level path.
Takeaway: High-level browser tools are the default. **browser\_cdp** is what you use when the default abstraction leaks.
## What browser\_cdp actually does
browser\_cdp is a raw Chrome DevTools Protocol passthrough.
That means Hermes can call CDP methods such as:
```plaintext
Target.getTargets
Runtime.evaluate
Page.handleJavaScriptDialog
Network.getAllCookies
DOM.*
Emulation.*
Storage.*
Browser.*
```
The official Hermes docs describe it as an escape hatch for browser operations not covered by normal browser tools.
Common uses include:
- native dialog handling
- iframe-scoped evaluation
- cookie inspection
- network control
- direct JavaScript evaluation
- browser/tab target inspection
- low-level DOM operations
- debugging page state
Example:
```plaintext
browser_cdp(method="Target.getTargets")
```
That lists available browser targets/tabs.
Then a page-level command can target a specific tab:
```plaintext
browser_cdp(
method="Runtime.evaluate",
params={
"expression": "document.title",
"returnByValue": true
},
target_id="<tabId>"
)
```
This is not “click the button.”
This is “ask Chrome directly what the page is doing.”
Takeaway: browser\_cdp turns Hermes from a browser user into a browser debugger.
## The setup is intentionally explicit
browser\_cdp is not always available.
Hermes only exposes it when a reachable CDP endpoint exists at session start.
The clean path is local Chrome via:
```plaintext
/browser connect
```
Inside the Hermes CLI, you can run:
```plaintext
/browser connect
/browser status
/browser disconnect
```
By default, Hermes connects to:
```plaintext
ws://localhost:9222
```
You can also provide a specific CDP endpoint:
```plaintext
/browser connect ws://host:port
```
If Chrome is not already running with remote debugging, Hermes can attempt to launch it.
The manual Linux path looks like this:
```bash
google-chrome \
--remote-debugging-port=9222 \
--user-data-dir=$HOME/.hermes/chrome-debug \
--no-first-run \
--no-default-browser-check &
```
Then launch Hermes from the terminal and run:
```plaintext
/browser connect
```
Important detail:
**/browser connect** is an interactive CLI slash command.
It is not dispatched by Telegram, Discord, WebUI, or other gateway chats. If you type it into a gateway chat, it will be treated like normal text instead of executing the slash command.
Takeaway: browser\_cdp is a CLI/operator feature. Start Hermes locally when you want raw browser control.
## The mental model: two browser layers
Hermes browser automation has two layers.
**Layer 1: Agent-friendly browser tools**
Use these for normal workflows:
```plaintext
browser_navigate
browser_snapshot
browser_click
browser_type
browser_scroll
browser_press
browser_console
browser_vision
```
This layer is optimized for LLMs.
Hermes converts the page into a text-based accessibility tree. The agent sees refs like [@e3](https://x.com/@e3), clicks them, types into fields, and verifies results.
This is the default path.
**Layer 2: Raw CDP passthrough**
Use this when the first layer cannot see or control what you need:
**browser\_cdp**
This layer is optimized for power users. It exposes the Chrome DevTools Protocol directly.
The agent can inspect targets, evaluate JavaScript, read cookies, handle dialogs, and interact with browser internals.
Takeaway: Hermes does not force every browser task through one abstraction. It gives you a clean path and a low-level fallback.
![图像](https://pbs.twimg.com/media/HGpCAqTbUAAGT0l?format=jpg&name=large)
## When I would use browser\_cdp
**Good use cases**
1\. Native JavaScript dialogs
If a site opens a blocking alert, confirm, or prompt, the normal click/type loop may get stuck.
CDP can handle it directly:
```plaintext
Page.handleJavaScriptDialog
```
2\. Debugging frontend state
If the page looks wrong, ask the browser directly:
```plaintext
Runtime.evaluate
```
Example expression:
```plaintext
document.title
```
Or:
```plaintext
window.location.href
```
Or app-specific state if exposed globally.
3\. Cookie/session inspection
For auth issues, CDP can inspect cookies:
```plaintext
Network.getAllCookies
```
That is much more precise than guessing whether a login “worked.”
4\. Iframe problems
Some pages hide the useful interaction inside nested frames.
CDP gives the agent a way to inspect targets and work closer to the browser’s real structure.
5\. Frontend QA
For local web-app testing, Hermes can combine:
```plaintext
browser_console
browser_snapshot
browser_cdp
```
That means it can see the page, check the console, and query browser internals.
Takeaway: Use browser\_cdp when the browser state matters more than the visible page text.
## The workflow I’d teach operators
Here is the practical Hermes browser-debug workflow.
Step 1: Start Chrome with CDP
```bash
google-chrome \
--remote-debugging-port=9222 \
--user-data-dir=$HOME/.hermes/chrome-debug \
--no-first-run \
--no-default-browser-check &
```
Step 2: Open Hermes locally
```plaintext
hermes
```
Step 3: Connect the browser
```plaintext
/browser connect
```
Step 4: Verify the connection
```plaintext
/browser status
```
Step 5: Use normal browser tools first
Ask Hermes to navigate, inspect, click, type, and read the page normally.
Example:
```plaintext
Open my local app at http://localhost:3000, inspect the login flow, and check for JS errors.
```
Step 6: Drop to CDP only when needed
If the normal browser tools cannot reach the right state, ask Hermes to use browser\_cdp.
Example:
```plaintext
Use browser_cdp to list targets, identify the active tab, and evaluate document.title on that tab.
```
Step 7: Disconnect when done
```plaintext
/browser disconnect
```
Takeaway: Treat CDP as the escalation path, not the first move.
## Final takeaway
Most browser agents give you a steering wheel. Hermes also gives you access to the diagnostic port.
That is the difference.
For normal tasks, use the clean high-level browser tools. But when the page gets weird, Hermes can drop into Chrome DevTools Protocol and debug the browser directly.
That is why browser\_cdp is one of the most technical and underrated Hermes features.
It is not flashy, It is better than flashy, It is the escape hatch you want when real web automation breaks.