Obsidian/openclaw/Bug VisionImage Recognition Broken Across Multiple Channels · Issue 23452 · openclawopenclaw.md

title, source, author, published, created, description, tags

title	source	author	published	created	description	tags
[Bug] Vision/Image Recognition Broken Across Multiple Channels · Issue #23452 · openclaw/openclaw	https://github.com/openclaw/openclaw/issues/23452	zlxdengtianhei	2026-02-22	2026-04-10	Summary OpenClaw has systemic issues with image/vision handling across multiple channels (Discord, Telegram, OpenWebUI). Images sent by user	clippings

Summary

OpenClaw has systemic issues with image/vision handling across multiple channels (Discord, Telegram, OpenWebUI). Images sent by users are not properly forwarded to vision-capable models, causing models to hallucinate or respond as if no image was provided. This affects both primary vision models and imageModel fallback configurations.

Related Issues

• [telegram] Images not being passed to local Ollama vision model (qwen2.5vl) #7564 - Telegram images not passed to imageModel
• [Bug]: #6410 - Discord image model doesn't receive image, hallucinates
• [Bug]: /v1/chat/completions drops image inputs from OpenWebUI (Qwen coder-model + vision-model split) #18583 - /v1/chat/completions drops image inputs from OpenWebUI
• [Bug]: Discord attachments download returns same cached file for all new uploads #7917 - Discord attachments download returns same cached file
• [Bug] Agent read tool returns image but not sent to Telegram channel #11735 - Agent read tool image not sent to Telegram channel
• [Bug]: image tool ignores agents.defaults.models parameter overrides (maxTokens), resulting in "Image model returned no text" error #8096 - image tool ignores model parameter overrides

Problem Description

Current Behavior

When users send images to OpenClaw through various channels:

1. Discord: Images are not passed to vision models. Models hallucinate descriptions or claim they cannot see images.
2. Telegram: Images are received by gateway but not passed to configured imageModel. Text model responds instead.
3. OpenWebUI (via /v1/chat/completions): Image parts are dropped before reaching the agent. Only text content is processed.
4. Caching Issues: Discord attachments sometimes return stale/cached content instead of fresh uploads.

Expected Behavior

Per OpenClaw Documentation:

agents.defaults.imageModel is used only when the primary model can't accept images.

The expected flow:

1. User sends image attachment
2. Gateway detects image and extracts media
3. If primary model supports vision (input: ["text", "image"]), forward image to primary model
4. If primary model doesn't support vision, use configured imageModel
5. Model receives actual image data (not just text placeholders)
6. Model responds with accurate image description

Actual Flow (Broken)

User sends image → Gateway receives → [BROKEN] → Model receives text placeholder only
                                      ↓
                              Image not forwarded to:
                              - Primary vision model, OR
                              - Configured imageModel

Evidence

Issue #7564 (Telegram)

User configured:

• Primary: ollama/qwen3-coder:latest (text-only)
• imageModel: ollama/qwen2.5vl:7b (vision)

Result: Text model responds "I'm receiving images but cannot see them"
Logs show: No media, photo, image, or imageModel references

Direct API test to Ollama with base64 image: Works correctly (450+ prompt tokens)

Issue #6410 (Discord)

User sends image with "describe this image"
Model: Claude Opus 4.5 via LiteLLM
Result: Model hallucinates/fabricates description

Issue #18583 (OpenWebUI)

Code analysis reveals:

• extractTextContent() in src/gateway/openai-http.ts only keeps text parts
• Image parts are not parsed or forwarded to agentCommand()
• /v1/responses endpoint handles images correctly (proven working path)
• /v1/chat/completions drops image inputs

Root Cause Analysis

Based on code review and issue analysis:

1. Incomplete Image Parsing:
   • OpenAI-compatible endpoint doesn't parse image_url content parts
     • Only text content is extracted and forwarded
2. Broken Model Routing:
   • imageModel configuration ignored when primary model claims vision support
     • Primary model may claim input: ["text", "image"] but not actually receive image data
3. Channel-Specific Issues:
   • Discord: Attachment caching/download issues
     • Telegram: Media not passed to image processing pipeline
4. Missing Integration:
   • Media attachments converted to text placeholders: [Image: source: /path/to/image.jpg]
     • Actual image bytes not forwarded to models

Impact

• Severity: High
• Frequency: 100% reproducible in affected configurations
• User Experience: Vision/multimodal capabilities completely broken
• Workaround Complexity: High - requires manual intervention

Workarounds (Current)

1. Manual image tool call:

   image({ image: "/path/to/local/image.jpg", prompt: "Describe this image" })
   

2. Use /v1/responses endpoint (for API users) - confirmed working
3. Avoid vision workflows - use text-only interactions

Proposed Solutions

Short-term

1. Fix /v1/chat/completions image handling:
   • Parse image_url content parts (URL and data-URI/base64)
     • Validate via existing media helpers
     • Forward to agentCommand() for both stream and non-stream paths
     • Raise default max body size (currently 1MB too small for base64 images)
2. Fix Discord image ingestion:
   • Resolve attachment caching issues
     • Ensure fresh downloads for each upload
3. Fix Telegram imageModel routing:
   • Ensure images are passed to configured imageModel
     • Add logging for media detection and routing decisions

Long-term

1. Unified image handling: Align all endpoints and channels to use same image processing pipeline
2. Better model capability detection: Actually verify vision models receive and can process images
3. Comprehensive testing: Add integration tests for vision workflows across all channels

Environment

• OpenClaw versions: 2026.1.24 through 2026.2.14
• Affected channels: Discord, Telegram, OpenWebUI
• Affected providers: Ollama, LiteLLM, OpenRouter, and others
• OS: Linux, Windows, macOS

Additional Context

The /v1/responses endpoint demonstrates that OpenClaw can handle images correctly. The issue is that this functionality hasn't been extended to:

• Discord/Telegram channel handlers
• /v1/chat/completions endpoint
• Proper imageModel fallback routing

This suggests the fix is primarily about connecting existing working components rather than building new functionality from scratch.

---

Comments

apps/openclaw-barnacle · 2026-03-03

This issue has been automatically marked as stale due to inactivity.
Please add updates or it will be closed.

rstanuwijaya · 2026-03-03

i think still relevant. I cant get any image recognition working on 2026.03.01

marcin-osowski · 2026-03-08

My models hallucinate on every incoming picture. Worse, they don't know about it. It's the same when they do read().

king123-sys · 2026-03-14

Bug confirmed in OpenClaw 2026.3.11. Telegram images are received but NOT passed to image tool. Workaround: manually copy to workspace and call image tool directly.

jimmywongz · 2026-03-18

push!!!

sweetyoungthing · 2026-03-18

The Feishu channel also has the same problem.

agent-eli · 2026-03-24

Reproducing on 2026.3.22 / 2026.3.23-1

Seeing this regression on macOS (arm64) after updating to 2026.3.22. Still present on 2026.3.23-1.

Symptoms:

• Discord inbound images are partially recognized — only the top portion of the image description is visible to the model
• Gateway error log shows:
  • [tools] image failed: Local media file not found: /Users/.../.openclaw/media/inbound/<uuid>.png
    • [tools] image failed: Unsupported media type: document
    • [tools] image failed: No media-understanding provider registered for anthropic

Environment:

Item	Value
OpenClaw Version	2026.3.23-1 (also reproduced on 2026.3.22)
OS	macOS 26.3 (arm64)
Channel	Discord
Models tested	anthropic/claude-sonnet-4-6, openai-codex/gpt-5.4

Notes:

• Was working correctly before 2026.3.22
• Issue is reproducible regardless of model (Claude, GPT)
• The 3.22 release notes include Agents/media replies: migrate the remaining browser, canvas, and nodes snapshot outputs onto details.media — this may be related

tyler6204 · 2026-03-26

Partially addressed by #51373, merge commit: 00e932a. Thanks for the report! Remaining scope: this merged fix only restores prompt ref image hydration for CLI routed inbound turns, so broader multi channel vision pipeline issues here, including non CLI ingestion paths and other channel specific failures, still remain.

apps/openclaw-barnacle · 2026-04-03

This issue has been automatically marked as stale due to inactivity.
Please add updates or it will be closed.

apps/openclaw-barnacle · 2026-04-09

Closing due to inactivity.
If this is still an issue, please retry on the latest OpenClaw release and share updated details.
If you are absolutely sure it still happens on the latest release, open a new issue with fresh repro steps.