--- title: "Qwen3.5-35B-A3B: All You Need is a Single RTX 3090. 112 tok/s at Full 262K Context." source: "https://x.com/sudoingX/status/2028496331992707373" author: - "[[Unknown]]" published: 2026-02-27 created: 2026-03-03 description: tags: - "clippings" --- then the community showed up. 5 days. 15 GPUs. 176 tok/s peak. Qwen dropped a 35 billion parameter model that only activates 3 billion per token. Gated Delta Networks hybrid. 256 experts, 8 routed per token, attention every 4th block. i put it on a single RTX 3090 and hit 112 tokens per second at full 262K context. no speed degradation from 4K to 262K. that's not a typo. the architecture makes context scaling nearly free. then i posted the numbers. 48 hours later the community had submissions from 5090s, 4090s, 3090s, AMD cards. the optimization flags i shared doubled most people's speeds overnight. a model that started at 50 tok/s on default settings was running at 176 on a 5090 by day 5. this article is everything. the hardware, the flags, the builds, the agent harness bugs nobody documented, the community leaderboard, and the full optimization stack. one model. one GPU. 24GB VRAM. zero API costs. ![图像](https://pbs.twimg.com/media/HCaE-_KboAEYDpi?format=jpg&name=large) [https://huggingface.co/Qwen/Qwen3.5-35B-A3B](https://huggingface.co/Qwen/Qwen3.5-35B-A3B) ## THE ARCHITECTURE (why it's different) most 35B models are pure transformers. every layer builds a KV cache that grows with context length. more context = more VRAM = slower generation. that's why most models on 24GB VRAM start choking past 32K context. Qwen3.5-35B-A3B doesn't work that way. only 10 out of 40 layers use traditional attention with KV cache. the other 30 are Gated DeltaNet layers with fixed-size recurrent state. context goes up, memory stays flat across 75% of the model. the result: 112 tok/s at 4K. 113 tok/s at 262K. same speed. the model genuinely does not care how long your context is. VRAM at full 262K context: 18.5GB model + 2.7GB KV cache + 63MB recurrent state = 22.4GB total. 2.1GB spare on a 24GB card. for comparison, a pure transformer at 35B params would need roughly 10GB of KV cache at 262K. that's 30GB+ total. doesn't fit on 24GB. game over. 0:00 / 0:23 ## THE FLAGS (what doubled everyone's speed) most people downloaded the model, ran llama.cpp with defaults, and got 40-70 tok/s. reasonable. but not close to what this model can do. five flags changed everything: \-ngl 99 puts all 41 layers on GPU. default offloads some to CPU. don't. \--cache-type-k q8\_0 --cache-type-v q8\_0 halves your KV cache VRAM. zero speed loss. this is the single most impactful flag. without it, 262K context OOMs on 24GB. \-c 262144 sets the full native context window. default is 2048. you're leaving 99% of the model's capability on the table. \-np 1 drops the recurrent state buffer from 251MB to 63MB. free VRAM. only matters if you're running single user inference, which you are on a local GPU. \-fa enables flash attention. less VRAM, faster prefill. the result: people went from 50 tok/s to 112 overnight. same hardware. same model. different flags. the command that unlocks everything: ```bash \`llama-server -m qwen3.5-35b-a3b-ud-q4_k_xl.gguf -ngl 99 -c 262144 -fa --cache-type-k q8_0 --cache-type-v q8_0 -np 1 --host 0.0.0.0 --port 8080\` ``` > 2月27日 > > the numbers coming in from this thread: 5090: 166 tok/s (z33b0t), 153 tok/s (EmmanuelMr) 4090: 122 tok/s (StubbyTech) 3090: 112 tok/s (sudo), 100 tok/s (Eduardo) 6800XT: 20-30 tok/s (Dark) Qwen3.5-35B-A3B. 4-bit quant, 19.7 GB on disk. fits entirely on any single 3090 24GB card ## THE COMMUNITY (from 50 to 176 tok/s) i posted my 112 tok/s result and the optimization flags. 48 hours later, people started sending their numbers. the leaderboard built itself: - 2x RTX Pro 6000 Max Q: 2,629 tok/s (Richard Hundt) - 5090: 176 tok/s (dabiggmoe), 166 tok/s (z33b0t) - R9700 32GB: 128 tok/s on Vulkan (wild\_zones) - 4090: 122 tok/s (StubbyTech) - 3090: 112 tok/s (sudo), ~100 tok/s (Clemolifts, Eduardo) - 6800XT: 20-30 tok/s ROCm (Dark) submissions used the same flags from my post. some people doubled their speed just by adding --cache-type-k q8\_0. one model, tested across 15 GPUs in 5 days, all community-driven. the 5090 hitting 176 tok/s was the headline. but the real story is the R9700 at 128 tok/s on Vulkan. AMD matching NVIDIA consumer territory with zero optimization. local AI isn't just for people with 4090s anymore. > 3月1日 > > Qwen3.5-35B-A3B on consumer GPUs. the leaderboard keeps growing. 2x RTX Pro 6000 Max Q: 2,629 tok/s (@HundtRichard) 5090: 176 tok/s (@dabiggmoe), 166 tok/s (@z33b0t) R9700 32GB: 128 tok/s Vulkan (@wild\_zones) 4090: 122 tok/s (@StubbyTech) 3090: 112 tok/s (sudo), ~100 tok/s ## the AMD surprise the R9700 hitting 128 tok/s wasn't on ROCm. it was Vulkan. llama.cpp's Vulkan backend, no driver pain, no ROCm setup. just worked. meanwhile the 6800XT on ROCm struggled at 20-30 tok/s. nobody submitted optimized ROCm numbers. everyone on AMD chose Vulkan instead. that tells you something about the state of ROCm. the hardware isn't the bottleneck. the software is. AMD GPUs have the memory bandwidth. the optimization stack just isn't there yet. > 2月27日 > > AMD just entered the chat. R9700 32GB pulling 128 tok/s on Vulkan. stock LM Studio. no flags, no tuning, no llama.cpp from source. Qwen3.5-35B-A3B, 4-bit quant. that's the first AMD submission matching NVIDIA consumer territory. and the AMD side hasn't even started optimizing ## the build (octopus invaders) one prompt. one shot. 3,400 lines of code across 8 modules. ![](https://pbs.twimg.com/amplify_video_thumb/2028466221218398209/img/uWb7z771qPsfwNTE.jpg?name=large) i gave the model a single detailed prompt: build a space shooter game with collision detection, particle systems, procedural audio, and three iteration rounds. build it, test it, fix it, polish it. it scaffolded the project structure, wrote the game loop, built the collision system, spawned particle effects, generated audio from scratch, caught its own bugs, and iterated three times without being asked twice. fully autonomous. no hand holding. the entire build ran on a single RTX 3090 at 112 tok/s. 3 billion active parameters wrote a playable game in one session. this is the part that's hard to explain with benchmarks alone. MATH-500 scores and tok/s numbers don't tell you what it feels like to watch a 3B active model scaffold 8 files and debug its own collision detection. you have to see it. > 3月1日 > > i built a full game on a single GPU with a 3B model and this is the worst local AI will ever be. this was supposed to be a benchmark test. load the model, measure tokens per second, write it up, move on. instead i spent 20 minutes playing Octopus Invaders because the game is ## THE COMPARISON (3B vs 80B) i ran the same task on two models. build a particle simulation. Qwen3-Coder-Next: 80 billion parameters. 2x RTX 3090. 46 tok/s. 564 lines. took 2 iterations to get it right. Qwen3.5-35B-A3B: 3 billion active parameters. 1x RTX 3090. 112 tok/s. 461 lines. first try. 2.4x faster. half the GPUs. fewer lines. cleaner code. one attempt. the smaller model won. not because it's smarter per token. because MoE + speed + long context means more iterations per hour on cheaper hardware. in local inference, the model that ships faster wins. raw intelligence per token matters less when you can run 3x more attempts in the same time. > 2月27日 > > 5 days ago it took 2 GPUs to build this. today it takes 1. same prompt. same particle simulation. completely different model. Qwen-Coder-Next (80B) on 2x 3090s. 46 tok/s. 564 lines. 2 iterations to get it working. 48GB VRAM across two cards just to hold it. Qwen3.5-35B-A3B on a ## the agent problem the model is fast. the model is capable. the agent harness is where it breaks. i tested two setups. same model, same GPU, same localhost endpoint. Claude Code and OpenCode. Claude Code breaks the chain every 3-5 minutes. tool call fails, flow stops, you reprompt. it recovers. 2 minutes later it stops again. the model is fine. the harness chokes on local inference latency. OpenCode doesn't break. the chain runs smooth, fully autonomous. but when the model loses state tracking, it loops. same Read call repeated 10+ times. 3B active params can't always track where it is across identical tool calls. Claude Code's chain break is a crude safeguard. OpenCode's persistence is a feature until it's a bug. the takeaway: local inference is ready. the models are ready. the agent harnesses aren't. the tooling layer between you and the model is now the bottleneck. not the GPU. not the weights. the wrapper. > 3月2日 > > let me save you 3 hours of head scratching. if you're running local models like Qwen3.5-35B-A3B through Claude Code via llama.cpp's Anthropic endpoint, the chain will break every 3 to 5 minutes. tool call fails. flow stops. you reprompt. it recovers. 2 minutes later it stops a used RTX 3090 costs $800. the model is free. the stack is free. no API costs. no subscriptions. no rate limits. that's it. 3 billion active parameters built a 3,400 line game in one session. ran at 112 tok/s at full 262K context. a community of strangers optimized it across 15 GPUs in 5 days without anyone asking them to. and this is the worst local AI will ever be. every number in this article is a floor, not a ceiling.