Obsidian/X-Articles/Recursive Language Models by MIT, Clearly Explained!.md

title, source, author, published, created, description, tags

title	source	author	published	created	description	tags
Recursive Language Models by MIT, Clearly Explained!	https://x.com/_avichawla/status/2029807609608564917	Unknown	2026-03-06	2026-03-06		clippings

Researchers from MIT recently proposed a clever solution to one of the biggest limitations of modern LLMs: context rot.

Here's what context rot looks like:

You paste a 200-page document into ChatGPT. You ask a simple question. The answer comes back wrong, even though the information is on Page 50.

You didn't exceed the context window. The model just got worse at reasoning because there was too much to process at once.

The elegant fix they proposed is called Recursive Language Models. The results are striking, and the core idea is surprisingly intuitive.

Let's understand it step-by-step.

GIF

The problem

A model can accept 100K+ tokens, but that doesn't mean it can reason well over all of them.

You've probably noticed this yourself:

• Long Claude Code sessions get sluggish
• Extended ChatGPT conversations require more repetition
• The model isn't hallucinating, it's just getting dumber as context grows

Frontier models ace needle-in-a-haystack tests. But ask them to count, classify, or reason over thousands of buried entries, and performance collapses.

The fix: recursive language models (RLMs)

The core idea is simple:

Instead of forcing the model to process everything at once, let it break context into smaller pieces and handle them recursively.

The key shift is context-centric decomposition:

• Agents decompose tasks based on human-designed steps
• RLMs let the model decompose the context itself

The model becomes a programmer analyzing a dataset, not a student cramming for an exam.

How do RLMs work?

1. Separate query from context

In a normal LLM call, you send the query and full context together. In an RLM, the context lives in memory like a Jupyter notebook variable. The model only sees the query, plus access to tools and a REPL environment.

2. The model gets tools

• Peek at context (view first 2,000 chars to understand structure)
• Grep using regex to filter relevant lines
• Partition into smaller chunks
• Call itself recursively on those chunks

3. Strategy emerges from the task

The model decides how to decompose. It might grep first, then partition. Or peek, then summarize. It figures this out on its own rather than following a human-designed workflow.

A concrete example

Say you have 5,000 customer support tickets and you ask: "Among users 12345, 67890, and 11111, how many questions are about billing?"

A normal LLM receives all 5,000 tickets, tries to scan everything, and makes counting errors.

An RLM handles it differently:

1. Peeks at structure: "Each line has Date, User ID, Question."
2. Greps for target users, reducing 5,000 lines to 50
3. Spawns a recursive call: "Classify each as billing or other."
4. Returns: Final Result

The root model's context stayed small throughout. No rot.

Why does it matter?

• No context rot: Accuracy holds regardless of document size
• Unlimited context: 10M tokens? Just partition more
• Interpretable: See exactly what the model did
• Cost-efficient: Smaller calls beat one massive API call
• Future-proof: As LLMs improve, RLMs improve automatically

RLMs treat context as data to be programmatically explored.

The model combines code execution and language reasoning. It's not summarization. It's not a rigid agent. It decides how to decompose the problem based on what it discovers.

To read more about this, I am sharing the research paper and a startup code on RLMs that you can build on top of.

Paper: https://arxiv.org/abs/2512.24601v1

GitHub: https://github.com/alexzhang13/rlm

That's a wrap!

If you enjoyed this tutorial:

Find me → @_avichawla

Every day, I share tutorials and insights on DS, ML, LLMs, and RAGs.