coco
119 lines
4.7 KiB
Markdown
119 lines
4.7 KiB
Markdown
---
|
|
title: "Recursive Language Models by MIT, Clearly Explained!"
|
|
source: "https://x.com/_avichawla/status/2029807609608564917"
|
|
author:
|
|
- "[[Unknown]]"
|
|
published: 2026-03-06
|
|
created: 2026-03-06
|
|
description:
|
|
tags:
|
|
- "clippings"
|
|
---
|
|
Researchers from MIT recently proposed a clever solution to one of the biggest limitations of modern LLMs: context rot.
|
|
|
|
Here's what context rot looks like:
|
|
|
|
You paste a 200-page document into ChatGPT. You ask a simple question. The answer comes back wrong, even though the information is on Page 50.
|
|
|
|
|
|
|
|
You didn't exceed the context window. The model just got worse at reasoning because there was too much to process at once.
|
|
|
|
The elegant fix they proposed is called Recursive Language Models. The results are striking, and the core idea is surprisingly intuitive.
|
|
|
|
Let's understand it step-by-step.
|
|
|
|
<video preload="auto" tabindex="-1" playsinline="" aria-label="嵌入式视频" poster="https://pbs.twimg.com/tweet_video_thumb/HCtKkc0aUAczixd.jpg" src="https://video.twimg.com/tweet_video/HCtKkc0aUAczixd.mp4" type="video/mp4" style="width: 100%; height: 100%; position: absolute; background-color: black; top: 0%; left: 0%; transform: rotate(0deg) scale(1.005);"></video>
|
|
|
|
GIF
|
|
|
|
## The problem
|
|
|
|
A model can accept 100K+ tokens, but that doesn't mean it can reason well over all of them.
|
|
|
|
You've probably noticed this yourself:
|
|
|
|
- Long Claude Code sessions get sluggish
|
|
- Extended ChatGPT conversations require more repetition
|
|
- The model isn't hallucinating, it's just getting dumber as context grows
|
|
|
|
Frontier models ace needle-in-a-haystack tests. But ask them to count, classify, or reason over thousands of buried entries, and performance collapses.
|
|
|
|
## The fix: recursive language models (RLMs)
|
|
|
|
The core idea is simple:
|
|
|
|
Instead of forcing the model to process everything at once, let it break context into smaller pieces and handle them recursively.
|
|
|
|
The key shift is **context-centric** decomposition:
|
|
|
|
- **Agents** decompose tasks based on human-designed steps
|
|
- **RLMs** let the model decompose the context itself
|
|
|
|
The model becomes a programmer analyzing a dataset, not a student cramming for an exam.
|
|
|
|
|
|
|
|
## How do RLMs work?
|
|
|
|
**1\. Separate query from context**
|
|
|
|
In a normal LLM call, you send the query and full context together. In an RLM, the context lives in memory like a Jupyter notebook variable. The model only sees the query, plus access to tools and a REPL environment.
|
|
|
|
**2\. The model gets tools**
|
|
|
|
- **Peek** at context (view first 2,000 chars to understand structure)
|
|
- **Grep** using regex to filter relevant lines
|
|
- **Partition** into smaller chunks
|
|
- **Call itself recursively** on those chunks
|
|
|
|
**3\. Strategy emerges from the task**
|
|
|
|
The model decides how to decompose. It might grep first, then partition. Or peek, then summarize. It figures this out on its own rather than following a human-designed workflow.
|
|
|
|
|
|
|
|
## A concrete example
|
|
|
|
Say you have 5,000 customer support tickets and you ask: "Among users 12345, 67890, and 11111, how many questions are about billing?"
|
|
|
|
A normal LLM receives all 5,000 tickets, tries to scan everything, and makes counting errors.
|
|
|
|
An RLM handles it differently:
|
|
|
|
|
|
|
|
1. **Peeks** at structure: "Each line has Date, User ID, Question."
|
|
2. **Greps** for target users, reducing 5,000 lines to 50
|
|
3. **Spawns a recursive call:** "Classify each as billing or other."
|
|
4. **Returns:** Final Result
|
|
|
|
The root model's context stayed small throughout. **No rot**.
|
|
|
|
## Why does it matter?
|
|
|
|
|
|
|
|
- **No context rot:** Accuracy holds regardless of document size
|
|
- **Unlimited context:** 10M tokens? Just partition more
|
|
- **Interpretable:** See exactly what the model did
|
|
- **Cost-efficient:** Smaller calls beat one massive API call
|
|
- **Future-proof:** As LLMs improve, RLMs improve automatically
|
|
|
|
RLMs treat context as **data to be programmatically explored**.
|
|
|
|
The model combines code execution and language reasoning. It's not summarization. It's not a rigid agent. It decides how to decompose the problem based on what it discovers.
|
|
|
|
To read more about this, I am sharing the research paper and a startup code on RLMs that you can build on top of.
|
|
|
|
Paper: [https://arxiv.org/abs/2512.24601v1](https://arxiv.org/abs/2512.24601v1)
|
|
|
|
GitHub: [https://github.com/alexzhang13/rlm](https://github.com/alexzhang13/rlm)
|
|
|
|
That's a wrap!
|
|
|
|
If you enjoyed this tutorial:
|
|
|
|
Find me → [@\_avichawla](https://x.com/@_avichawla)
|
|
|
|
Every day, I share tutorials and insights on DS, ML, LLMs, and RAGs. |