One thing that quietly limits most AI systems today is that most models get slower and more expensive the longer the context grows. Every token adds to what the model has to process. It’s a scaling problem that doesn’t really go away with standard approaches.
That’s worth sitting with for a second. A standard transformer pays attention across everything it’s been given, every time it generates. So a conversation that’s ten times longer isn’t ten times more expensive in a neat, linear way. Attention scales worse than that. Double the context and you don’t double the work, you do more. The window keeps getting bigger and the cost of using it keeps climbing with it. People paper over this with tricks. Sliding windows that throw away old tokens. Retrieval that fetches only the chunks that seem relevant. Both help, but both are working around the problem rather than removing it. The model is still, fundamentally, re-reading.
A different angle
Test-Time Training (TTT-E2E) takes a different angle. Instead of reading everything every time, the model processes input in chunks and actually updates its own weights on the fly. It’s compressing what it learns into its parameters, rather than carrying the whole conversation forward.
This is the part that flips your mental model. Normally weights are frozen at inference. You train the thing, you ship it, and from then on it only reads. TTT breaks that rule. As it moves through the input, it keeps adjusting itself. So instead of hauling the entire conversation along as raw tokens the model has to reprocess, it folds what it has seen into its own parameters. The past stops being a growing pile of text to re-read. It becomes part of the model.
Think of the difference between someone who re-reads the whole book from page one every time they want to answer a question about it, and someone who read it once and remembers. The second person doesn’t get slower as the book gets longer. That’s the shift TTT is reaching for.
Two loops, and the one that matters
There’s a two-loop structure underneath it. One loop handles the real-time learning, chunk, predict and update. The other trains the model to get better at learning from context over time, which is the part I find very interesting, it’s meta-learning, teaching the model how to learn not just what to learn.
The inner loop is the on-the-fly part. Take a chunk, predict, update the weights, move on. That’s the thing running while the model is actually working through your input.
The outer loop is the interesting one. It doesn’t teach the model facts. It teaches the model how to update itself well when it sees new context. That’s meta-learning, and the distinction matters. Most training is about what to learn. Here a chunk of the training is about how to learn. You’re not just filling the model with knowledge, you’re tuning the mechanism it uses to absorb knowledge at test time. Get that mechanism right and the model gets good at teaching itself from whatever context it’s handed, even context it never saw in training.
What you actually get
What this gets you is stable inference cost as context length increases, with accuracy that holds up well against standard transformers. Training becomes more complex, but that’s a fixed cost rather than a recurring one.
Flat inference cost is the headline. The context can grow and the per-step cost doesn’t run away from you, because the model isn’t reprocessing everything, it’s carrying a compressed version in its weights. And it holds accuracy against standard transformers while doing it, which is the part that makes it more than a cost trick. Cheaper is easy if you’re willing to be worse. Cheaper while staying competitive is the harder thing.
Yes, training gets more complex. Two loops is more machinery than one. But that complexity is paid once, up front, during training. It doesn’t come back to bill you on every single inference the way a growing context window does. Fixed cost versus recurring cost. That’s a trade most people running these systems at scale would take.
The broader shift here is framing long-context handling as a learning problem rather than a storage problem.
Link to the paper: https://arxiv.org/abs/2512.23675