This week I came across a debugging thread in a technical group that I haven’t stopped thinking about. A production model, mid-conversation, out of nowhere, started printing tokens like [control_28] into its output. Not once. Repeatedly, as if it had gotten stuck.
The instinct is to blame the model. Bad weights, bad prompt, hallucination, whatever word people reach for when a language model does something inexplicable. But this one wasn’t a model problem. It’s a known failure mode called control-token leakage, and it lives in the infrastructure around the model, not in the model’s judgment.
What’s actually happening
Every LLM serving stack has a layer of special tokens the model was trained to use for structure, not conversation. Chat markers that open and close a turn. Padding tokens. Reserved slots a model provider left open for future use, things like [control_28], which are never meant to reach a user’s screen. In normal operation, the serving stack strips these out before anything gets displayed, and the tokenizer maps text to IDs exactly the way the model expects.
Leakage happens when that mapping breaks, and it breaks in a few familiar ways. Sometimes it’s a tokenizer mismatch: the model gets converted from one framework to another, say from a Hugging Face checkpoint to a quantized GGUF file for local serving, and the tokenizer file that ships alongside it doesn’t quite match the vocabulary baked into the model’s embedding table. The IDs shift by a handful of positions, and the model reads token 6028 as something completely different from what it was trained on. Sometimes it’s quantization: compressing weights down to 4-bit or 8-bit can corrupt the embedding rows for tokens sitting in the reserved or rarely used range, because those rows got almost no training signal to begin with and are the first to degrade under aggressive rounding. And sometimes it’s just corrupted metadata: a config file defining vocab size, special token IDs, or the chat template drifts out of sync with the actual weights, usually after a manual conversion step or an export that didn’t quite finish.
None of that is a story about the model misunderstanding the conversation. It’s plumbing.
Why it can’t just apologize its way out
The part of that debugging thread that stuck with me: once this starts, the model can’t recover on its own, even when it clearly tries.
You’d expect it to notice the garbage, apologize, and move on cleanly. Sometimes it does exactly that. “Sorry, let me try that again.” But by then the damage is done. Those broken tokens are already sitting in the context window, already baked into the KV cache the model uses to generate every token after them. A transformer doesn’t get to un-see its own context. It conditions the next token on everything that came before it, including its own garbage. So the model apologizes, and then, still conditioned on that same corrupted state, falls straight back into the same pattern. It’s less a mistake being corrected and more a scratch in a record repeating the same few seconds of the song.
The actual lesson
It’s tempting, when picking an AI stack, to spend all your attention on model selection: which one reasons best, which one is cheapest, which one has the longest context. That’s a real decision and it matters. But this bug is a reminder that two teams can run the exact same model and get completely different reliability, because one of them has a serving stack that maps tokens correctly and the other doesn’t.
The model is the part everyone talks about. The tokenizer, the conversion pipeline, the quantization step, the metadata file nobody double-checks after an export, that’s the part that actually decides whether your production system holds up. Get the infrastructure wrong and the best model in the world will still start typing [control_28] at your users.