I obtained a very fascinating query final week from Mike Loukides, my editor at Radar, after he learn the third a part of this trilogy on context administration. “One other situation I’ve examine,” Mike requested, “is the tendency for a mannequin to disregard the center of the context. I’ve seen that significantly for the fashions with very giant context home windows. Is there something to be stated about that?”
Wonderful query, Mike, and sure, there’s. In that very same e mail he identified that clearing the context and reloading it with simply what’s vital does a reasonably good job coping with this “ignore the center” downside when it occurs, however that’s clearly a stopgap.
It’s price a deeper dive into what’s truly occurring when an AI begins forgetting what’s in the midst of its context, as a result of the issue is deeper (and extra fascinating!) than it may appear at first. It seems that there’s a primary downside that’s elementary to how LLMs handle context, and we’re nonetheless studying about it as an trade. That downside is known as a U-shape. There’s been lots of actually fascinating analysis into the U-shape downside just lately, and a number of other helpful methods have emerged that may enable you to handle it. And it’s in all probability not a coincidence that I’ve had to make use of all of them in my ongoing experiments with AI-driven growth and agentic engineering (even when I didn’t at all times notice that’s what I used to be doing on the time).
Just a few weeks in the past, in truth, I bumped into the precise failure mode that Mike described. I used to be operating the High quality Playbook, my open supply code high quality engineering talent, and bumped into hassle with one in all its phases—the one which writes up the bugs the sooner phases discover. There’s part of the bug writeup course of the place it had simply created a file referred to as BUGS.md that had an summary of every of the bugs, and needed to create particular person writeups for every bug it discovered. However as a substitute of filling within the particulars accurately, it produced skeletal-looking stub recordsdata, with a generic template that had clean values as a substitute of populated ones.
The factor is, the directions for write a populated writeup had been within the immediate. The precise bug knowledge was in BUGS.md. I used to be completely sure that all the things the agent wanted was sitting in its context window, as a result of I may see that it hadn’t compacted but, and the talent’s intermediate artifacts let me see that earlier phases had learn and reasoned about each recordsdata (which I talked about in my final article on this sequence). However the agent was producing stubs anyway. It actually regarded just like the agent had all the things it wanted sitting in plain sight, and simply wasn’t utilizing the knowledge it had. Irritating!
I believed on the time that the mannequin was simply an fool (which, arguably, was true however irrelevant). It seems that I had run straight into the U-shaped context downside.
Within the earlier three articles I lined what context is and why it disappears, hold vital data in recordsdata as a substitute of leaving it within the agent’s context window, and detect and get better when context has been compacted out from underneath you. All three had been about dropping context, by means of fragmentation, by means of compaction, by means of lengthy periods that overrun the window. This text is about this fully completely different U-shaped failure mode, the place the context remains to be sitting within the window and the mannequin simply isn’t utilizing it.
The U-shape failure, and why greater home windows don’t repair it
The U-shape is an lively space of educational investigation, so I’m going to start out by going into slightly little bit of that analysis, as a result of I feel it is going to truly assist us pin down what’s occurring. I’ll begin with an experiment run by Nelson Liu, an AI researcher at Stanford, who examined how language fashions truly use the contents of lengthy inputs by giving them paperwork with the related reply positioned at completely different positions and measuring whether or not the mannequin may nonetheless discover it. An fascinating factor his findings present is that the U-shape didn’t seem like a quirk of a single mannequin. The U-shape confirmed up throughout mannequin households, and even fashions with bigger context home windows nonetheless exhibited it.
If in case you have time, it’s truly price looking on the paper that Liu and his crew wrote, referred to as “Misplaced within the Center: How Language Fashions Use Lengthy Contexts.” (It’s surprisingly readable for a tutorial paper.) The end result they reported was a sturdy U-shape: The mannequin carried out finest when the related data was initially of its context window or on the current finish and worst when it was within the center. Efficiency on questions the place the reply was buried mid-context fell off sharply, even when the reply was sitting proper there in plain sight. The sphere now makes use of the phrases primacy bias and recency bias for these two preferences, and the U-shape is what you get while you plot them collectively in opposition to place.
I’m going to lean slightly into academia right here, as a result of lots of researchers are nonetheless studying about how LLM context truly works and what habits has emerged in it.
One motive the U-shape issues greater than “simply one other LLM quirk” is that current analysis has began displaying it’s a structural property of how transformers work, not a discovered artifact. A 2025 ICML paper referred to as “On the Emergence of Place Bias in Transformers” defined it because the equilibrium between two opposing forces contained in the mannequin: The causal masks amplifies the affect of the primary few tokens (the primacy bias), whereas place encodings like RoPE closely weight the tokens closest to the place the mannequin is producing (the recency bias). The center is the place these two forces cancel out. A 2026 paper by Borun Chowdhury, a researcher at Meta, referred to as “Misplaced within the Center at Start: An Actual Principle of Transformer Place Bias,” took the argument even additional by proving mathematically that the U-shape exists in the mean time of initialization, earlier than any coaching has occurred, with random weights.
That issues as a result of the pure assumption about giant context home windows is that extra room means fewer issues. Most of immediately’s frontier fashions provide you with one million tokens or extra, with some pushing effectively previous two million, and a few have made actual progress on the best model of the lost-in-the-middle take a look at, the needle-in-a-haystack benchmark, the place the mannequin has to retrieve a single sentence buried in an extended doc. Google’s Gemini 1.5 Professional reported near-perfect single-needle recall at 1M tokens, and present Gemini 3 fashions are comparable.
So the correct model of “greater home windows don’t repair it” is that this: Greater home windows have made easy single-fact retrieval significantly better. They haven’t made long-context agent work dependable by default. A two-million-token window means an even bigger center to fall into.
The vital concept that’s rising right here is that it’s more and more wanting just like the U-shape isn’t only a bug in immediately’s fashions that may finally be labored out or skilled away by extra knowledge or higher fine-tuning. As an alternative, it looks as if the U-shape may very well be a geometrical property of the LLM structure itself.
In different phrases, we’re all going to need to take care of the U-shape. And meaning we want methods for managing it, and any efficient method we use isn’t prone to change into out of date any time quickly. And that’s my objective on this article: to point out you the methods which have emerged for managing U-shaped context reminiscence loss that you should use immediately in your personal work.
5 methods to assist with U-shaped context issues
The earlier article on this sequence laid out a sample for detecting and recovering from context loss, which I referred to as externalize-recognize-rehydrate. The methods beneath lengthen the identical self-discipline to the lost-in-the-middle downside. The precept I hold coming again to is that working reminiscence is untrustworthy, and the self-discipline that follows from it’s to externalize what issues, curate what stays in context, and confirm what the agent claims to know in opposition to what’s on disk. The 5 methods are how I do this in observe, and each is drawn from an actual second within the High quality Playbook’s growth.
Curate, don’t accumulate
That is the method which, in its most brute-force type, is strictly what Mike talked about in his e mail to me: simply clear the context and reload it with simply what issues, periodically and intentionally. In different phrases, don’t belief an amassed session to remain coherent; construct the artifact, then begin recent in opposition to it. And if in case you have the AI write down the vital elements of the context (like we’ve talked about all through this sequence), then you can begin a brand new session with refreshed AI that has a extra focused, curated context as a place to begin.
I bumped into this through the v1.5.2 launch prep for the High quality Playbook. I used to be utilizing an extended Claude Code session that had been working by means of a sequence of fixes. However I observed that it was simply beginning to present its age: It had forgotten a few issues it ought to know, and its considering instances had been beginning to develop.
When it got here time to land the ultimate 4 fixes for the discharge, I labored with the AI to jot down a context temporary, or a separate doc with all the things the implementing session wanted. The query was whether or not to maintain utilizing the prevailing session, which already “knew” the codebase from the sooner work, or open a recent CLI session and level it on the temporary. I requested one other session what to do:
Ought to we run that in a brand new cli session relatively than proceed my presentclaude code session that has the prevailing context?
The AI gave me a great reply—begin a recent session, utilizing a beginning immediate to learn the temporary—and it gave three causes which have caught with me. First, the temporary was self-contained, together with file paths, line numbers, actual diffs, regression take a look at our bodies, and preflight greps. Something the brand new session wanted to know was already there, and persevering with context purchased nothing. Second, recent context is stricter about adherence. A session that already “is aware of” the codebase tends to skim the brand new directions and improvise from prior assumptions. Surgical fixes are precisely the case the place you need the agent to learn the temporary rigorously relatively than depend on reminiscence of what felt proper final spherical. And third, the audit path: The temporary is the artifact, and the implementing session is reproducible from simply the temporary. If the identical work needs to be redone in six months by a special mannequin, you level on the temporary and say, “That is the enter.”
The strategy labored very well. I used to be in a position to choose up growth seamlessly, and the mannequin’s reminiscence issues disappeared.
Place important data on the edges
The U-shape says the mannequin attends finest to the start and finish of its context. The pure transfer is to place your most load-bearing data in these positions and hold the center for belongings you don’t want the mannequin to concentrate on. Something vital that lives solely in the midst of an amassed context tends to slip out of consideration.
The opposite facet of this system is what not to place within the center. If one thing issues, don’t bury it in an extended preamble of context you’ve been accumulating; transfer it to the sides, restate it the place the mannequin will act on it, and let the center take in the much less vital materials. Fortunately, there’s a helpful method that may assist with this downside.
In Claude Code, for instance, one actually clear approach to put data initially of context is to make use of the system immediate. The CLI offers you --append-system-prompt for precisely this. (Many of the different suppliers’ CLI instruments have comparable choices.) When you put your temporary (or chosen elements of it) there, the agent will attend to it strongly all through the session, and that in flip will assist hold the per-turn person immediate targeted on the motion you need the agent to take proper now.
Quick periods over lengthy ones
Don’t run one lengthy session. Run many quick ones, every studying recent from disk. It will enable you to iterate in your temporary and your exterior growth context, so as a substitute of counting on an opaque context window, you will have a visual and consistently altering set of paperwork that provide you with much more visibility into—and management over—your AI’s context.
One thing helpful I began doing was taking all my chat historical past from Gemini, ChatGPT, Claude, and Cowork and placing it right into a single folder I may hold up to date and listed for quick search. I constructed out a whole system to handle this, which seems to be an incredible device after I’m writing articles like this, as a result of I can search by means of my growth historical past for particular examples and methods that I’ve used. The system makes use of Haiku 4.5 to learn by means of chat historical past, summarize what occurred, and create an index. Haiku turned out to be a sensible sufficient mannequin to learn every particular person interplay in a chat and write a helpful index entry for it. However the mannequin being good sufficient to do one abstract didn’t imply its context administration may sustain throughout all 18,000 information. I ran smack into the U-shape downside.
The primary try tried to maintain dedupe state and progress counts within the mannequin’s head, and it failed spectacularly. The mannequin actually didn’t need to hold monitor of particular deterministic issues like correct numbers or the present state. Haiku 4.5, specifically, appears particularly dangerous at this. What labored was reframing the structure fully. Right here’s the precise immediate that I gave it to repair the issue:
okay, so we want context administration. it would not want to recollect issues,it simply wants to jot down them down as they go. we had this similar contextadministration downside with High quality Playbook, when it was operating out ofcontext. Simply write down after every message.
The protocol I greenlit for the total run made the short-session self-discipline express:
- Resume processing from the cursor recorded in progress.json, working by means of every enter file so as.
- Replace progress.json after each line.
- Count on to expire of context effectively earlier than ending—that’s high quality. Simply cease cleanly after every step (or a gaggle of steps), then spin up a recent session that reads progress.json and continues.
- When all recordsdata are full, set standing: “full” in progress.json and report again.
Merchandise 3 is the method in a single line: count on context loss, so be sure you’ve written your state down, and construct recent restarts into the method. The technical particulars, like spinning up subagents, orchestrating with script, and many others., will change, however the core concept stays the identical. In lots of methods, you’ll be able to consider treating the agent like a pipe, not a database. The state lives on disk, and the session is one thing you throw away and substitute.
Restate key information near the purpose of use
When the mannequin wants a constraint to use proper now, repeat it proper now. Don’t belief an instruction from earlier within the session to hold ahead by means of the center of the context.
That is the method that mounted the issue I opened the article with, the place the High quality Playbook appeared to overlook all the things it had simply written right into a file referred to as BUGS.md and produced stubs when it wanted to jot down the identical data into extra detailed recordsdata, and as a substitute writing generic clean templates with the bug-specific fields left clean.
The repair was to restate the read-the-source rule proper earlier than the motion that wanted it, utilizing this immediate:
Earlier than writing BUG-NNN.md, re-read the BUG-NNN entry in BUGS.md.Copy the Spec foundation, Minimal replica, Location, Anticipated habits,Precise habits, Regression take a look at identify, and Patches fieldsfrom that entry into the writeup. Don't paraphrase from reminiscence.
“Don’t paraphrase from reminiscence” is the road that did the precise work. The instruction couldn’t belief the agent’s reminiscence of what BUGS.md stated, although BUGS.md was sitting proper there within the context window. So the instruction pressured a recent learn of the file in the mean time of writing. The restatement and the fresh-read collectively mounted the bug.
The identical sample applies any time a rule was acknowledged earlier within the session and the mannequin must act on it now. Restate the rule subsequent to the motion, and pressure the mannequin again to the supply relatively than letting it work from reminiscence.
Take a look at the center
The earlier 4 methods are about avoiding lost-in-the-middle failures. This one is about catching them. When you don’t know whether or not the agent is definitely utilizing the knowledge you assume it’s utilizing, discover out, with a deterministic test relatively than a judgment name.
The sample is the one I used within the Haiku summarizer that I described earlier: evaluate what the agent claims to know in opposition to what’s on disk. You could have one thing the agent claims to know (its progress, its present state, the newest model of a rule), and you’ve got one thing on disk that’s the bottom reality (a file, a log, a database file). In the mean time the agent’s declare needs to be trusted, you test it.
Within the summarizer’s resume protocol, each new session began by cross-checking progress.json in opposition to the precise final line written to the abstract file, and the agent printed a checkpoint report when it did—at session begin, and periodically by means of the run. A consultant one regarded like this:
Checkpoint Report:
✓ progress.json confirmed: cursor for cowork_04_06 is at 238, standing is"operating"
✓ Disk state verified: Final line in summaries/cowork_04_06.md is [237]assistant: Device invocation repeating chat file learn.
⚠ Discrepancy famous: The prior session left a bulk notice claiming information238–296 are duplicates however did not write particular person traces for them. Peryour directions, I need to write one line per file, even for duplicates,within the format [idx]: Duplicate of file [X] ( ).
Standing: Cursor matches disk state. Able to resume from file 238.
The agent doesn’t must introspect whether or not it misplaced context, solely to check two recordsdata. Once they agree, the agent proceeds; once they disagree, the agent flags the discrepancy and stops earlier than including any new work on prime of a damaged state. Disagreement is the sign.
You possibly can construct this sort of test into any agent that does multistep work. Choose one thing the agent has to trace, choose the file that’s the supply of reality for it, and have the agent evaluate the 2 at each session begin. When the agent’s view of the world drifts from the file, you discover out earlier than the drift turns into a buried bug.
The self-discipline behind these methods
Once I constructed the High quality Playbook’s multi-phase structure, I used to be fixing the compaction downside. Lengthy pipeline runs had been filling the context window and triggering silent compaction in the midst of work. Breaking the pipeline into separate phases that learn recent from disk and stopped after every section mounted it.
What I didn’t notice till later was that the identical structure additionally helps with the lost-in-the-middle downside. Every section has its personal quick, targeted context, with the section temporary initially and the newest progress replace on the finish, so there’s virtually no center for data to fall into. The architectural transfer that helped with working reminiscence disappearing seems to additionally assist with working reminiscence being there and unused.
That’s the lesson I need to land. Each failure modes, context loss and lost-in-the-middle, are issues of working-memory unreliability, and the self-discipline that addresses them is identical: hold the working set small, put the load-bearing data on the edges of the window, and test the agent’s claims in opposition to floor reality on disk when it issues.
Context home windows will hold getting greater, and compaction will get smarter. Among the methods in these 4 articles might finally be pointless. However the underlying constraint received’t disappear. In spite of everything, we’ve added much more RAM to our computer systems because the 1MB 286 I wrote about within the final article, and reminiscence administration has gotten rather more advanced since then. And plenty of of those issues are structural; for instance, it’s more and more wanting just like the U-shape itself is a geometrical property of the transformer structure, not a coaching artifact that extra compute will clean out.
The underside line is that in case your agent’s capability to do its job is determined by data, that data must reside someplace extra sturdy than working reminiscence. That was true for my dad’s 32 kilobytes of core reminiscence at Princeton within the Nineteen Seventies, it was true for my 640 kilobytes of standard RAM on my 286 within the Eighties, it was true for the 200K-token home windows in final yr’s fashions, and it will likely be true for no matter comes subsequent.
