The hidden cost of context
Every conversation about agent economics focuses on output tokens, which are the visible half of a bill dominated by the invisible half. Context is the line item nobody budgets and almost everybody overpays.
Ask an engineering leader what their agents cost and you will get a number derived from a provider invoice. Ask what drives that number and the answer is usually about output: how much code is being generated, by how many people, how often.
The invoice tells a different story. On most real workloads, input tokens dominate, frequently by an order of magnitude. The bill is not mostly paid for what the agent produced. It is paid for what the agent was told before it produced anything.
This has consequences that go well past cost accounting, because the same thing that drives the bill also drives quality, and the two move in opposite directions from where most people assume.
Where the tokens actually are
Consider one ordinary task: fix a bug in a payments module. The agent reads the failing test, several source files, some configuration, the schema, and a couple of related modules to understand the interfaces. It produces forty lines of code.
Now add the property that makes this compound: in a multi-turn task, the conversation carries forward. Turn two resends turn one. Turn ten resends everything. A task that takes fifteen turns does not pay the context cost once, it pays a growing fraction of it fifteen times.
This is why long agent sessions have a cost curve that surprises people. The relationship between task duration and cost is not linear. It is closer to quadratic, and the tasks that run long are, unhelpfully, the ones that were already difficult.
The counterintuitive part
Here is where the naive optimisation goes wrong, and it goes wrong in a way that costs more than doing nothing.
If input tokens dominate, the obvious move is to send less. Cut the context, restrict the files, truncate the history. The invoice responds immediately and gratifyingly.
Then the quality drops. The agent, lacking the interface definition, invents one. Lacking the neighbouring module, it duplicates logic that already exists. Lacking the schema, it guesses a column name. Each of these produces a change that fails verification, which triggers another attempt, which costs another full context load. Or worse, it produces a change that passes verification and is wrong, and you pay in review time and eventually in an incident.
Starving the context
- Per-call cost falls, visibly
- Agent invents what it cannot see
- More failed attempts per task
- Each retry pays full context again
- Total cost per completed ticket rises
Curating the context
- Per-call cost falls, less dramatically
- What is sent is what is relevant
- Fewer attempts, higher first-pass rate
- Cache absorbs the stable portion
- Total cost per completed ticket falls
The unit that matters is cost per completed, verified, merged change. Cost per call is a metric that can be improved indefinitely while the real number gets worse, and it is the metric most dashboards show.
Four levers that actually work
Retrieval precision. The difference between sending twenty relevant files and two hundred plausible ones is roughly a factor of ten on the dominant term, and the twenty-file version usually produces better output because the signal is not diluted. This is the highest-value investment available and it is an engineering problem rather than a purchasing one: better indexing, dependency-aware selection, respecting module boundaries.
Caching the stable portion. Most of what gets sent repeatedly does not change: the system instructions, the architecture notes, the conventions, the core interfaces. Providers offer mechanisms to cache this at a substantial discount. Taking advantage requires structuring requests so the stable part is a stable prefix, which is a small engineering change that most teams have not made because nobody looked at the invoice closely enough to care.
Task decomposition. Since cost grows super-linearly with conversation length, three well-scoped tasks are cheaper than one sprawling one, often considerably. They also verify better, because a small change has criteria you can actually write. This is the lever with the best secondary effects and the one that depends most on the quality of your ticket writing.
Compaction. For genuinely long tasks, summarising earlier turns rather than resending them. Useful, and the one to be most careful with: a summary that drops the constraint that mattered produces an agent that confidently violates something it was told forty minutes ago.
Why this is a documentation argument
An implication that surprises people: the strongest cost lever in the medium term is the quality of your written material, and not for the reason you would guess.
An agent that has a clear architecture document and accurate interface documentation does not need to read fifteen files to work out how a subsystem fits together. It reads one page. The page is a few thousand tokens; the fifteen files are fifty thousand, and the page is more accurate because it states intent rather than requiring intent to be inferred from implementation.
The same holds for the codebase itself. Consistent structure means fewer files needed to establish a pattern. Clear module boundaries mean retrieval can be scoped without losing something important. Dead code that nobody has deleted is dead code an agent reads, pays for, and is occasionally misled by.
None of this is new advice. What is new is that it now has a number attached, which changes how the argument goes with a finance-minded director. “Deleting the abandoned v1 client reduces our context load on every payments task” is a better sentence than “it would be tidier.”
Where this breaks down
Retrieval precision has a floor and overshooting it is expensive. Scope too narrowly and the agent misses the caller that also needed changing. That failure is worse than an inflated bill because it produces a change that looks complete and is not. Precision work needs a quality metric next to it, not just a token count.
Caching only helps genuinely repeated prefixes. If every task assembles context in a different order, nothing caches. Getting the benefit requires deliberate structuring, and teams frequently implement caching, see no discount, and conclude the feature does not work.
Decomposition transfers cost to humans. Three well-scoped tickets require someone to scope them. That person is expensive and is usually your constraint. There is a real point past which finer decomposition costs more in human time than it saves in tokens, and it arrives sooner than the arithmetic suggests.
Provider pricing moves. Input costs have fallen substantially and may keep falling, which would make some of this optimisation not worth the engineering. The structural point survives, since the input to output ratio is a property of the work rather than of pricing, but the urgency is genuinely uncertain and anyone claiming otherwise is guessing.
Attribution to a unit of work is hard. Everything here assumes you can compute cost per completed ticket, which requires joining provider usage to source control to your tracker. Most organisations cannot do this today, which is why the conversation defaults to the invoice total.
The takeaway
The bill is dominated by input, not output. That single fact reorders the entire optimisation problem, and most teams are working on the smaller half.
Do not starve the context. Curate it. Precise retrieval, a cacheable stable prefix, and smaller tasks all reduce cost per completed change, which is the only unit worth measuring. Cutting context reduces cost per call and raises cost per outcome, and it will look like a win on the dashboard for about a month.
If you take one thing into next week: look at your provider usage and check the ratio of input to output tokens. If you have never looked, the number will be larger than you expect, and it will tell you where your money is going with more clarity than any other single measurement available.
The next piece takes the other half of the economics: choosing which model to use, treated as a cost and reliability decision rather than as a matter of taste.