The cheapest token is the one you never send

When the AI invoice starts climbing, the reflex is to go hunt for a discount: a cheaper model, a volume deal, a better rate. That's the small lever. From the chair of someone who owns the brief and builds through Claude, the bill is mostly a design decision made upstream. It comes down to how many tokens each call carries, and how many of those tokens you'd already paid to process a hundred times over. The cheapest token is the one you never send.

This isn't a fringe concern anymore. Two years ago about a third of teams were actively managing their AI spend. By the FinOps Foundation's 2026 survey it was essentially all of them (98%), ahead of even SaaS and licensing. Token spend is a board-level line item now, so it's worth being precise about which levers actually move it, and in what order.

Don't pay to put your whole corpus in context

Most cost conversations skip past the mechanics. At scale, input tokens dominate the bill, and they scale linearly with whatever you stuff into each call. Send a hundred thousand tokens of context per request across ten thousand requests a month, and you're paying for a billion input tokens before the model has said a word. Double the context you paste in and you double that floor.

Anthropic's own framing calls this context engineering: the goal is the smallest set of high-signal tokens that gets the job done. The largest set that happens to fit is just the lazy default. And this isn't only a cost argument. As the token count climbs, recall accuracy actually degrades, because a bigger window stretches the model's attention thin. Token spend and answer quality end up moving in the same direction, so the right five thousand tokens beat a lazy two hundred thousand on both counts. The instinct to "just give it the whole repo," or "paste in all the docs," or "we have a million-token window, so use it," is the single most expensive habit at scale.

The fix is architectural: retrieve the slice that matters instead of shipping the whole corpus every time.

graphify: the sharpest version of that idea

The cleanest example I've seen of this principle is graphify. Point it at a project (a codebase, docs, PDFs, even video) and instead of leaving you a pile of files to feed the model, it builds a knowledge graph: entities and the typed relationships between them, saved to a graph.json you can query later without re-reading anything.

Two design choices matter for cost specifically.

First, for code the graph is built locally, with no model calls at all. graphify parses the source with tree-sitter, the same AST machinery editors use, across roughly 40 languages. Same graph every run, no network call, nothing sent to a model. You pay zero tokens to build the map of your codebase. That's the part that flips the usual economics. Most graph-retrieval systems spend a fortune having an LLM read everything to construct the index; graphify front-loads that work with a parser instead. (One caveat: non-code inputs like prose docs, PDFs, and images still cost model calls to ingest. The free-build claim is about the code graph, which is exactly the slice most engineering orgs care about.)

Second, you query a sparse slice, not whole files. Ask "what connects auth to the database?" and you get back the relevant subgraph, a handful of nodes and edges, rather than ten files dumped into context. It surfaces god nodes (the hubs everything routes through, so an agent can orient on structure instead of reading the whole thing) and clusters related entities into communities. Every edge is typed and confidence-tagged, whether found in the source, inferred, or flagged ambiguous, so you know what's real and what's a guess.

What it really does is turn a repeated whole-repo read into a small lookup. The bigger the corpus, the bigger the win. Honestly, though, the win is close to nothing when the corpus already fits in a context window. Six files don't need a graph. A million lines of legacy code queried a thousand times absolutely do.

The rest of the stack compounds

Retrieval is the structural lever. These are the ones that stack on top of it, roughly in order of effort to payoff:

Cache the parts that repeat. With prompt caching, reads cost about a tenth of the normal input price, and you're ahead by the second call that shares the same prefix. On a large, stable prompt, the cached portion runs up to ~90% off. It matches on an exact prefix and nothing else, so a timestamp, a session ID, or a per-request token near the top of your prompt silently voids everything after it. "We turned caching on" is not the same as "we're hitting cache." Put the stable content first and the volatile content last, then actually check the cache-read counter on your responses.

Batch anything a human isn't waiting on. The Batch API is a flat 50% off, same model and same quality, in exchange for a queue that usually clears in under an hour. Nightly enrichment, backfills, evals, bulk classification and extraction: none of that needs to be real-time. It also stacks with caching, which pushes the effective input cost on a cache hit down toward a twentieth of standard synchronous pricing. The rule I use: if a person is waiting on it, keep it live; if a machine consumes it on a schedule, batch it.

Right-size the model, and tune how hard it thinks. Haiku is 5× cheaper than Opus per token, so route routine classification, retrieval, and lookups down the ladder and reserve the frontier model for the genuinely hard reasoning. But Anthropic's own guidance is that tuning effort is often a better lever than switching models: dialing reasoning depth down on the easy steps. Opus at medium effort matched a prior Sonnet's coding score while using 76% fewer output tokens, and output runs 5× the price of input, so thinking less on the easy work is real money.

For agents, the bill is mostly re-sent history. Every step of an agent loop resends the growing transcript, so a long run can burn 10 to 100 times a single call. Context editing (dropping stale tool output) and compaction (summarizing old turns) claw that back hard. Anthropic measured an 84% token cut on a 100-turn task, with quality going up, not down. Keeping rarely-used tool definitions out of context until they're needed took one 50-tool setup from about 77K startup tokens to under 9K.

Govern it so it can't surprise you. Per-workspace spend caps and threshold alerts turn an open tab into a bounded envelope. The usage-and-cost API breaks spend down by workspace, model, and (the part people miss) cached versus uncached tokens, so you can actually see whether your caching is working instead of hoping. And count tokens with the real endpoint, not a generic tokenizer, or your cost math is wrong before you even start.

Where "optimization" quietly backfires

Every lever above can degrade quality silently if you pull it without watching, so none of them ship without an eval to catch the regression.

The point from the chair

The invoice is set upstream, in the architecture: what each call carries, and how much of that you've already paid to process before. That's a design decision, and it belongs in the brief, not in a panicked renegotiation after the invoice lands. Start with the token you never send, and retrieve the slice instead of shipping the corpus. After that it's housekeeping: cache the repeats, batch anything nobody's waiting on, drop routine work to Haiku, and cap the rest.

I've folded the practical version of this into the adoption kit. The executive one-pager frames the cost levers for a sponsor, and the governance brief covers the spend caps, attribution, and usage reporting that keep it controllable. The technology rewards the teams that design their bill on purpose, and quietly taxes the ones who wait for the invoice to tell them what it is.