mailwal

AI Context Windows: Comparing the Cost of an Accepted Coding Result

Markus Eisele

The patterns behind modern Java. From Quarkus to AI inside the JVM
Write boring code. Ship exciting systems.

AI Context Windows: Comparing the Cost of an Accepted Coding Result

How repeated input, caching, compaction, and retries affect the economics of coding agents beyond the advertised token price.

I see two things coming up regularly in our internal IBM Bob Slack channel. People want a larger context window, and people are surprised how expensive their conversations become. Well, both make sense when you look at what the agent sends to the model with each request.

In this episode, I explain what a context window contains and how IBM Bob, Codex, Claude Code, and Gemini CLI decide what goes into it. We follow twenty model requests, starting at 10,000 input tokens and adding another 10,000 each time. The last request contains 200,000 tokens, but the total across those requests is 2.1 million. At the standard GPT-6.1 Sol rate used in the episode, that is $4.20 for input alone, with caching excluded and output or tool charges added separately. Then we look at how caching and compaction change the calculation.

I also share guidance from the Book of Bob, my collection of agentic development best practices on an internal IBM website. Unfortunately, I haven’t been able to open source it so far. We go through standing context, tool descriptions, focused task briefs, and short handover notes, using a Java email-validation example to make the advice concrete.

And, of course, there is the business model. The labs sell tokens, but serving those tokens costs them money too. I consider those incentives alongside the cost of reaching a result I can accept, including retries and review time. A larger window can help with a change across several modules; the capacity number alone doesn’t tell us what the task will cost or how reliably the model will connect the details.

Model limits and API prices are a snapshot from October 14, 2026. The Codex defaults mentioned came from my local installation and can differ across accounts and sessions.

The episode is narrated locally with Fish S2 Pro and my trained LoRA voice adapter.

Chapters

  • 00:00 — Intro

  • 00:12 — Wanting more context and wondering about the bill

  • 01:52 — What goes into a context window

  • 04:25 — Model limits and the applications around them

  • 06:57 — How 200,000 tokens become 2.1 million across a conversation

  • 09:45 — Caching, pricing tiers, and response time

  • 13:01 — Long-context reasoning and what compaction can lose

  • 15:11 — Context practices from the Book of Bob

  • 20:24 — Selling tokens and the cost of an accepted result

  • 22:52 — What to measure in your own coding sessions

  • 24:45 — Outro

Links and references

Context windows and coding tools

Conversation costs and caching

  • OpenAI API pricing — Source for the GPT-6.1 Sol example: $2 per million uncached input tokens at the standard short-context rate on the research date.

  • OpenAI conversation state — Continuing a conversation with a response ID, including how earlier input is billed.

  • OpenAI prompt caching — Prefix reuse and the conditions for cache hits.

  • Claude API pricing — Cache rates and model-specific long-context pricing.

Context practices and research

Thanks for reading The Main Thread! Subscribe for free to receive new posts and support my work.

Thank you for reading.

If this piece was useful, sharing it is the simplest way to support the work and help others find it.

Thoughts and corrections are always welcome. I read every comment.

If this email was forwarded to you, you can subscribe to receive future issues directly.

Write boring code. Ship exciting systems.

© 2026 Markus Eisele
🇩🇪