Research digest
New research on tokenizers and model costs, with a plain answer for each paper to one question: does this change what you pay? Published monthly.
Affects API costs nowchanges what you pay today
Self-hosting onlymatters if you run models yourself
Research, no cost impact yetworth knowing, changes nothing today
October 2026
Published 11 October 2026
Research, no cost impact yet
Meister, Altıntaş and Bosselut. 8 October 2026. Preprint.
- What it is
- The authors trained 123 small research models that differ only in their tokenizer, then measured how well each model handled 31 languages. The authors describe it as the largest controlled study of its kind.
- What it means for your bill
- Nothing directly. But if you build for a language with less training data, such as Tamil or Bengali, quality varies more from one tokenizer to the next. Test a few models on your own text before choosing.
- Worth knowing
- Giving under-represented languages a bigger share of the data used to build the tokenizer didn’t reliably help them, and sometimes hurt. The obvious fix isn’t a safe one.
Research, no cost impact yet
Ali et al. 8 October 2026. Under review.
- What it is
- A method for building tokenizers from the structure of words, such as roots and suffixes, before deciding what goes in the vocabulary. Tested on 104 languages, it reports fewer tokens per word and modestly better benchmark results.
- What it means for your bill
- Nothing today: no model you can pay for uses it. And the gap in cost between languages stays about the same as with standard methods, so it isn’t a fix for the extra tokens Indian languages cost.
- Worth knowing
- Its central argument is that squeezing text into fewer tokens isn’t the same as representing it well. A cheaper prompt isn’t automatically a better one.
Self-hosting only
Fu et al., Tsinghua University. 8 October 2026. Accepted at NeurIPS 2026. Code released.
- What it is
- Serving software for setups where a small, cheap model writes most of an answer and a larger model steps in only for the hard parts, switching word by word. It builds on R2R, a routing method from the same group published at NeurIPS 2025.
- What it means for your bill
- Nothing if you use hosted APIs: you can’t switch between providers in the middle of an answer without resending the whole conversation each time. What you can do today is route whole requests, sending routine ones to a cheap model and hard ones to a flagship. Try the model mix on the calculator to see what that saves.
- Worth knowing
- The headline speedup of up to 64 times is measured against serving software that was never designed for this kind of routing. It’s a speed result, not a cost reduction.
Affects API costs now
From Token Bench. Our own study.
- What it is
- We measured the same 1,012 sentences in fourteen Indian languages and English on eight tokenizers, including Claude and Gemini through their counting APIs.
- What it means for your bill
- For identical text in an Indian language, the choice of model can change your token bill several times over. Gemini was the cheapest of the eight we measured.
Price changes since September
We re-verified every price on the calculator against each provider’s own pricing page on 11 October 2026. Here’s what moved.
| OpenAI | A new lineup leads: GPT-6.1 Sol at $2 / $10 per million input and output tokens, and GPT-6 Luna at $0.10 / $0.50. GPT-6 Astra is unchanged at $10 / $50. |
|---|
| Anthropic | Claude Opus 5.5 is $4 / $20, a 20% cut from Opus 5. Claude Haiku 5.5 launched on 7 October at $0.10 / $0.50, rising to $0.50 / $2.50 for prompts over 100,000 tokens. Sonnet 5.5 kept Sonnet 5’s $2 / $10, with cheaper cached input. |
|---|
| Google | No change. Gemini 3.8 Flash’s introductory $0.75 / $3.75 doubles on 1 January 2027. |
|---|
| xAI | Grok 4.7 replaced Grok 4.6 as the flagship, at the same $2 / $6. |
|---|
| DeepSeek | V4.1 Flash replaced V4 Flash, at $0.15 / $0.60 off-peak. A correction to our September figures: since 16 August, DeepSeek prices have doubled during peak hours, which in India fall on weekday mornings and early afternoons, about 6:30 to 9:30 and 11:30 to 15:30. We missed that last month. |
|---|
Also changed on the calculator: OpenAI token counts now use o200k, the tokenizer OpenAI published for its recent models, instead of the older cl100k. For Indian-language text, cl100k overstated OpenAI’s cost considerably. Sources: the pricing pages of OpenAI, Anthropic, Google, xAI and DeepSeek.
Prices and tokenizers change every month. See what your own prompt costs today, across every major model.
Measure your own prompt