Where do your AI tokens go?

01 / The unit

What are you actually paying for?

You pay for tokens. Anthropic, OpenAI and Google all price their text models per million tokens. [3] [4] [5] A token is a small chunk of text that a model reads or writes, often a piece of a word. The model never sees letters. It sees a list of tokens.

OpenAI's rule of thumb for English is that one token is about four characters, or about three-quarters of a word, so 100 tokens is about 75 words. [1] Google gives the same four characters for Gemini, and says 100 tokens is about 60 to 80 English words. [2]

Here's the bit that trips people up: a token isn't a fixed size. The software that splits text into tokens is called a tokenizer, and different models can use different ones. OpenAI notes that the same text can give different counts depending on the model and the language. [1] Anthropic says its tokenizer for Claude 4.7 and later models produces about 30% more tokens for the same text than earlier models. [3] So two models with the same price per token can still charge you different amounts for the same job.

Pictures become tokens too. Claude cuts an image into 28×28-pixel patches and counts each patch as a token, so a 1000×1000 image costs 1,296 tokens. [31]

On your phone

You pay for megabytes, whatever app uses them. A megabyte is always the same size.

In your AI app

You pay for tokens, whatever part of the request they come from. A token's size depends on the model's tokenizer.

02 / The price list

How is an AI bill actually calculated?

Every call has two sides. Input tokens are everything you send to the model. Output tokens are everything it writes back. They're priced separately, and output costs more.

Sources: Anthropic [3], OpenAI [4], Google [5]. Prices change, so check the pricing page before you budget.

Notice the pattern. In every row, output costs five times as much as input. The maths is simple:

cost = input tokens × input price ÷ 1,000,000 + output tokens × output price ÷ 1,000,000

Say a support answer sends 3,000 input tokens and gets 500 back on a $2 / $10 model. Input costs $0.006 and output costs $0.005, so the call costs $0.011. That looks like nothing. At 10,000 calls a day, it's $3,300 in a 30-day month. And the 500 output tokens, about a seventh of the total, are almost half the bill.

The price also depends on how you call the model. Batch jobs, cached input, very long prompts and regional processing each have their own rates on these pricing pages. [3] [4] We'll use some of those in chapter 9.

On your phone

Your plan charges a price per gigabyte, and every app draws from it.

In your AI app

Each model has a price per million input tokens and a higher price per million output tokens.

03 / Background data

Where do the tokens you never typed come from?

You type a 30-word question. The usage report says more than 8,000 input tokens. What happened?

Your app wrapped your question in a lot of other text before sending it. Each request can carry:

• A system prompt. The standing instructions your app sends on every call: tone, rules, product facts.
• Conversation history. Everything said so far in the chat. Chapter 4 shows why this grows.
• Tool definitions. If the model can call tools, their names, descriptions and schemas are input tokens. Anthropic also adds a small tool-use system prompt, 286 tokens on Claude Sonnet 5.5 with the default tool choice. [3]
• Retrieved text and tool results. Documents your app looks up, search results and web pages. Anthropic estimates an average 10 kB web page at about 2,500 tokens, and counts web search results as input in later turns too. [3]
• Images and files, at the rates from chapter 1.

On the way back, there's the answer you see, and sometimes thinking you don't see. That's chapter 5.

On your phone

Background refresh, auto-updates and the family group's videos downloading on their own.

In your AI app

The system prompt, history, tool schemas and retrieved text sent with every request.

04 / The growing thread

Why does a long chat cost more with every turn?

The first message in a chat is cheap. The twentieth isn't. Why should replying to the same conversation get more expensive?

Because the model doesn't remember your chat between calls. Anthropic's docs put it plainly: the Messages API is stateless, so you always send the full conversation history. [6] Every reply resends everything before it.

Even when a provider stores the conversation for you, you can still pay for it. OpenAI's docs say that when you chain responses with previous_response_id, all previous input tokens in the chain are billed as input tokens. [7]

So in a chat like this, the input per turn grows steadily, and the total grows much faster. In the toy chat below, 30 turns send 162,000 input tokens to deliver 1,500 tokens of new questions. Long chats also run into the context window: the maximum number of tokens a single request can use. [7]

You have three ways out: cache the part that repeats (chapter 9), trim or summarise old turns, or start a fresh thread when the topic changes.

On your phone

Imagine a chat app that downloaded the whole conversation again every time you sent a reply.

In your AI app

Each call resends the conversation so far as input tokens, so every turn costs more than the last.

05 / The invisible output

What are thinking tokens, and why can't you see them?

You ask a reasoning model a short question. It gives a two-line answer, yet the usage says it wrote thousands of output tokens. Where are they?

Many current models think before they answer. They work through the problem in tokens of their own, then write the reply. OpenAI says reasoning tokens aren't visible as answer text, but they count toward output usage and are billed as output tokens. [1] Google's price table for Gemini 3.8 Flash labels its output price as including thinking tokens. [5]

Anthropic is just as direct. Claude can return a summary of its thinking, but you're billed for the full thinking it generated, so the billed output count doesn't match what you can see. [8]

The good news is that the meter shows it. Claude reports thinking in usage.output_tokens_details.thinking_tokens [8], and OpenAI's Responses API reports output_tokens_details.reasoning_tokens. [11] You can also turn it down. On Claude, the effort setting is the main control for how often and how deeply the model thinks, and thinking counts toward max_tokens, the hard cap on output. [8]

On your phone

A video playing in a hidden tab still uses your data, even though you never watch it.

In your AI app

Thinking you never see is still billed at the output price.

06 / The meter

Where can you see the meter?

On your phone, the data counter lives in Settings. Where's the AI equivalent?

It comes back with every response. Each API returns a usage object: a small block of numbers that says how many tokens that one call used. The catch is that every provider names and counts the fields differently.

Sources: Anthropic [8] [9], OpenAI [11], Google [12].

Look closely at the first row, because this is the most common tracking bug. Anthropic's input_tokens only counts tokens after the last cache breakpoint. The total is cache_read_input_tokens + cache_creation_input_tokens + input_tokens. [9] Gemini's prompt_token_count already includes cached content. [12] If you log only the field called "input" from each, your Claude totals will look far too small once caching starts.

You can also check the meter before you spend anything. Anthropic's token counting endpoint takes the same request you'd send and returns the input token count. It's free, with its own rate limits, and the count is an estimate. [13]

For the whole organisation, providers keep their own records. Anthropic's Usage and Cost API returns token usage and cost with an admin key, filtered or grouped by API key, workspace, model and more. [14] OpenAI has a Usage API and a Costs API, also with an admin key; the Usage API can group by project and model. [15]

These reports know your keys, projects and models. They don't know your features, your teams or your customers. That's the gap the next chapter fills.

On your phone

The data counter in Settings, and the monthly usage report from your operator.

In your AI app

The usage object on every response, and the provider's usage and cost reports.

07 / Usage by app

How do you find out which feature spent it?

Your phone can show data usage by app. Your AI bill can't, unless you build that view yourself. How?

The answer is observability: recording what your system did, so you can ask questions about it later. The standard tool is a trace, the path one request takes through your application. A trace is made of spans, and each span is one unit of work, such as a model call or a tool call. [33]

OpenTelemetry is an open-source, vendor-neutral framework for producing traces, metrics and logs. [33] It has conventions for AI calls. A model-call span is named after the operation and the model, like chat claude-sonnet-5-5, and records gen_ai.usage.input_tokens and gen_ai.usage.output_tokens. The input count should include cached tokens, so for Claude you add the cache fields in, just like chapter 6. [16] These conventions are still marked as in development and recently moved to their own repository, so check which version your tools follow.

Then comes the step that makes it FinOps: add your own labels to each trace. The feature name, the team that owns it, the customer or plan, the prompt version. Now you can group tokens by the things your business cares about.

You don't have to build the dashboards. Langfuse is an open-source platform you can host yourself that tracks usage and cost for every model call. It can take usage straight from the response or estimate it from its own price list, and when it has both, the reported numbers win. [17] OpenLLMetry adds OpenTelemetry instrumentation for AI providers and sends it to backends you may already run, such as Datadog or Honeycomb. [18] Gateways such as LiteLLM and Helicone sit between your app and the providers, so every call passes through one place that tracks cost. LiteLLM's proxy tracks spend per project and user. [19] [20]

Whichever you pick, prefer the provider's own usage numbers over estimates. Estimates are a fallback, not a source of truth.

On your phone

The "data usage by app" screen that shows exactly who ate your pack.

In your AI app

Traces with token counts on each span, labelled by feature, team and customer.

08 / Managing the pack

What does FinOps add on top of tracking?

You can now see where tokens go. Who decides whether that spend was worth it?

That's FinOps. The FinOps Foundation defines it as an operational framework and cultural practice that maximises the business value of technology, helps teams make timely, data-driven decisions, and creates financial accountability through collaboration between engineering, finance and business teams. [21] It works in three phases: Inform (look at cost, usage and efficiency data), Optimize (find ways to improve efficiency and value) and Operate (put those changes into practice). [22]

AI is now the normal case, not the exception. In the State of FinOps 2026 survey, 98% of the 1,192 respondents said they manage AI spend, up from 31% two years earlier. [24]

The key idea is unit economics: cost per useful thing, not cost per month. The FinOps Foundation describes tokens as a unit of cost and a unit of value, and suggests measures such as cost per inference and cost per outcome. [23] For our support helper, that's the AI cost per resolved ticket. If the monthly bill doubles while tickets resolved triples, things got better.

Three habits turn this into practice:

• Give every feature a budget. Anthropic lets you set a monthly spend limit on a workspace, with alerts at thresholds you choose. [27] OpenAI supports spend alerts, which only notify, and hard spend limits for an organisation or project; once a hard limit is reached, affected requests get a 429 error. [28] Separate workspaces or projects per feature make these limits line up with your labels.
• Store tokens, not just dollars. Prices move. Google's pricing page shows Gemini 3.8 Flash at $0.75 per million input tokens through 31 December 2026, and $1.50 from 1 January 2027. [5] If you kept the tokens and the price you applied, you can recompute any month.
• Review it monthly. Inform, optimise, operate, then go round again.

The industry is standardising this too. FOCUS, an open specification for a common format of cost and usage data, plans a version 1.5 by December 2026 that surfaces AI model identity and token consumption. [26] And in June 2026, the Linux Foundation announced its intent to launch a Tokenomics Foundation for open standards in AI cost management. [25]

On your phone

A family plan where each person sees their own usage and gets a warning near the limit.

In your AI app

Budgets per feature, spend alerts, and cost per outcome reviewed every month.

09 / Saving data

How do you cut the bill without making the AI worse?

Once you know where tokens go, which levers actually move the bill, and which ones hurt quality?

Here they are, roughly in the order to try them.

1. Use the smallest model that passes your tests. Anthropic's own advice is Haiku for simple tasks, Sonnet for most production work and Opus for the hardest reasoning. [3] Prove the cheaper model is good enough with your own test set before you switch.
2. Cache what repeats. With prompt caching, the provider reuses a prefix it has already processed and charges less for it. On most Claude models, a cache read costs 0.1× the normal input price (less on the newest ones, such as 0.05× on Opus 5.5 and Sonnet 5.5), and writing to the 5-minute cache costs 1.25×. [3] OpenAI caches by default on supported models. [10] Put the parts that never change first: tools, then system prompt, then documents. [9]
3. Batch what can wait. Anthropic and OpenAI both charge 50% less for batch jobs, which run in the background with up to 24 hours to finish. Google's batch prices for Gemini 3.8 Flash are half its standard rates. [29] [30] [5] Nightly summaries and bulk tagging are perfect for it.
4. Cap output and thinking. max_tokens is a ceiling, not a target. [32] Lower effort for easy requests. [8]
5. Trim the context. Summarise old turns, send only the tools a request needs, retrieve fewer chunks, and shrink images. Anthropic suggests downsampling images when you don't need full detail. [31]

Caching and batching don't change what the model is asked, so they're the lowest-risk levers. Model choice, caps and trimming can change answers, so test them first.

If you only do one thing after reading this, make it this: log the normalised usage of every call with a feature label. Then fix the biggest bar first.

On your phone

Download updates on Wi-Fi overnight, keep offline maps, and stream at lower quality when it's fine.

In your AI app

Batch what can wait, cache what repeats, and use a smaller model where it passes.

10 / Hands-on

How do you check the meter yourself?

Reading about the meter is one thing. Want to see real numbers from a real call? These steps use Anthropic's API because its token counter is free to call. [13] You'll need an API key in the ANTHROPIC_API_KEY environment variable.

1. Count tokens before you send

This command is copied from Anthropic's token counting docs.

curl https://api.anthropic.com/v1/messages/count_tokens \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "content-type: application/json" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
    "model": "claude-opus-5-5",
    "system": "You are a scientist",
    "messages": [{
      "role": "user",
      "content": "Hello, Claude"
    }]
  }'

The docs show this response: { "input_tokens": 14 }.

2. Send a message and read the usage

This one is copied from Anthropic's Messages API guide. [6]

curl https://api.anthropic.com/v1/messages \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{
    "model": "claude-opus-5-5",
    "max_tokens": 1024,
    "messages": [
      {"role": "user", "content": "Hello, Claude"}
    ]
  }'

The docs' example response ends with "usage": { "input_tokens": 12, "output_tokens": 6 }. Your numbers may differ slightly.

3. Log one line per call, with a label

This is the habit from chapter 9, in a few lines of Python. This one isn't from the docs; it's a small example written for this article. It uses the official anthropic SDK, adds the cache fields into the input total, and prices each kind of token at Claude Haiku 5.5's list rates. [3]

import json
import anthropic

client = anthropic.Anthropic()  # reads ANTHROPIC_API_KEY

# Claude Haiku 5.5 list prices, USD per million tokens (prompts up to 100,000 tokens)
PRICE = {"input": 0.10, "cache_write_5m": 0.125, "cache_read": 0.01, "output": 0.50}


def ask(question, feature):
    message = client.messages.create(
        model="claude-haiku-5-5",
        max_tokens=300,
        messages=[{"role": "user", "content": question}],
    )
    u = message.usage
    cache_read = u.cache_read_input_tokens or 0
    cache_write = u.cache_creation_input_tokens or 0
    cost = (
        u.input_tokens * PRICE["input"]
        + cache_write * PRICE["cache_write_5m"]
        + cache_read * PRICE["cache_read"]
        + u.output_tokens * PRICE["output"]
    ) / 1_000_000
    print(json.dumps({
        "feature": feature,
        "model": message.model,
        "input_tokens": u.input_tokens + cache_read + cache_write,
        "output_tokens": u.output_tokens,
        "estimated_cost_usd": round(cost, 6),
    }))
    return "".join(block.text for block in message.content if block.type == "text")


print(ask("In one sentence, what is a token?", feature="faq-bot"))

Each call prints one JSON line you can send to any log store. Group by feature, and you have your own "data usage by app" screen. Treat the cost as an estimate and reconcile it with the provider's cost report each month. [14]

On your phone

Open Settings, check the counter, and note which app used what.

In your AI app

Count before sending, read usage after, and log both with a label.

Glossary

The words, in one place

Token

A small chunk of text, or part of an image, that a model reads or writes. The unit of AI pricing.

Tokenizer

The software that splits text into tokens. Each model family has its own, so counts differ.

Input tokens

Everything sent to the model: system prompt, history, tools, documents and your message.

Output tokens

Everything the model writes, including thinking. Priced higher than input.

Thinking (reasoning) tokens

Tokens a model generates while working out an answer. Billed as output, even when hidden.

Context window

The maximum number of tokens a single request can use.

Prompt caching

Reusing an already processed prompt prefix at a lower input price.

Batch API

A way to submit many requests that finish later, at half the normal price.

Usage object

The token counts returned with every API response.

Trace and span

A trace is one request's path through your system. Spans are its steps, such as one model call.

OpenTelemetry

An open-source, vendor-neutral framework for producing traces, metrics and logs.

FinOps

The practice of managing technology spend for business value, shared by engineering, finance and business teams.

Unit economics

Cost per useful outcome, such as AI cost per resolved support ticket.

Spend limit

A cap on spending for a workspace, project or organisation, often with alerts before it.