Infrastructure
Cost per Token
Also known as: token pricing
How AI services usually charge, by the number of tokens processed, both what you send in and what the model generates.
Cost per token is how most AI services bill you: by the amount of text going in and out, measured in tokens (small chunks of words). Longer prompts and longer answers cost more. Understanding this helps you keep bills predictable, sending a whole book into the AI on every request adds up fast, while concise prompts and answers stay cheap.
Most AI providers price by tokens, usually with separate rates for input (what you send) and output (what the model generates), and output is often more expensive. Your bill scales with total tokens across all requests.
What drives your cost
- Prompt size: long instructions, pasted documents and conversation history all count as input every time.
- Output length: longer answers cost more.
- Model choice: bigger, more capable models charge more per token; small models are cheaper.
- Extra thinking: reasoning models generate many internal tokens, raising cost.
Keeping it under control
Trim prompts, cap output length, use retrieval to send only relevant context, and pick the smallest model that does the job, all central to the ROI of an AI product.
Per-token pricing is the dominant commercial model for hosted LLMs, typically asymmetric (output priced above input, reflecting the memory-bandwidth-bound cost of autoregressive decode versus parallel prefill). Total spend is Σ(input + output) tokens × rate, so cost is driven by prompt design, history management, output caps and model tier, and inflated by reasoning traces and long-context usage.
Optimisation levers include prompt/context compression and retrieval (send only relevant chunks), history summarisation, prompt caching (discounts for reused prefixes), output-length control, model routing/right-sizing, and quantized or self-hosted serving where volume justifies it. Note tokenization asymmetries: non-English scripts fragment into more tokens, raising effective cost per word (a fairness and budgeting concern). At scale, cost-per-token is the unit economics underlying AI product margins and a central input to any ROI analysis, alongside latency and quality.