The bill and the length limit are both counted in pieces smaller than words. A token is a chunk of text the model reads and writes. It is often a word, often a piece of a word, and sometimes a bit of punctuation glued to something else. A token is not a word, and it is not the login token you use as a password. Vendors bill hosted APIs, the paid programming interface, in tokens. They also cap how many tokens fit in one request. That cap is the context window.
Karpathy’s Zero to Hero lectures and the Hugging Face course linked above build the same idea from characters and tokens. This lesson only needs the consequence: length is not measured the way a word processor measures it.
What the window holds
The window is the text in this request. It includes your message, the instructions, the earlier turns the product sends back, and any tool results the product pastes in. When the window is full, something has to be dropped, summarized, or refused. A summary can keep the plot and lose the number. If the number mattered, the summary failed, even if it reads well.
People say the model “remembers the conversation.” What they usually mean is that the app sends the previous turns again. A turn is one message in the chat. Close the tab, switch devices, or start a new chat, and that resend may stop. A memory feature that stores notes between chats is a database the product keeps. Saved notes between chats are separate from the text the model sees in one request.
Why the same paragraph costs more than you think
A pasted PDF, a long tool result, and a long system prompt, the standing instructions the app sends, all spend the window. The model uses that text on the way to an answer, and the host charges for the tokens it processed and the tokens it wrote back. Input and output prices can differ. Check the provider’s current pricing page for rates. The tool profiles on this site link the pricing pages they opened, with the date.
Anthropic’s prompt-engineering overview is about how you write the text that occupies that window: role, examples, and the format you want back. It is not a billing calculator. Read it when a prompt is failing. Read the pricing page when the invoice is the question.
A check you can do without a tokenizer
You do not need a library to see the shape.
- Paste a short email into a note. Count the words.
- Treat that word count as the wrong unit. Tokens are not words, so a word count is not a token estimate. Code, URLs, and names in other scripts split into more pieces than English prose.
- If a product shows a usage meter, write down the number it shows and the date. That number is the product’s count. A word count is not a substitute for it.
A long handbook often will not fit in the window. The usual alternatives are to retrieve one passage, or to hand the model a tool that can look the passage up.
Sources & limits: This page does not print a price per token. Use the provider’s current pricing page. The lectures linked above cover the mechanism in more detail.