Your model has a desk, not an infinite warehouse
A token is a unit a model processes; it can be a word fragment, punctuation or another representation depending on the model and modality. The Hugging Face course explains the language-model pipeline. Avoid turning a rough words-to-tokens shortcut into an invoice guarantee. Languages, formatting, code and media change the relationship.
Context is the information available during a model call. It can include instructions, your message, earlier conversation and retrieved material. A product’s “memory” is different: it may store information and reintroduce selected pieces later. Our desk analogy is conceptual: the context window is the working surface, while stored material is a filing system outside it. The prompting guide explains how provided context and instructions shape responses. A large desk does not guarantee that every paper on it will be used correctly.
Why the distinction costs money
API providers commonly price input and output separately, with additional rules for caching, tools and processing tiers. The OpenAI rate card is a concrete example. A repeated conversation can bring previous text back into later requests. “I only typed one new sentence” is therefore not the same as “the model processed one sentence.” The exact bill depends on the product and request construction.
Here is an intentionally hypothetical calculation: if input costs $2 per million tokens, then 10,000 input tokens cost $0.02 before output, tools and other charges. That is arithmetic, not a quote for a specific model or an observed bill. The formula is tokens divided by one million, multiplied by the applicable rate. Repeat it for each billable component. Do not mix a monthly chat subscription with an API unit price.
Make context earn its seat
Our recommendation is to send the smallest sufficient evidence set. Put the task and output constraints where they are easy to identify; include the relevant document sections, not an entire drive “just in case.” That is an application-design choice informed by prompting guidance, not a guarantee that shorter prompts always perform better.
Record the model, input size, output size and any tool calls when comparing workflows. If memory supplied a stale detail, correct or remove that stored detail rather than piling another instruction on top. Check product-specific memory controls before assuming a conversation is remembered or forgotten.
Check yourself
Exercise: A vendor advertises a million-token context window. Does that establish how much your app can submit, how much it will cost or whether it will recall every detail?
Answer: No. You still need the exact model’s input/output limits, product overhead, rates and evaluation on your own task. Context capacity is a limit, not a quality score. See the rate card’s long-context distinctions for an example of why one headline number is insufficient.
The takeaway: tokens measure processing; context is working information; memory is a product behavior. Different questions, different checkboxes.