TOPIC AI
Tokens & Context what the model can 'see'
IN 10 SECONDS
Tokens are text snippets (words or subwords). The context window is the maximum number of prompt and generation tokens a model can process in a single request.
GOTCHA As the context window fills up, model recall accuracy can degrade, sometimes overlooking information in the middle of long prompts.
HOW CONTEXT LIMITS OPERATION
01 Prompt load user sends a large context document. Tokenizer splits text into token IDs.
02 Limit check compares prompt tokens against model limit (e.g. 200K tokens).
03 Processing attention layers compute relationships across all tokens simultaneously.
04 Overflow if prompt size exceeds limit, API throws a context-exhaustion exception.
POKE IT YOURSELF
python -c "import tiktoken; enc = tiktoken.get_encoding('cl100k_base'); print(len(enc.encode('hello context')))" — count tokens in prompt string
Drill this topic →
~65 sec read