TOPIC AI
LLM large language models that generate text by predicting the next token
IN 10 SECONDS
A Large Language Model is a neural network trained on vast text corpora. Given a prompt, it generates tokens (words or subwords) one at a time by predicting the most likely next token based on the preceding context. The 'magic' is pattern completion at massive scale — trillions of parameters in the largest models.
GOTCHA LLMs don't 'know' anything — they predict tokens. They can produce confident-sounding falsehoods (hallucinations) with no awareness they're wrong. Always verify critical information independently.
HOW AN LLM GENERATES A RESPONSE
01 Input prompt is tokenized — split into tokens and mapped to numeric IDs using the model's vocabulary.
02 Transformer layers process the tokens through stacked self-attention and feed-forward layers, building a context-aware representation.
03 Output head produces a probability distribution over the vocabulary for the next token.
04 Sampling strategy selects the next token — greedy (highest probability), top-k, top-p (nucleus), or with temperature for creativity.
05 Decode the selected token ID is mapped back to text, appended to the output, and fed back in for the next token.
POKE IT YOURSELF
curl -X POST -H "Content-Type: application/json" -d '{"model":"gpt-4","messages":[{"role":"user","content":"Hello"}]}' https://api.openai.com/v1/chat/completions — call an LLM API directly
ollama run llama3 — run an LLM locally with Ollama
Drill this topic →
~85 sec read