AI / A CONCEPT NOTE
Temperature & Sampling
why answers vary
Overview · mechanism
pitfall · examples
01 / THE SHORT VERSION
The idea in a few sentences.
Temperature scales the token probability distribution. High temperature flattens probabilities (more creative/random); low temperature peaks probabilities (more deterministic).
02 / FOLLOW THE MECHANISM
How tokens are selected
Output logits
model computes raw math scores (logits) for potential next tokens.
Temp scale
divides logits by temperature. Temp=0 sets highest token probability to 1.
Probability mapping
runs softmax to get clean percentage probabilities per token.
Selector
samples next token from probability distribution.
04 / COMMAND NOTES
Read the command, then the result.
Inspect the flags and arguments before trying an example. Snippets can need local setup, replacement values, or resources in your own environment.
query local model with explicit temperature settings
curl -X POST -d '{\"model\":\"llama3\",\"prompt\":\"hi\",\"options\":{\"temperature\":0.0}}' http://localhost:11434/api/generate05 / CHECK YOURSELF
Could you explain Temperature & Sampling to a teammate?
Try it out loud in two sentences: what it is, and the one detail that changes the picture. If you stall, the gap is the part to reread.
Up next in AI engineeringModel Servingputting a model behind an endpoint