TOPIC AI
Temperature & Sampling why answers vary
IN 10 SECONDS
Temperature scales the token probability distribution. High temperature flattens probabilities (more creative/random); low temperature peaks probabilities (more deterministic).
GOTCHA Setting temperature to 0.0 guarantees identical responses for the same prompt, but disables creative brainstorming capabilities.
HOW TOKENS ARE SELECTED
01 Output logits model computes raw math scores (logits) for potential next tokens.
02 Temp scale divides logits by temperature. Temp=0 sets highest token probability to 1.
03 Probability mapping runs softmax to get clean percentage probabilities per token.
04 Selector samples next token from probability distribution.
POKE IT YOURSELF
curl -X POST -d '{\"model\":\"llama3\",\"prompt\":\"hi\",\"options\":{\"temperature\":0.0}}' http://localhost:11434/api/generate — query local model with explicit temperature settings
Drill this topic →
~65 sec read