TOPIC AI
Top-p Sampling choosing from cumulative probability p
IN 10 SECONDS
Nucleus sampling. Dynamically selects from the smallest pool of tokens whose combined probability exceeds threshold 'p' (e.g. p=0.9), keeping options flexible based on confidence.
GOTCHA Combining high temperature with high top-p can make model outputs incoherent and erratic.
HOW DYNAMIC FILTERING FLOWS
01 Probability array ranks all vocabulary tokens by probability.
02 Cumulative sum accumulates probabilities: token-1 (50%), token-2 (30%), token-3 (12%) = 92%.
03 Cutoff check since sum exceeds p=0.9, all lower tokens are dropped.
04 Sample select randomly selects from the top 3 tokens.
POKE IT YOURSELF
curl -X POST -d '{"model":"llama3","prompt":"hi","options":{"top_p":0.9}}' http://localhost:11434/api/generate — query local model with top-p limits
Drill this topic →
~65 sec read