AI / A CONCEPT NOTE
Top-p Sampling
choosing from cumulative probability p
Overview · mechanism
pitfall · examples
01 / THE SHORT VERSION
The idea in a few sentences.
Nucleus sampling. Dynamically selects from the smallest pool of tokens whose combined probability exceeds threshold 'p' (e.g. p=0.9), keeping options flexible based on confidence.
02 / FOLLOW THE MECHANISM
How dynamic filtering flows
Probability array
ranks all vocabulary tokens by probability.
Cumulative sum
accumulates probabilities: token-1 (50%), token-2 (30%), token-3 (12%) = 92%.
Cutoff check
since sum exceeds p=0.9, all lower tokens are dropped.
Sample select
randomly selects from the top 3 tokens.
04 / COMMAND NOTES
Read the command, then the result.
Inspect the flags and arguments before trying an example. Snippets can need local setup, replacement values, or resources in your own environment.
query local model with top-p limits
curl -X POST -d '{"model":"llama3","prompt":"hi","options":{"top_p":0.9}}' http://localhost:11434/api/generate05 / CHECK YOURSELF
Could you explain Top-p Sampling to a teammate?
Try it out loud in two sentences: what it is, and the one detail that changes the picture. If you stall, the gap is the part to reread.
Up next in AI engineeringGroundinganchoring answers in verified facts