TOPIC AI
RLHF Reinforcement Learning from Human Feedback
IN 10 SECONDS
Aligning LLMs by collecting human preferences (rating model outputs A vs B), training a reward model to score outputs, and optimizing the LLM using reinforcement learning.
GOTCHA If human raters select answers that look confident but are factually wrong, RLHF can inadvertently teach the model to hallucinate convincingly.
HOW THE REWARD LOOP FLOWS
01 Model outputs LLM generates two alternative answers (A and B) for the same user prompt.
02 Labeling human raters select the better response (A over B).
03 Reward model trains a neural network to score responses like a human rater.
04 RL Optimization tunes the LLM using PPO to maximize high-scoring generations.
POKE IT YOURSELF
python -c "from trl import PPOTrainer; ..." — load RLHF reinforcement optimization libraries
Drill this topic →
~70 sec read