In a nutshell
A pretrained base model can predict text but isn't tuned to what people actually want. RLHF aligns it in three stages. First, supervised fine-tuning: keep training on human-written ideal answers to teach the tone of a helpful assistant. Second, build a reward model: people repeatedly pick the better of two answers, and those choices train a separate model to score any response. Third, reinforcement learning: the model (the policy) writes answers, the reward model scores them, and the policy is nudged to earn higher scores — while a leash to the base model keeps it from drifting into nonsense. The payoff is an assistant tuned to be helpful, harmless, and honest.