In a nutshell
Policy gradient learns the behaviour directly instead of first estimating how good each state is. The policy is a set of probabilities over the actions; the agent samples an action, sees the reward, then adjusts the probabilities — gradient ascent — to make rewarding actions more likely and poor ones less likely. Over many samples the policy drifts toward whatever earns the most reward. The classic version of this is the REINFORCE algorithm.