In a nutshell
A Markov decision process (MDP) is the standard way to frame a decision-making problem: an agent in some state takes an action, lands in a new state, and collects a reward. Value iteration solves it by asking each state 'how good is it to be here?' — the answer is the best reward reachable through its neighbours, with future reward discounted a little. Repeating this backs value outward from the goal until the best action in every state is clear; those best actions together form the optimal policy.