In a nutshell
A recurrent neural network reads a sequence one item at a time, keeping a hidden state — a running summary of everything seen so far — and feeding it back in at each step. That loop gives the network memory, so earlier words can shape later predictions. But over long sequences the learning signal fades on its way back (the vanishing gradient problem), so plain RNNs struggle to remember things from far away.