Learn AI Create the Future — with Pranay
Main site

AI glossary

RNN (recurrent neural network)

In one sentence An RNN reads a sequence one step at a time, carrying a memory of what it has seen so far in a hidden state.

By Updated

A recurrent neural network, or RNN, processes a sequence one element at a time, passing a running summary of everything so far from each step to the next.

It reads the way you listen to a story: one sentence at a time, updating what you hold in your head after each one. That held summary — who the characters are, what has happened — is the RNN's hidden state. At every step the network combines the new input with the current hidden state to produce the next hidden state, using the same weights at every position.

word 1 → [RNN] → word 2 → [RNN] → word 3 → [RNN] → prediction
            ↓        ↑        ↓        ↑
          hidden state carried forward each step

That loop is the "recurrent" part, and it made RNNs the standard tool for text, speech and time series for two decades.

Two weaknesses ended that reign. The memory is a bottleneck: everything seen so far must be squeezed into one fixed-size state, so distant details fade. And the step-by-step chain gives the vanishing-gradient problem its worst case — the LSTM was invented to patch exactly this. Sequential processing also cannot be parallelised across the sequence, which is the practical reason the transformer replaced RNNs almost everywhere. They survive today in small, low-latency settings such as on-device audio.

Where to go next