As illustrated on the left side of Figure below an RNN receives some input (which could be a word or character), feeds it through the network, and outputs a vector called the hidden state.
At the same time, the model feeds some information back to itself through the feedback loop, which it can then use in the next step. This can be more clearly seen if we "unroll" the loop as shown on the right side of the figure above: the RNN passes information about its state at each step to the next operation in the sequence. This allows an RNN to keep track of information from previous steps, and use it for its output predictions.
To learn more about RNNs, checkout Transformer Basics where we apply the RNN to an encoder-decoder architecture.