Long short-term memory
In episode 117, I expressed some concern. It seemed that neural network implementations lacked a way of holding onto information over time. It turns out that the problem has been addressed by recurrent neural networks. Recurrent networks remember, though not very well. Today, we look at a modification of recurrent networks that allow artificial neural networks to remember much more, for much longer.
Here is one of the best videos I’ve ever seen for explaining how a neural network functions, that explains how a long short-term neural network works.
Here are a couple of articles on long short-term memory neural networks.