LoginSign Up
Long short-term memory

Long short-term memory

A variation of recurrent neural network that can be backpropagated through time and layers

Long short-term memory network (LSTM) is a variation of recurrent neural network. It was proposed by the German researchers Sepp Hochreiter and Juergen Schmidhuber as a solution to the vanishing gradient problem.

LSTMs hold information outside the normal flow of the recurrent neural network in its memory blocks or cells. The information can be stored in, written to or read from a cell as if it is data in a computer. The memory blocks are responsible for remembering things and manipulations and regulated by structures called gates. The gating mechanism contains three non-linear gates, input, output and forget gate.

LSTMs are implemented with element-wise multiplication by Sigmoids layers output of one and zero. It has the advantage of being differentiable and suited for backpropagation.

LSTMs are used in text generation, handwriting recognition, handwriting generation, music generation, language translation and image captioning.

Timeline

People

Name
Role
Related Golden topics

Further reading

Title
Author
Link
Type

LONG SHORT-TERM MEMORY

Sepp Hochreiter and Jurgen Schmidhuber

Academic paper

Simplified Gating in Long Short-term Memory (LSTM) Recurrent Neural Networks

Yuzhen Lu and Fathi M. Salem

Academic paper

Documentaries, videos and podcasts

Title
Date
Link

Companies

Company
CEO
Location
Products/Services