US Patent 10474709 Deep reinforced model for abstractive summarization

Disclosed RNN-implemented methods and systems for abstractive text summarization process input token embeddings of a document through an encoder that produces encoder hidden states; applies the decoder hidden state to encoder hidden states to produce encoder attention scores for encoder hidden states; generates encoder temporal scores for the encoder hidden states by exponentially normalizing a particular encoder hidden state's encoder attention score over its previous encoder attention scores; generates normalized encoder temporal scores by unity normalizing the temporal scores; produces the intra-temporal encoder attention vector; applies the decoder hidden state to each of previous decoder hidden states to produce decoder attention scores for each of the previous decoder hidden states; generates normalized decoder attention scores for previous decoder hidden states by exponentially normalizing each of the decoder attention scores; identifies previously predicted output tokens; produces the intra-decoder attention vector and processes the vector to emit a summary token.