ELMo

ELMo is a deep contextualized word representation from AllenNLP that models both characteristics of word use (e.g., syntax and semantics), and how these uses vary across linguistic contexts (i.e. to model polysemy).

ELMo is a deep contextualized word representation that models both characteristics of word use (e.g. syntax and semantics), and how these uses vary across linguistic contexts (i.e. to model polysemy). These word vectors are learned functions of the internal states of a deep bidirectional language model (biLM), which is pre-trained on a text corpus. They can be easily added to existing models on a range of NLP problems, including question answering, textual entailment and sentiment analysis.

All models except for the 5.5B model were trained on the 1 Billion Word Benchmark, approximately 800M tokens of news crawl data from WMT 2011. The ELMo 5.5B model was trained on a dataset of 5.5B tokens consisting of Wikipedia (1.9B) and all of the monolingual news crawl data from WMT 2008-2012 (3.6B). In tasks where a direct comparison was made, the 5.5B model has slightly higher performance then the original ELMo model.

ELMo models have been trained for other languages and domains.

Timeline

People

Name
Role
LinkedIn

Further reading

Title
Author
Link
Type
Date

Deep contextualized word representations

Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, Luke Zettlemoyer

Documentaries, videos and podcasts

Title
Date
Link

Companies

Company
CEO
Location
Products/Services

News

Title
Author
Date
Publisher
Description
March 4, 2021
WebWire
SC Johnson and Sesame Workshop are teaming up on an initiative to create new tools and resources for young children and caregivers worldwide to promote healthy and environmentally sustainable practices at home. Today's release on SesameStreet.org/caring features brand new videos starring Elmo and his father, Louie, modeling healthy routines around bedtime, bath time and sick days. Part of Sesame Workshop's broader Caring for Each Other initiative to support families during COVID-19 and bey...
Simon Cohen
January 5, 2021
Digital Trends
It's never easy keeping kids from boredom, but right now it's especially challenging. If you have an Alexa smart speaker, these skills can help fill the time.
December 2, 2020
WebWire
Breakfast just got more fun, delicious, and educational with the launch of Sesame Street Cereal, made by General Mills. The new cereal, which will begin to appear on U.S. store shelves in January 2021, supports growing bodies with many essential nutrients and engages curious young minds with activities and stories. The cereal features two varieties: berry flavor with number-shaped pieces and cinnamon flavor with letter-shaped pieces, and the back of each box opens like a book, featuring du...
Michael Bizzaco
November 27, 2020
Digital Trends
Did you know that there are tons of games you can play with Alexa? From choose-your-own adventures to family trivia, here are some of our favorite Echo games.
Sahar Esfandiari
December 13, 2019
Business Insider
It's a British tradition for comedy candidates to run against the sitting Prime Minister, and so it was in Johnson's Uxbridge and South Ruislip seat.

References

Golden logo
Text is available under the Creative Commons Attribution-ShareAlike 4.0; additional terms apply. By using this site, you agree to our Terms & Conditions.