Meta learning

Meta learning is a sub-field of machine learning. The underlying concept of meta learning is how a machine learns to learn. Studying the field of meta learning began in the late 1980's by Jurgen Schmidhuber, as well as Yoshua Bengio and Samy Bengio. Machine learning differs from meta learning, as machine learning is a focus on a single task, and meta learning is a way of teaching a machine how to learn multiple tasks efficiently. Meta learning is studied to improve artificial intelligence systems to help them operate outside of learned applications.

There are a few main meta learning systems including hyperparameter optimization, fast reinforcement learning, and few-shot image recognition. These systems can fall under certain meta learning methods such as recurrent models, metric learning, and learning optimization. On a basic level, each system is trained through a high exposure of tasks and then tested on their ability to learn new and different tasks. Each meta learning system has two separate optimizations working to increase efficiency, including the learner and the meta-learner (also known as an optimizer).

Recurrent Models

Recurrent Models use long short-term memory (LSTM) to analyze and process data-sets. The meta learner uses gradient descent, and the learner outputs the recurrent network. This method is common with few-shot classifications and meta-reinforcement learning.

Metric Learning

Metric Learning is a form of meta learning that entails learning a metric space when learning is especially efficient. Metric learning involves the meta-learner performing gradient descent, and the learner's role is based on a comparison scheme within the metric space. This method is also common in few-shot classification systems.

Learning Optimizers

Learning Optimizers operate by having the meta learner constantly update the learner until the learner can perform the task. The meta learner acts as a recurrent network, and can be trained with reinforcement or supervised learning. This method is popular in the field of neural network optimizations.