Human-in-the-loop

Human-in-the-loop (HITL) is a branch of AI that brings together AI and human intelligence to create machine learning models. It refers to any system that allows humans to give direct feedback to a model for predictions below a certain level of confidence. HITL approaches use real-time manual interventions to continually train and calibrate models to achieve the designed result. This could be to inspect, validate, or change some part of the process to train and deploy a model into production. HITL can also extend to people preparing and structuring data for machine learning. Machine learning applications utilizing HITL could include chatbots and recommendation engines.

When machine learning models are deployed, there is a high chance they come across situations they are unprepared to handle due to under-representation or misrepresentation in their training data. In these situations, incorporating HITL can verify predictions of the AI model and provide feedback that either replaces the AI-generated prediction or is used for re-training and fine-tuning of the model.

HITL aims to combine AI and human intelligence to achieve what would not be possible relying solely on one. When a machine learning algorithm isn’t able to solve a problem, humans can intervene. This process creates a continuous feedback loop that helps the algorithm learn and produce better results.

HITL is used in many scenarios:

• When the algorithm does not understand the input

• When the data input is incorrectly interpreted
• When algorithms don't know how to perform a task
• To make humans more efficient and accurate
• To make the machine learning model more accurate
• When the cost of errors in ML development is too high
• When the data you’re looking for is rare or unavailable

HITL is an important tool for organizations looking to incorporate AI technology while maintaining human oversight to help remove the risk of wrong outcomes or accusations of bias. A lack of trust in the robustness of these types of technology has become a barrier to entry for many organizations looking to innovate. This is particularly true for high-risk industries, such as healthcare.

The benefits of HITL extend across the AI model lifecycle:

• People are used for data labeling, quality control (QC), and modeling, which continuously improves machine learning and deep learning systems

• Automation speeds up the development process through people building, maintaining, and monitoring exceptions
• People oversee the process by which an AI solution is integrated into an organization
• When AI models fail, people can step in to mitigate risk and resolve problems

Typically, HITL can be separated into one of two machine learning algorithm approaches: supervised or unsupervised learning.

Experts use labeled data sets to train algorithms to produce appropriate functions. These can then help to map new examples. Doing this will allow the algorithm to correctly determine functions for unlabeled data.

Unlabeled datasets are fed to the algorithms. Thus, they need to learn on their own to find a structure in the unlabeled data and memorize it accordingly. This falls under the human-in-the-loop deep learning approach.

HITL can be applied at various stages of the AI lifecycle:

HITL can be involved during model training, validation, and testing in order to accelerate the learning process. Humans can first demonstrate how tasks should be performed and then provide feedback on model performance. This can be done by correcting the model’s outputs or evaluating them, which creates a reward function for reinforcement learning. Learning from a combination of human demonstrations and evaluations has been demonstrated to be faster and more sample-efficient when compared with traditional supervised learning algorithms.

HITL workflows are especially important when the availability of training data is limited or the data is unbalanced or not comprehensive. In these instances, it can be difficult for models to handle all potential edge cases. In addition, even if the model usually achieves high accuracy, human monitoring and double-checking may be needed if model mistakes have the potential to be costly, for example, in cases like content moderation of user-generated content where false negatives may result in irreparable damage. In both cases, the model can be connected to a labeling interface in which outputs below a given threshold of certainty are routed so they are checked and verified by a human, either in real-time or in batches for future re-training.

One of the biggest challenges in implementing and sustaining a machine learning model is maintaining an integrated system that can be continuously operated in production. HITL can help in production in all of the ways they helped develop the model, up to this point. When teams work with the same data analysts throughout the full process, they can benefit from the improvements and optimizations HITL teams help deploy across the entire model development process.