Explainable artificial intelligence (XAI)

Explainable AI (XAI) refers to the set of technologies and practices required to make the outputs of an AI model understandable and interpretable by human experts.

All edits by  Jason D. Rowley 

Edits on 22 Jun, 2020
Jason D. Rowley
Jason D. Rowley edited on 22 Jun, 2020
Edits made to:
Categories (+1 topics)
Related Topics (+3 topics)
Categories
Related Topics
Jason D. Rowley
Jason D. Rowley edited on 22 Jun, 2020
Edits made to:
Article (-4 characters)
Article

The Artificial Intelligence is sometimes characterized as a black box system because it can be difficult to determine how a particular output or behavior resulted from a given set of inputs.

Jason D. Rowley
Jason D. Rowley approved a suggestion from Golden's AI on 22 Jun, 2020
Edits made to:
Article (+16/-16 characters)
Article
  • As machine learningmachine learning and AI techniques become part of the standard analytical toolkit across many scientific disciplines, unexplainable "black box" machine learning models make reproducing scientific experiments and analysis difficult.
Jason D. Rowley
Jason D. Rowley approved a suggestion from Golden's AI on 22 Jun, 2020
Edits made to:
Article (+23/-23 characters)
Article

The Artificial IntelligenceArtificial Intelligence is sometimes characterized as a black box system because it can be difficult to determine how a particular output or behavior resulted from a given set of inputs.

Edits on 19 Jun, 2020
Jason D. Rowley
Jason D. Rowley edited on 19 Jun, 2020
Edits made to:
Timeline (+1 events) (+463 characters)
Article (+1 images) (+1656/-32 characters)
Further reading (+1 rows) (+4 cells) (+170 characters)
Article
via DARPA
The "black box AI" problem

The Artificial neural networksIntelligence areis sometimes characterized as a black box systemssystem because it iscan oftenbe difficult to determine how a particular output or behavior resulted from a given set of inputs.

This presents a number of challenges for AI researchers and engineers in academia and in the private sector:

  • As machine learning and AI techniques become part of the standard analytical toolkit across many scientific disciplines, unexplainable "black box" machine learning models make reproducing scientific experiments and analysis difficult.
  • In commercial settings, unexplainable AI models may produce results which are difficult to audit, and thus difficult to trust. Black box AI models may also be biased in ways that are difficult to detect and resolve.

Risks imposed by mysterious, unexplainable behavior in AI models prompted AI and machine learning researchers to seek out ways to produce more explainable models and build artificial intelligence systems which can be understood and appropriately trusted.

Opening the black box

There are several approaches to the challenge of "opening the black box," including:

  • Model explanation, wherein the explanation involves the global logic of the black box classifier
  • Outcome explanation, wherein the goal is to locally understand the reasons for the decision of a particular output or result
  • Model inspection, where the objective is to understand how the black box behaves internally, which can involve using alternate inputs and other tools to unpack the inner workings of a model
Research programs

There are many research efforts into explainable AI by academic institutions, the private sector, and governments. DARPA—the research and development arm of the U.S. military—launched its Explainable Artificial Intelligence program in August 2016.

Further reading

Title
Author
Link
Type
Date

Interesting resources related to XAI (Explainable Artificial Intelligence)

Przemysław Biecek

Github repository

Timeline

August 11, 2016

DARPA launches its Explainable Artificial Intelligence research program

In its announcement, DARPA said that "The goal of Explainable Artificial Intelligence (XAI) is to create a suite of new or modified machine learning techniques that produce explainable models that, when combined with effective explanation techniques, enable end users to understand, appropriately trust, and effectively manage the emerging generation of Artificial Intelligence (AI) systems."

Edits on 19 Jun, 2020
Jason D. Rowley
Jason D. Rowley edited on 19 Jun, 2020
Edits made to:
Article (+355 characters)
Article

Explainable AI (XAI) refers to the set of technologies and practices required to make the outputs of an AI model understandable and interpretable by human experts.

...

Artificial neural networks are sometimes characterized as black box systems because it is often difficult to determine how a particular output or behavior resulted from a given set of inputs.

Jason D. Rowley
Jason D. Rowley edited on 19 Jun, 2020
Edits made to:
Article (-14 characters)
Further reading (+1 rows) (+4 cells) (+265 characters)
Article

Explainable AI

Further reading

Title
Author
Link
Type
Date

Explanation in Human-AI Systems: A Literature Meta-Review, Synopsis of Key Ideas and Publications, and Bibliography for Explainable AI

Shane T. Mueller, Robert R. Hoffman, William Clancey, Abigail Emrey, Gary Klein

February 5, 2019

Jason D. Rowley
Jason D. Rowley edited on 19 Jun, 2020
Edits made to:

Explainable AI Explainable artificial intelligence (XAI)

Explainable AI (XAI) refers to the set of technologies and practices required to make the outputs of an AI model understandable and interpretable by human experts.

Jason D. Rowley
Jason D. Rowley edited on 19 Jun, 2020
Edits made to:
Description (+163 characters)
Article (+14 characters)
Further reading (+1 rows) (+5 cells) (+180 characters)

Explainable artificial intelligence (XAI)

Explainable AI (XAI) refers to the set of technologies and practices required to make the outputs of an AI model understandable and interpretable by human experts.

Article

Explainable AI

Further reading

Title
Author
Link
Type
Date

Artificial Intelligence Confronts a 'Reproducibility' Crisis

Gregory Barber

Web

September 16, 2019

Jason D. Rowley
Jason D. Rowley edited on 19 Jun, 2020
Edits made to:
Further reading (+1 rows) (+5 cells) (+268/-17 characters)
Further reading

Title
Author
Link
Type
Date

Explainable AI in industry: practical challenges and lessons learned: implications tutorial

Krishna Gade, Sahin Cem Geyik, Krishnaram Kenthapadi, Varun Mithal, and Ankur Taly

Conference paper

January 2020

Explainable AI: The Basics

The Royal SocietyThe Royal Society

PDF / Policy Brief

November 2019

Jason D. Rowley
Jason D. Rowley edited on 19 Jun, 2020
Edits made to:
Article
Further reading (+5 rows) (+21 cells) (+953 characters)
Documentaries, videos and podcasts (+4 rows) (+15 cells) (+683 characters)
Companies (+1 rows) (+1 cells)
Article

Companies

Company
CEO
Location
Products/Services

Further reading

Title
Author
Link
Type
Date

A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI

Erico Tjoa, Cuntai Guan

August 2015

Explainable AI: The Basics

The Royal Society

PDF / Policy Brief

November 2019

Explainable Artificial Intelligence

Dr. Matt Tureck

Web

Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI

Alejandro Barredo Arrieta, Natalia Díaz-Rodríguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador García, Sergio Gil-López, Daniel Molina, Richard Benjamins, Raja Chatila, Francisco Herrera

October 22, 2019

Towards Explainable Artificial Intelligence

Wojciech Samek, Klaus-Robert Müller

September 26, 2019

Documentaries, videos and podcasts

Title
Date
Link

Explainable AI for Science and Medicine

Explaining the Predictions of Machine Learning Models with Carlos Guestrin - The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)

October 9, 2016

Real World Model Explainability with Rayid Ghani - Talk #283

July 18, 2019

The Problem with Black Boxes with Cynthia Rudin - Talk #290

August 12, 2019

Jason D. Rowley"Initial creation data"
Jason D. Rowley edited on 19 Jun, 2020
Edits made to:
Infobox (+1 properties)
Infobox
Is a
Industry
Jason D. Rowley"Created via: Web app"
Jason D. Rowley created this topic on 19 Jun, 2020
Edits made to:

 Explainable AI

Explainable AI (XAI) refers to the set of technologies and practices required to make the outputs of an AI model understandable and interpretable by human experts.

Golden logo
Text is available under the Creative Commons Attribution-ShareAlike 4.0; additional terms apply. By using this site, you agree to our Terms & Conditions.