Adversarial machine learning

Adversarial machine learning is a branch of machine learning research focused on the development of secure and robust models through a process of attempting to deceive models using malicious or false inputs.

All edits by  Jason D. Rowley 

Edits on 5 Jun, 2020
Jason D. Rowley
Jason D. Rowley approved a suggestion from Golden's AI on 5 Jun, 2020
Edits made to:
Article (+16/-16 characters)
Article

Adversarial training entails intentionally incorporating statistical noise into the training data with the initial intent to deceive the model, thus identifying vulnerabilities and ways to improve model robustness and resilience. In the context of machine learningmachine learning, robustness refers to reliable operation of a system across a range of conditions (including attacks) and resilience refers to adaptable operations and recovery from disruptions (including attacks).

Edits on 3 Jun, 2020
Jason D. Rowley
Jason D. Rowley edited on 3 Jun, 2020
Edits made to:
Article (+217/-3 characters)
Article

TheFor developers and maintainers of machine learning models, the ultimate goal of incorporating adversarial methods is to train a model to accommodate and process inputs which may be malicious or otherwise differ from a narrow set of expected inputs. For malicious actors, the goal is to identify a vulnerability in the system which allows them to destroy, invalidate, or subvert a machine learning model.

Jason D. Rowley
Jason D. Rowley edited on 3 Jun, 2020
Edits made to:
Article (+1 images) (+747/-215 characters)
Article

Adversarial training entails intentionally incorporating statistical noise into the training data with the initial intent to deceive the model, thus identifying vulnerabilities and ways to improve model robustness and resilience. In the context of machine learning, robustness refers to reliable operation of a system across a range of conditions (including attacks) and resilience refers to adaptable operations and recovery from disruptions (including attacks).

Adversarial training entails intentionally incorporating statistical noise into the training data with the initial intent to deceive the model, thus identifying vulnerabilities and ways to improve model robustness. The ultimate goal of incorporating adversarial methods is to train a model to accommodate and process inputs which may be malicious or otherwise differ from a narrow set of expected inputs.

Taxonomy of attacks, defenses, and consequences

In October 2019, the National Institute of Standards and Technology (NIST) released and draft taxonomy and terminology guide for adversarial machine learning.

Taxonomy of Attacks, Defenses, and Consequences in Adversarial Machine Learning
Jason D. Rowley
Jason D. Rowley edited on 3 Jun, 2020
Edits made to:
Further reading (+1 rows) (+4 cells) (+211 characters)
Further reading

Title
Author
Link
Type
Date

A taxonomy and terminology of adversarial machine learning

Elham Tabassi, Kevin J. Burns, Michael Hadjimichael, Andres D. Molina-Markham, Julian T. Sexton

October 30, 2019

Jason D. Rowley
Jason D. Rowley edited on 3 Jun, 2020
Edits made to:
Categories (+2 topics)
Related Topics (+4 topics)
Categories
Related Topics
Jason D. Rowley
Jason D. Rowley edited on 3 Jun, 2020
Edits made to:
Article (+1 images) (+1126 characters)
Further reading (+1 rows) (+4 cells) (+151 characters)
Article
Adversarial examples

Adversarial examples are intentionally manipulated data which are fed into a neural network with the intent of deceiving it. An adversarial example is generated by introducing a small perturbation to a sample of known-good training data, such that the newly-generated adversarial example reliably causes undesired behaviors or outputs (ex. consistently mis-classifying images) from a machine learning model.

To simulate real-world malicious behavior against a neural network, adversarial examples often appear indistinguishable from legitimate samples from the training data. Adversarial examples of image or audio data, for example, may look or sound nearly identical to legitimate samples to avoid detection by human observers of the input stream.

An example of adversarial example generation applied to GoogLeNet.

Adversarial examples of image data can also be generated by printing images on paper, and then taking a photo of the resulting image printed onto a piece of paper.In addition to these real-world methods, there are open source software tools which can be used to generate adversarial examples.

Further reading

Title
Author
Link
Type
Date

Explaining and Harnessing Adversarial Examples

Ian J. Goodfellow, Jonathon Shlens, Christian Szegedy

December 20, 2014

Jason D. Rowley
Jason D. Rowley approved a suggestion from Golden's AI on 3 Jun, 2020
Edits made to:
Article (+25/-25 characters)
Article

Neural networks execute tasks such as clustering, classification, association and prediction.An artificial neural networkartificial neural network is a computational model which is developed based on iterative exposure to large sets of training data which affects the statistical weights and balances of the model.

Jason D. Rowley
Jason D. Rowley edited on 3 Jun, 2020
Edits made to:
Article (+694 characters)
People (+4/-3 characters)
Further reading (+2 rows) (+10 cells) (+291 characters)
Article

Neural networks execute tasks such as clustering, classification, association and prediction.An artificial neural network is a computational model which is developed based on iterative exposure to large sets of training data which affects the statistical weights and balances of the model.

...

Adversarial training entails intentionally incorporating statistical noise into the training data with the initial intent to deceive the model, thus identifying vulnerabilities and ways to improve model robustness. The ultimate goal of incorporating adversarial methods is to train a model to accommodate and process inputs which may be malicious or otherwise differ from a narrow set of expected inputs.

People

Name
Role
LinkedIn

Ian Goodfellow

Credited forwith inventing Generative Adversarial Networks (GAN)

Further reading

Title
Author
Link
Type
Date

Attacking Machine Learning with Adversarial Examples

Ian Goodfellow

Web

February 24, 2017

Is Supervised Learning With Adversarial Features Provably Better Than Sole Supervision?

Litu Rout

Preprint

October 30, 2019

Jason D. Rowley
Jason D. Rowley edited on 3 Jun, 2020
Edits made to:
Infobox (+1 properties)
People (+14/-14 characters)
Documentaries, videos and podcasts (+1 rows) (+4 cells) (+121 characters)
People

Name
Role
LinkedIn

Ian GoodfellowIan Goodfellow

Credited for inventing Generative Adversarial Networks (GAN)

Documentaries, videos and podcasts

Title
Date
Link

'How neural networks learn' - Part II: Adversarial Examples

January 11, 2018

Infobox
Is a
Industry
Jason D. Rowley
Jason D. Rowley edited on 3 Jun, 2020
Edits made to:
Infobox (+3 properties)
Infobox
Related industries
Machine learning
Edits on 15 May, 2020
Jason D. Rowley
Jason D. Rowley edited on 15 May, 2020
Edits made to:
People (+1 rows) (+2 cells) (+74 characters)
People

Name
Role
LinkedIn

Ian Goodfellow

Credited for inventing Generative Adversarial Networks (GAN)

Jason D. Rowley
Jason D. Rowley edited on 15 May, 2020
Edits made to:
Further reading (+1 rows) (+3 cells) (+145 characters)
Further reading

Title
Author
Link
Type
Date

Adversarial Machine Learning Reading List

Nicholas Carlini

Jason D. Rowley
Jason D. Rowley edited on 15 May, 2020
Edits made to:
Description (+207 characters)

Adversarial machine learning

Adversarial machine learning is a branch of machine learning research focused on the development of secure and robust models through a process of attempting to deceive models using malicious or false inputs.

Jason D. Rowley
Jason D. Rowley edited on 15 May, 2020
Edits made to:
Infobox (+1 properties)
Further reading (+3 rows) (+15 cells) (+536 characters)
Documentaries, videos and podcasts (+4 rows) (+16 cells) (+446 characters)
Further reading

Title
Author
Link
Type
Date

Adversarial Machine Learning -- Industry Perspectives

Ram Shankar Siva Kumar, Magnus Nyström, John Lambert, Andrew Marshall, Mario Goertzel, Andi Comissoneru, Matt Swann, Sharon Xia

PDF

February 4, 2020

Adversarial Machine Learning at Scale - Google Research

Alexey Kurakin. Ian J. Goodfellow. Samy Bengio,

Web

2017

Introduction to Adversarial Machine Learning

Arunava Chakraborty

Web

October 16, 2019

Documentaries, videos and podcasts

Title
Date
Link

Adversarial Attacks on Neural Networks - Bug or Feature?

September 10, 2019

Adversarial Machine Learning

November 20, 2019

Generative Adversarial Networks (GANs) - Computerphile

October 25, 2017

Lecture 16 | Adversarial Examples and Adversarial Training

August 11, 2017

Infobox
Is a
Technology
Golden logo
Text is available under the Creative Commons Attribution-ShareAlike 4.0; additional terms apply. By using this site, you agree to our Terms & Conditions.