Show Your Workings

Author: Mike Lloyd | Date: 31 Jan 2019

How can trust in algorithmic decisions be regained?

2-minute explainer

Al algorithms are increasingly being used to make decisions that affect us. 

  • In France a centralised algorithm matches teachers to jobs across the country. 
  • Algorithmic decisions increasingly help decide which candidates to hire.
  • Lenders use algorithms to decide which applicants to accept and what interest rates to set.
  • A number of states in the USA use algorithms to make Criminal Justice decisions on sentencing, bail and recidivism. 
  • Decisions about medical treatments are increasingly being informed by algorithms.

Many of these algorithms are embedded in AI.

But people don’t trust AI1.  

Nearly 3⁄4 of consumers want to know more about how their data is used (CBI). 

GDPR requires that organisations must provide meaningful information about decision-making logic. 

But is GDPR enough to win back trust?

How can people gain trust in AI & algorithm-based decisions?

It should be possible to check:

  1. The basis for the algorithm
  2. Its past performance
  3. The reasoning behind its current claim
  4. Any uncertainty behind its current claim
  5. Explanations should be open to different levels of expertise

Case study – the NHS’s ‘Predict’ is a system for women choosing treatment for breast cancer. 

Predict make their workings clear on 4 different levels, and the full implementation is available on GitHub.

Predict clearly explains the maths behind the algorithms.

Explainable AI (XAI) aims to enable human users to understand, appropriately trust, and effectively manage artificially intelligence3.

New XAI methods aim to produce more explainable models, while maintaining a high level of learning performance3.

The challenge ahead is to make AI and algorithmic decision-making simple enough for non-experts to understand.

References 

1.https://www.scientificamerican.com/article/people-dont-trust-ai-heres-how-we-can-change-that/

2.The Information Commissioners Office – https://ico.org.uk/for-organisations/guide-to-the-general-data-protection-regulation-gdpr/individual-rights/rights-related-to-automated-decision-making-including-profiling/ 

3. DARPA XAI – https://www.darpa.mil/program/explainable-artificial-intelligence

Acknowledgements

Alan Turing Institute Lecture: Be prepared to show your working!

Professor Sir David Spiegelhalter, Statistical Laboratory at the University of Cambridge