A new method to help better detect discrimination in AI and machine learning systems created by academics at the Oxford Internet Institute, University of Oxford, has been implemented by Amazon in their new bias toolkit, ‘Amazon SageMaker Clarify’, for use by Amazon Web Services customers.

In their paper, ‘Why Fairness Cannot Be Automated: Bridging the gap between EU non-discrimination law and AI’ Professor Sandra Wachter, Associate Professor, and Dr Brent Mittelstadt of the Oxford Internet Institute, University of Oxford and Dr Chris Russell, propose a new test for ensuring fairness in algorithmic modelling and data driven decisions, called ‘Conditional Demographic Disparity’ (CDD). The test is significant as it aligns with the approach used by courts across Europe in applying non-discrimination law.

The anti-discrimination test developed by the Oxford AI experts helps users look for bias in their AI systems and is particularly relevant for those seeking help in detecting unintuitive and unintended biases as well as heterogenous, minority-based and intersectional discrimination.

The Oxford test forms part of a wider suite of materials available to Amazon Web Services customers looking to detect bias in their datasets and their models, and for wider support in understanding how their models make predictions.

Associate Professor and Senior Research Fellow, Dr Sandra Wachter said,

“I’m incredibly excited to see our work being implemented by Amazon Web Services as part of their cloud computing offering. I’m particularly proud of the way our anti-bias test can be used to detect evidence of intersectional discrimination, which is an area that is often overlooked by developers of AI and machine learning systems.

“It’s less than a year since we published our paper, ‘Why Fairness Cannot Be Automated: Bridging the Gap Between EU Non-Discrimination’ and it’s very rewarding to see our work having such impact.”

The Oxford experts have also achieved similar success in the field of explainable AI with their counterfactual explanations. Their method makes AI systems explainable and accountable in a meaningful way without the need to fully understand the logic of the algorithm and without infringing on trade secrets or IP rights.  It was embedded by Google into their popular What If interface for TensorFlow and GoogleCloud.

Download the paper ‘Why Fairness Cannot Be Automated: Bridging the Gap Between EU Non-Discrimination’.
Watch Professor Wachter speaking about her co-authored paper.

Notes for editors

About Professor Sandra Wachter

Professor Sandra Wachter is an Associate Professor and Senior Research Fellow in Law and Ethics of AI, Big Data, and robotics as well as Internet Regulation at the Oxford Internet Institute at the University of Oxford. She is also a Fellow at the Alan Turing Institute in London, a Fellow of the World Economic Forum’s Global Futures Council on Values, Ethics and Innovation, a Faculty Associate at The Berkman Klein Center for Internet & Society at Harvard University.

About Dr Brent Mittlestadt

Dr Brent Mittlestadt is a Senior Research Fellow at the Oxford Internet Institute and British Academy Postdoctoral Fellow

About Dr Chris Russell

Dr Chris Russell is a Research Associate at the Oxford Internet Institute and Group Leader in Safe and Ethical AI at The Alan Turing Institute.

About the OII

The Oxford Internet Institute (OII) is a multidisciplinary research and teaching department of the University of Oxford, dedicated to the social science of the Internet.  Drawing from many different disciplines, the OII works to understand how individual and collective behaviour online shapes our social, economic and political world. Since its founding in 2001, research from the OII has had a significant impact on policy debate, formulation and implementation around the globe, as well as a secondary impact on people’s wellbeing, safety and understanding. Drawing on many different disciplines, the OII takes a combined approach to tackling society’s big questions, with the aim of positively shaping the development of the digital world for the public good.