AI Policy Congress – Part 2 Democratizing AI through Transparency and Education

2019-02-01 - 9 minutes read

Written by Natalie Lao

Demystifying Machine Learning for Regulators

On Wednesday, January 16, Professor Hal Abelson and I led a Machine Learning (ML) workshop for members of the Organisation for Economic Co-operation and Development (OECD) Artificial Intelligence Expert Group (AIGO), where we aimed to demystify ML and discuss challenges to developing effective public policy in the space. As a PhD student working on ML research and education, I was looking forward to hearing the perspectives of people who would be in charge of crafting the AI Policy Principles, which will represent consensus views of over 30 countries.

I had two takeaway messages for participants:

  1. In order to ensure fairness in ML technology, everyone needs to be able to participate in its creation and in application decisions instead of solely relying on technical reviewers and legal oversight.
  2. Anyone can understand ML with the right learning tools — it is not difficult to support this level of engagement in ML policy and regulation.

Since the goal of our work is to democratize ML, both the access to, and use of, our teaching materials are available at the bottom of this post. In the rest of this post, I will briefly outline the format of the workshop and give my view on the most salient themes that I felt came out of our discussion.

Machine Learning Workshop Part 1: Understanding the Basics

Most of the participants were policy experts, and Hal and I spent the first part of the workshop giving an overview of how ML algorithms work (by crunching tons and tons and tons of data to populate weights in a network that represent some pattern from the data). Participants also experimented with a live demonstration in small groups where they could train an image classifier through their laptop’s webcam using a Google-developed application called Teachable Machine. The discussion that ensued focused on three separate topics: interpretability, privacy, and transparency. Participants found that the algorithm may not necessarily be “learning” the patterns they meant for it to learn with the training images they gave each class, but may actually be picking up on some other patterns.

For example, one group wanted to train Class 1 as one person and Class 2 as another, but found out that the Teachable Machine had actually learned the difference in distances of each face from the webcam instead. It’s sometimes very hard to decode the actual patterns in the data that ML algorithms are “learning” during training, especially in large neural networks with thousands of parameters. The group training Teachable Machine had thought that the differences in facial structure would be the most salient features that the neural network would pick up on, but through lots of testing they found that it had actually learned something else.

An image showing the Teachable Machine interface in use.
Screenshot showing the Teachable Machine interface.

Participants then asked how developers can tell what kind of “reasoning” a neural network goes through during its computations to reach its decisions. There is currently no good answer to this —interpretability in deep learning networks is still a largely unsolved problem. This then begged the question: If a ML algorithm cannot explain itself, testify in court, or be cross-examined, what does the chain of responsibility look like?

Imagine a case where a police body camera misclassifies an innocent man as a criminal, which starts a sequence of events leading to his death. In this scenario, which parties will take responsibility? The police officers? The police department? The body camera company? The software developers and testers? The regulators? Will society come to accept some amount of potentially terrible false-positives in exchange for higher overall accuracy?

Several participants also noted privacy and data control concerns during the Teachable Machine exercise: “How do we know that this isn’t taking pictures of our face and giving it to Google somewhere? Or using our metadata?” The computer scientists in the room explained that they could look at the code to ensure that everything was happening in their browser and that no data was exiting their laptops, but participants were rightly unsatisfied. This was a call for transparency and a reminder for developers to make products such that people without any coding background can tell if and how their data is being used.

Machine Learning Workshop Part 2: A Discussion on Fairness and Bias

The second part of the workshop focused on fairness and bias in AI. As seen throughout the AI Policy Congress the day before the workshop, these aspects of AI have an impact on the deployment of AI across many contexts that impact individuals’ lives, from transportation to healthcare to criminal justice. There are many concerns regarding regulation (how to regulate, how much to regulate, should we regulate…), but it is difficult to set specific rules and guidelines without enough understanding of the technology. Our workshop dug deep into the technical weeds on a few famous case studies of bias in AI and possible ways forward.

Schematic showing the machine learning process.
The machine learning process. Image taken from Natalie Lao’s and Harini Suresh’s presentation slides, available at the bottom of this post.

While regulation is often geared at the output of an ML product, we should be looking at its whole lifecycle, starting with the data used to train its underlying algorithm. ML engineers (each with their own biases) make decisions at each step in the creation of a ML algorithm. Many people have thought of ML regulation as an end-process: once all the work is done, what evaluations can we make? However, we argue that checkpoints for bias and fairness should be added to every part of the ML production lifecycle – in the same way that we require many safety checks throughout the design and construction of a  bridge, we should have a similar process to check for bias and fairness in ML products.

Workshop participants noted definitional issues that have been plaguing ML engineers as well: how does one define fairness, nationally and internationally, and what are the ramifications for fixing or adjusting models as a result? Maybe, instead of defining fairness (which could be different based on culture, community, and situation), policymakers should focus on enforcing transparency as a way to distinguish “a poorly-performing system from a poorly-performing society,” as one participant described.

Get the Machine Learning Workshop Materials

As a step toward increasing transparency, and to hopefully foster more discussion along the lines of the workshop, we have included our teaching materials below:

Learn More About the MIT AI Policy Congress

This post is part of a series on the first MIT AI Policy Congress, edited by Grace Abuhamad. Read the rest of the series on the IPRI Blog.