Participation

Please attend class regularly and participate (10% of our grade). This is a “seminar style” course. You are expected to take an active role in the discussions. Please study the assigned materials before class and be ready to discuss them.

Assignments

In addition, you should write ten Pass/Fail précis as well as three graded essays or a research paper (90% of your grade).

Wk 1 - 8/19

Introduction

The question of algorithmic fairness is technical (how do algorithms work?), practical (how do algorithms impact people’s lives?) and theoretical (what does “fairness” mean?). On the technical side, we will survey key concepts, such as algorithm, machine learning, supervised v. unsupervised learning, regression, classification v. clustering. This survey will be informal. Those who wish to go deeper may consult the additional materials. On the practical side, we will read stories about people whose lives were affected, often adversely, by algorithms. On the theoretical side, we will brainstorm ideas about what we mean by “fairness” and how an algorithm can be unfair. We will explore several definitions of fairness more deeply later in the course.

Key readings and materials

• Class notes Wk 1

• O’Neil, Weapons of Math Destruction - Introduction

• Kearns and Roth, The Ethical Algorithm - Chp. 1

• A Gentle Introduction to Machine Learning

• Eubanks, Automating Inequality - Chp. 4

Additional readings and materials

• Rogers and Girolami, A First Course in Machine Learning, Chp. 1

• Nielsen, Neural Networks and Deep Learning - Chp. 1

• What Is Backpropagation Really Doing?

• Impact Evaluation of a Predictive Risk Modeling Tool for Allegheny County’s Child Welfare Office

Wk 2 - 08/26

ProPublica v. Northpointe

To examine more closely the interplay between technical, theoretical and practical questions, we will look at a case study, the algorithm COMPAS used in criminal justice to help judges make decisions about bail, preventative detention and sentencing. The website of investigative journalism ProPublica argued that COMPAS is biased against black people. Northpointe (now Equivant), the company that designed COMPAS, rejected the accusation claiming that COMPAS is racially fair. They both provided numbers to back up their claims. So is COMPAS racially biased or not? The crux of the matter here seems to be a disagreement about the meaning of “fairness”.

To better understand this debate, you should be familiar with a bif of probability theory and machine learning, in particular, notions such as false positive, false negative, sensitivity, specificity, Area Under the Curve (AUC) and confusion matrix. We will review these notions as needed.

Key readings and materials

• Class notes Wk 2

• Machine Learning Fundamentals:

  • Sensitivity and Specificity
  • ROC and AUC
  • Confusion Matrix

• ProPublica, Machine Bias: There’s Software Used Across the Country to Predict Future Criminals. And It’s Biased Against Blacks.

• Dieterich, Mendoza and Brennan, COMPAS risk scales: Demonstrating Accuracy Equity and Predictive Parity Performance of the COMPAS Risk Scales in Broward County

Additional readings and materials

• ProPublica, How We Analyzed the COMPAS Recidivism Algorithm

  • Exploring the ProPublica COMPAS data

• Northpointe, Practitioner’s Guide to COMPAS Core

  • Sample Risk Assessment COMPAS

• Flores, Bechtel and Lowenkamp. False Positives, False Negatives, and False Analyses: A rejoinder to “Machine Bias”

• Parole Denied: One Man’s Fight Against a COMPAS Risk Assessment

Wk 3 - 9/2

(Mis)guided by Data

Let’s take a step back and examine the basis of algorithmic decision-making: data. Though incomplete and never truly conclusive, data can guide our decisions, often benefitting people. For example, data have been playing a key role in the response to the COVID-19 Pandemic (see,among others, Abedin et al, Using Data to Inform the COVID-19 Policy Response)

Another example, in the context of criminal justice, is the data-driven Public Safety Assessment (PSA) algorithm, now widely used in several jurisdictions in the United States to help judges make pre-trial decisions. According to a report from New Jersey, pre-trial jail population descrease significantly after PSA was adopted. The ACLU of New Jersey endorsed the use of the PSA algorithm because it has the potential to end the pre-trial system of money and bail that disproportionately harms the poor.

Although data can be helpful to make decisions, they can also be biased, partial or incomplete. Inadequate data can be harmful especially for already disadvantaged minorities. An important problem is the use of “proxies” in the historical data. That is, the algorithm is not trained on the actual outcomes (which often cannot be known), but on proxies of the actual outcome (say ‘arrest’ is used as proxy for ‘crime’). We will see a few examples of biased, partial and incomplete data and their harmful effects on people.

Key readings and materials

• Class notes Wk 3 (if interested, check out SVM with ProPublica COMPAS Data)

• Kearns and Roth, The Ethical Algorithm - Chp. 2 (only pp. 57-64)

• Hardt, How Big Data Is Unfair

• Technical Flaws of Pretrial Risk Assessments Raise Grave Concerns

• Perez, Invisible Women: Data Bias in a World Designed for Men - Chp. 8

Additional readings and materials

• Caliskan, Bryson, and Narayanan, Semantics Derived Automatically from Language Corpora Contain Human-Like Biases

• Hart and Cooke (2013), Another Look at the (Im-)precision of Individual Risk Estimates Made Using Actuarial Risk Assessment Instruments

  • Imrey and Dawid (2015), A Commentary on Statistical Assessmentof Violence Recidivism Risk

• Obermeyer et al (2019), Dissecting Racial Bias in an Algorithm Used to Manage the Health of Populations

• Lazer, Kennedy, King, and Vespignani, The Parable of Google flu: Traps in Big Data Analysis

• Buolamwini, Algorithms Aren’t Racist. Your Skin Is just too Dark

Wk 4 - 9/9

Feedback Loops

Another problem with data is that decisions based on inadequate data magnify errors on a large scale especially when they are automated. Even unbiased data can produce pernicious feedback loops. We will exmaine examples of feedback loops in algorithms used in education and policing.

Key readings and materials

• Class notes Wk 4 (if interested, check out my note on Simulating Feedback Loops in Predictive Policing)

• O’Neil, Weapons of Math Destruction - Chps. 3, 4 and 5

• Lum and Isaac, To Predict and Serve?

Additional readings and materials

• Ensign, Friedler, Neville, Scheidegger and Venkata, Runaway Feedback Loops in Predictive policing - video

• Harcourt, Against Prediction - Chp. 5

• Brantingham et al (2018), Does Predictive Policing Lead to Biased Arrests? Results From a Randomized Control Trial

First essay due

Wk 5 - 9/16

Meanings of Fairness

We have not yet asked the fundamental questions. What does it mean for algorithmic decisions to be fair? How is fairness different from accuracy? Computer scientists have formulated several different metrics of algorithmic fairness. By one count, there are as many as 21 definitions. Three of the most important definitions out there are called: demographic parity, classification parity and predictive parity. We will examine these definitions and their relationship to accuracy.

Key readings and materials

• Class notes Wk 5

• Kearns and Roth, The Ethical Algorithm, Chp. 2

• Corbett-Davies and Goel. The Measure and Mismeasure of Fairness: A Critical Review of Fair Machine Learning - video

Additional readings and materials

• Narayanan, 21 Fairness Definitions and Their Politics

• Barocas, Hardt and Narayanan, Fairness in Machine Learning - Chp. 2

Wk 6 - 09/23

Impossibility Theorems

It turns out it is mathematically impossible, under realistic conditions, for an algorithm to satisfy different conceptions of fairness at the same time. These impossibility theorems led scholars to consider trade-offs between different conceptions of fairness or be skeptical toward existing definitions.

Q&A with Deborah Hellman

The first part of the class will consist in a Q&A session with Deborah Hellman, Professor of Law at the University of Virginia School of Law and the author of one of the readings assigned for today. Please come prepared to ask questions.

Key readings and materials

• Class notes Wk 6 (also check out Chouldechova’s Impossibility: In Pictures)

• Hellman, Measuring Algorithmic Fairness

• Chouldechova, Fair Prediction with Disparate Impact

Additional readings and materials

• Grant, Is It Impossible to Be Fair?

• Hedden, On Statistical Criteria of Algorithmic Fairness - video

• Berk, Heidari, Jabbari, Kearns and Roth, Fairness in Criminal Justice Risk Assessments: The State of the Art

• Kleinberg, Mullainathan and Raghavan, Inherent Trade-offs in the Fair Determination of Risk Scores - video

Wk 7 - 09/30

Fair Risk Assessment

The debate about algorithmic fairness is nothing new. Since the development of statistical methods in insurance and banking, claims of racial and gender discrimination have been advanced against risk assessment tools used to price loans and insurance policies. We will review this history to understand analogies and differences between fair risk assessment in the 20th century and algorithmic fairness in the 21st century. We will also look at how ordinary people make distinctions between fair and unfair data-based risk predictions.

Key readings and materials

• Class notes Wk 7

• Ochigame, The Long History of Algorithmic Fairness

• Kiviat, Which Data Fairly Differentiate?

Second essay due

Wk 8 - 10/7

Structural Unfairness

The definitions of algorithmic fairness we have seen so far describe the extent to which actual rates of error are unevenly (and thus unfairly) allocated across different socially relevant groups. Scholars in computer science have also used more sophisticated definitions to formulate a counterfactual or causal definition of algorithmic fairness. These definitions attempt to capture the structural dimension of racial discrimination (as in the expression “structural racism”). Other scholars in law, sociology and philosophy have argued that these attempt are unsatisfactory.

Q&A with Lily Hu

The first part of the class will consist in a Q&A session with Lily Hu, a philosopher at Harvard interested in algorithmic bias and how the social sciences measure the causal effect of race. She is the author of some of the readings assigned for today. Please come prepared to ask questions.

Key readings and materials

• Class notes Wk 8

• Hu, What is ‘Race’ in Algorithmic Discrimination on the Basis of Race?

• Hu, Disparate Causes, Part I and Part II

• Kohler-Hausmann, Eddie Murphy and the Dangers of Counterfactual Causal Thinking About Detecting Racial Discrimination

Additional readings and materials

• Barocas, Hardt and Narayanan, Fairness and Machine Learning - Chp 5: Causality

• Kusner, Loftus, Russell and Silva, Counterfactual Fairness

• Chiappa and Gillam, Path-specific Counterfactual Fairness

Wk 9 - 10/14

Structural Unfairness (cont’ed)

We will continue our discussion of the structural approach to algorithmic fairness.

Q&A with Annette Zimmermann

The first part of the class will consist in a Q&A session with Annette Zimmermann, a political philosopher at the University of York working on the ethics of artificial intelligence and machine learning and the author of one of the readings assigned for today. Please come prepared to ask questions.

Key readings and materials

• Class notes Wk 9

• Zimmerman & Lee-Stronach, Proceed with Caution

Additional readings and materials

• Green, The False Promise of Risk Assessments: Epistemic Reform and the Limits of Fairness

Wk 10 - 10/21

Algorithmic Equal Protection

Algorithmic fairness is closely connected with antidiscrimination and equal protection jurisprudence, specifically the notions of “disparate treatment” and “disparate impact”. Any differential treatment that is due to racial animus or discriminatory intent is clearly illegal. But the Supreme Court has allowed the use of race in specified contexts, for example, in college admissions for the purpose of fostering diversity (see e.g. Fisher v. University of Texas (2016)). Evidence of disparate impact against a protected group is enough to make a prima facie case of discrimination. This applies to sectors such as employment and housing (see e.g. Title VII of the Civil Rights Act of 1964 and Hazelwood School District v. United States (1977)). The criminal justice system, however, seems exempt. An elaborate statistical analysis – showing that death penalty decisions in Georgia disproportionately targeted African Americans, controlling for several variables – was not enough to convince the Court that the system violated equal protection (see McClensky v. Kemp (1987)). It is worth thinking about whether the legal notions of disparate treatment and disparate impact can help to understand what algorithmic fairness consists in.

Key readings and materials

• Class notes Wk 10

• Barocas and Selbst, Big Data’s Disparate Impact

Additional readings and materials

• Any court opinion linked above

• Owes, Another Equality

Wk 11 - 10/28

Fairness v. Welfare

We now analyze the question of algorithmic fairness from the perspective of utilitarianism. Utilitarianism, roughly, says that the right action is one that maximizes social welfare. If utilitarianism is the correct ethical theory, algorithms should maximize social welfare and this may require to treat people unfairly. Or does it?

Key readings and materials

• Class notes Wk 11

• Huq, Racial Equity in Algorithmic Criminal Justice

Additional readings and materials

• Becker, The Economic Way of Looking at Life (mostly focus on the sections about discrimination and crime)

• Kaplow and Shavell, Fairness Versus Welfare

• Bentham, Introduction to the Principles of Morals and Legislation, Chps. I–II

Wk 12 - 11/4

Equality

Against utilitarianism, John Rawls, an influential political philosopher of the 20th century, argued that society must give everyone equal opportunities to develop their talents and that inequalities in the distribution of goods can be tolerated only if they work to the advantage of the worst-off in society. What would algorithmic fairness look like in light of Rawls’s view about equal opportunities and distributive justice? Other influential philosophers, Elizabeth Anderson and Tim Scanlon, have also defended egalitarian views. What would algorithmic fairness look like in light of these egaliatarian views?

Key readings and materials

• Class notes Wk 12

• Anderson, What is the Point of Equality?

• Binns, Fairness in Machine Learning: Lessons from Political Philosophy

• Rawls, Justice as Fairness, Part 2, Sections 12-18

Additional readings and materials

• Scanlon, Why Does Inequality Matter?

• Kolodny, Why Equality of Treatment and Opportunity Might Matter

• Heidari, Loi, Gummadi and Krause, A Moral Framework for Understanding fair ML Through Economic Models of Equality of Opportunity

Wk 13 - 11/11

Veterans Day, No class

Wk 14 - 11/18

Workshop

Students will workshop their research papers by giving short presentations accompanied by either a handout or slides and followed by Q&A. Each presenter will have 20 minutes to be split between the presentation itself and Q&A at the discretion of the presenter. The list of presenters is below:

3:00 | Filipp | Fairness, Welfare, and Algorithms

3:20 | Shade | Beyond Market Logic: Algorithmic Fairness in a Social World

3:40 | Ben | Amnesiac Supply Chains: Database Aggregation and Algorithmic Fairness

4:00 | Harits | Two Constraints on Using Criminal Algorithms

4:20 | 10 minutes break

4:30 | Emerson | The Highest Degree of Fairness is Not Necessarily the Most Fair

4:50 | Matthew | Algorithmic Gullibility

5:10 | Keith | Algorithmic Judicial Ethics

5:30 | Angela | Algorithms and individuals

Wk 15 - 11/25

Thanksgiving, No Class

Wk 16 - 12/2

Q&A with Seth Lazar and Jake Stone

In the first part of the, Seth Lazar and Jake Stone from the Australian National University will present their ongoing project ‘On the Site of Predictive Justice’

Individualized Decisions

Since algorithms often rely on statistical generalizations and correlations, some worry that algorithms disregard the individual circumstances of each person. Do people have a right to be treated as individuals? What would that mean?

Key readings and materials

• Class notes Wk 16

• Lazar and Stone, On the Site of Predictive Justice [check Canvas for the PDF]

• Renee Bolinger, Algorithms and the Individual in Criminal Law

Additional readings and materials

• Lippert-Rasmussen, We Are All Different

• Beeghly, Failing to Treat Persons as Individuals

• Eidelson, Treating People as Individuals

Third essay or research paper due