Why Algorithmic Fairness is Elusive
In 2016, Google photos classified an image of two African-Americans as”gorillas.” Two years after, Google had yet to perform over remove the word”gorillas” from its own database of classifications. In 2016, it was shown that Amazon was disproportionately offering one-way shipping to European-American consumers. In Florida, algorithms used to recommend detention and parole decisions on the grounds of risk of recidivism were shown to possess a higher error rate among African-Americans, for example African-Americans were prone to be incorrectly suggested for detention who would not move on to re-offend. When translating from a speech with gender-neutral pronouns, and into languages with gendered pronouns, Google’s word2vec neural network injects sex stereotypes in to dictionary, such that pronouns become”that he” when together with”doctor” (or”boss,””financier,” etc.) but become”she” when interpreted together with”nurse” (or”homemaker,” or”nanny,” etc.).
These issues arise from a constellation of triggers. Some are underlying social roots; should you train a machine learning algorithm on information created by biased humans, you’ll get a biased algorithm. Some are simply statistical artifacts; if you train a machine learning algorithm to get the ideal match for the total population, to the extent that minorities are distinct in a relevant way, their classifications or recommendations will necessarily have poorer match. And a few are a blend of both: biased humans lead to biased calculations which make recommendations that reinforce unjustified stereotypes (for example, harsher policing of poorer neighborhoods contributes to more crime reports in those neighborhood. More offense reports activate policing analytics to recommend deploying more cops to all those neighborhoods, and voila! You have a nasty feedback loop). The problem is it’s not at all clear how to create calculations honest. Figuring out the best way to define and measure fairness reflect broader ethical conversations taking place now.
I have recently had the pleasure of interviewing Sharad Goel, the executive manager of the Computational Policy Lab at Stanford. We got to chat about some of his work in algorithmic fairness. Specifically, we got to discuss the perks and shortcomings of 3 sides of this debate over how to conceptualize fairness algorithmically. Technical people can get a fuller treatment of the discussion in this newspaper , but I’m going to try and boil down it.
Three conceptualizations of fairness
Certain group labels should be off limits. This manner of thought maintains that calculations should not be permitted to take certain protected categories into consideration when making forecasts. For example, in this opinion, algorithms which are utilized to predict loan qualifications or recidivism shouldn’t be allowed to base predictions off of race or gender. This strategy to attain fairness is straightforward and easy to comprehend. Distinguishing between acceptable and unacceptable proxies of protected categories. When such classes are removed from an algorithm, the statistical variance explained by these protected categories tends to slide into other available factors. For example, while race might be excluded from loan software, zip code, that tends to be highly associated with race, may take on a greater predictive burden in the model and mask discrimination. For all intents and purposes, zip code becomes the new race variable. It’s challenging and problematic which proxies are illegitimate substitutes for secure categories, and that are acceptable, different factors. This fuzzy line brings us to another difficulty with making certain labels”off-limits.”
- The social (and sometimes private ) costs are high. Protected categories often can make a meaningful impact on the behaviours that the algorithms are made to predict. For instance, it is commonly understood that insurance premiums are higher for male drivers, because male drivers really do accounts for much more of the whole premiums. Eliminating gender from such calculations could cause auto insurance premiums to fall for guys, but it would increase the rates for girls. Whether women should be required to pay for more than their share of danger, such that sex is removed from danger algorithms, is debatable. In short, while this could generate exact equality, this seems to be missing the mark of what is proportionally equitable. Some might argue this approach is actually unfair.
Higher stakes can be seen in criminal justice settings. Removing protected categories like gender or race out of algorithms made to predict recidivism deteriorates the efficiency of the algorithm, meaning more individuals of lower actual risk are detained, and more people of greater real risk are let free. The consequences are that more crime occurs in general, and among communities that are already experiencing greater crime in particular. To see this, keep in mind that the majority of violent crime occurs between people who know each other. And so, communities already plagued with violent offenses may stand to go through the further re-offending violent crimes when algorithmic efficiency is slashed (when protected, but still explanatory, groups are disallowed).
Many men and women agree (including the law) that basing conclusions on protected classes when there isn’t any concrete rationale is morally reprehensible. The challenging part is when employing these protected categories appears to efficiently cut down harmful outcomes. Off this trade has led a few to take different approaches to defining equity. Is there a means to maximize predictive accuracy (allowing inclusion of purposeful protected classes ), while still being honest?
Algorithmic performance must work equally well across specific classes. Rather than ignoring protected categories such as race and sex (e.g. being color or sex blind), this method of equity rather argues that signs of an algorithm’s operation ought to be equivalent across the protected categories. For example, an algorithm that classifies offenders as either low or high risk of re-offending should make prediction errors equally for white and black offenders. This strategy is less intuitive than the color-blind strategy, but at least theoretically allows the algorithms to be more efficient in their predictions, also has the added benefit of preventing catchy decisions calls about that proxies (e.g. zip code as a crude substitute for race) are and aren’t acceptable for inclusion in calculations.
Still, this strategy is imperfect. To see why, it’s important to see that different groups of people may represent different populations–inhabitants with different average scores, deviations, skew, kurtosis, etc. (see picture above, and imagine trying to get 1 algorithm to execute equally for each category curve with the exact same cutoff threshold) . Generally, once we talk about fairness, we need all people, irrespective of their group membership, to be held to the very same criteria. But if the exact same cutoff thresholds are used for different inhabitants, predictive ability and error levels are most likely to differ across classes –this is simply the natural outcome of how statistics functions. If government law compels corporations to turn out algorithms which maintain the exact same performance across protected classes, corporations and institutions are inclined to discriminate under the obscuring power of statistical wizardry and employee NDAs.
They generally have two options: 1. Reduce the quality and efficiency of their calculations by toying with the code so that algorithmic performance is equivalent across classes (this alternative introduces the prospect of harm discussed previously, like releasing criminals with real, high risk scores), or 2. Corporations could embrace distinct algorithmic thresholds for various populations, for example cutoffs are distinct for different classes (sexes, races, people of different sexual orientations, etc.). But clearly this seems to break with notions of equity, and is usually morally frowned upon and considered illegal (with a notable exception being some thing like affirmative action). Negative effects of forced equalization of algorithmic operation across classes are not just theoretical–these negative impacts are documented, for example, at recidivism risk score databases as well as databases predicting likelihood of police finding contraband among black and white citizens.
Algorithmic scores should represent exactly the same things across members of distinct groups. A third way to attaining equity in algorithms is to ensure that an algorithm’s scores imply equivalent things across protected categories (for instance a woman receiving a risk rating of X on her own insurance program, should have comparable insurance premiums as a man who also receives a hazard score of X on his insurance policy application)? On the surface, it might seem that this approach is becoming at what we want–it appears fair. The problem is that it cannot guarantee fairness in the presence of intentionally discriminatory actions, and thus regulation of algorithms on the basis of the definition of equity will still leave room for obscured discriminatory treatment. There are at least two ways this can happen:
- Proxies (such as zip code for race) can still be used to gerrymander population scores below or above a algorithm’s cutoff thresholds.
- As mentioned above, different groups will have different statistical risk curves. If qualitative scores are discretized (for instance, substituting”high,””moderate,” or”non” labels in place of an individual’s precise score) within groups, these differences at the real risk curves can conceal different group cutoffs while maintaining the veneer that people labeled”high” hazard re-offend, default, and also get in car crashes at comparable rates across protected (race, sex, etc.) groups. For example, in the image above, assigning an individual a”high,””moderate,” or”low” hazard label on the grounds of their within-group percentile will effectively yield distinct group cutoff thresholds, while possibly maintaining the exact same algorithmic operation across those labelled”elevated” hazard for each protected group.
Though it seems like using these techniques would be somewhat rare for B2C businesses, who’d more often than not endure the loss of profits by discriminating in such manners, incentives still exist for both B2B corporations. For example, ad-matching companies have incentives to push certain groups above and below cutoff thresholds so as to justify ad targeting on the basis of protected classes. It’s not difficult to imagine political campaigns or lobbyists being drawn to the energy of those methods to influence public opinion among tactical subgroups while leaving behind few breadcrumbs, and convoluted breadcrumbs during that. (I am just saying, if US senators couldn’t understand Facebook’s business version , my religion in their understanding this issue is. . .well it is bad.)
Each strategy to algorithmically defining equity has its own weaknesses and strengths. I think what is most troubling is not so much the flaws that every approach faces, but instead these approaches are basically incompatible with one another. We can’t dismiss protected classes while using shielded classes since the baseline to detect equity. And we can not demand similar algorithmic error rates while demanding that similar danger scores really do involve similar results among groups. The race remains on to define fairness algorithmically. But my background in ethical psychology also gives me pause. Democrats, Republicans, and Libertarians can’t agree on what is fair, and I think that it’s too optimistic to treat algorithmic equity such as a mathematical, computer science issue. The problem isn’t solving some complex statistical rubix cube, so much as it is attempting to attest Plato’s perfect form of equity on a cave wall that’s only capable of capturing shadows. It’s hard to predict which alternatives we’ll adopt, and what the prices will be when those options socialize with regulatory and financial incentives. Algorithmic fairness is, at its heart, a socio-moral problem.