The mathematical impossibility of perfect fairness refers to the proven result that no AI classifier can simultaneously satisfy three intuitive definitions of fairness: equal selection rates across groups, equal accuracy/error rates across groups, and equal meaning of predictions across groups. Any system that satisfies two of the three must sacrifice the third whenever the underlying base rates differ between groups. The result was surfaced by Nita Farahany in Class 9 of her intro course (September 23, 2025) as a structural feature of AI bias governance debates.
Origin
The impossibility derives from the result published by Kleinberg-Mullainathan-Raghavan 2016 and the parallel Chouldechova 2016 work analyzing the COMPAS recidivism tool. Both showed that the conditions cannot hold together except in degenerate cases (equal base rates or perfect prediction). Farahany presents it not as a technical limitation that has yet to be solved but as a structural feature of fairness governance.
The three fairness definitions
Farahany illustrates the result with a bank that uses AI to approve loans. Historical data shows an 80% loan repayment rate in Neighborhood A and a 60% rate in Neighborhood B. In the example the difference is attributed not to trustworthiness but to systemic factors (fewer job opportunities, worse schools, less inherited wealth), with Neighborhood A standing in for Palo Alto and Neighborhood B for East Oakland. A bank seeking to make its AI "fair" can adopt any of three plausible definitions:
- Equal approval rates — approve 70% of applicants from both neighborhoods. If Neighborhood B faces more financial challenges, this can extend loans to people who cannot afford them, working against the intended beneficiaries.
- Equal accuracy — predict repayment equally well for both groups. Achieving this can mean approving fewer loans in Neighborhood B, which appears discriminatory in approval-rate terms.
- Equal meaning — a 75% repayment prediction means 75% regardless of neighborhood. Holding this constant produces different approval rates between the neighborhoods.
The three cannot be achieved simultaneously when base rates differ. Choosing any two requires sacrificing the third, and different choices reflect different value commitments about which kind of fairness matters most.
Application to NYC Local Law 144
NYC Local Law 144 requires a "bias audit" calculating "impact ratios" — the selection rate or scoring rate for each protected category — but the regulations never specify what ratio counts as acceptable or unacceptable. Farahany frames this as the legislature acknowledging that there are multiple, conflicting definitions of fairness and that the law cannot endorse one without sacrificing the others, rather than as a drafting omission.
The de facto threshold borrowed from the EEOC's 4/5ths rule (impact ratio ≥ 0.80) addresses one specific definition, equal selection rates. Farahany argues it creates predictable gaming behavior, with companies optimizing to ratios just above 0.80 to avoid lawsuits while maximally filtering out protected-class members.
Implications for governance
Farahany draws several consequences for fairness regulation from the impossibility. Any AI fairness regulation must choose which definition it prioritizes, and refusing to choose is itself a choice, usually defaulting to equal selection rates via the 4/5ths rule. Bias-audit frameworks that do not specify a fairness definition allow auditors to select whichever definition makes a system pass. Because demanding all three definitions at once demands the impossible, regulators must instead specify which trade-off they want and accept that the choice will appear unfair under another definition.
Relation to disparate impact doctrine
The three-step disparate impact framework (plaintiff shows statistical disparity → defendant shows business justification → plaintiff shows alternative method without discrimination) implicitly leans on equal selection rates as its baseline definition. Different fairness definitions would produce different disparate-impact findings on the same data.
Trump EO 14281 (April 23, 2025), directing federal agencies to stop enforcing the disparate impact standard, removes the federal floor that had selected this trade-off. States are left to choose their own: California FEHA (October 2025) implicitly chooses equal selection rates by extending vendor liability for systems that produce disparate selection.
Relationships
- surfaced-in: Inside My AI Law & Policy Class 9: When AI Discrimination Happens 1.1 Billion Times (Farahany, September 2025)
- based-on: Kleinberg-Mullainathan-Raghavan 2016; Chouldechova 2016
- related: AI Bias and Discrimination, NYC Local Law 144 (Automated Employment Decision Tools), Mobley v. Workday, Inc.
- constrains: Disparate Impact Doctrine (planned)
- instance-of: AI Governance (umbrella)