Author: Nita Farahany Source: https://nitafarahany.substack.com/p/when-invisible-algorithms-judge-you Published: November 5, 2025
A Substack essay by Nita Farahany, published November 5, 2025, recounting Class 19 of a 27-class AI law and policy course. The class examines algorithmic discrimination through three competing legal and ethical frameworks and three national theories of governance, using an employment-discrimination lawsuit and a live classroom pricing demonstration as anchors.
Anchors
The class is built around two examples. The first is Arshon Harper v. Sirius XM Radio and iCIMS, a case involving a plaintiff described as Black, over 40, and with disabilities who received 150 IT-job rejections over two years; the class-action complaint was filed in the U.S. District Court for the Eastern District of Michigan on August 4, 2025.
The second is a real-time classroom demonstration of dynamic pricing. Students searched the same RDU→NYC flights at the same time from the same physical room and saw prices of $27, $83, $147, and $326. Farahany describes the algorithm as judging each student on device type, browsing history, location, search patterns, and inferred income. The essay extends the example to employment, housing, credit, and insurance, noting that, unlike a flight search, a person cannot comparison-shop a rejection.
Three frameworks for the same harm
Farahany frames algorithmic discrimination as "the same crime scene" viewed by three different "detectives," each representing a distinct framework with its own remedy logic and failure mode.
The Prosecutor (US civil rights law) is reactive, focused on individual remedy, and oriented toward preserving innovation, working through Title VII and Section 1981. Cited precedents include Louis v. SafeRent ($2.275M, November 2024), Mobley v. Workday (class certified May 2025), and California FEHA vendor-as-agent regulations (October 2025). The stated failure mode is that Harper still needs three to five years of litigation and expert witnesses he cannot afford while the system continues operating.
The Forensic Expert (equity-centered design) is proactive, centers affected communities, and accepts slower innovation. The essay cites Daryl Lim's framing of "equity, and the rule of law as yardsticks of socially responsible AI" and the WEF 2022 Equity & Inclusion Blueprint. It poses four framework questions: (1) Whose experience mattered during development? (2) Who bears the burden when the system gets it wrong? (3) What fallback mechanisms exist? (4) Does the system perpetuate or disrupt historical patterns? The stated failure mode is that this approach is not legally enforceable and carries no penalty for ignoring it.
The International Inspector (human rights norms) emphasizes proactive duties and dignity over efficiency, grounded in the UN Guiding Principles on Business and Human Rights, with 193 member states adopting a March 2024 resolution on AI lifecycle human rights. The essay notes that the State Department's July 2024 Risk Management Profile maps to the NIST AI RMF (Govern/Map/Measure/Manage). The stated failure mode is that these norms are aspirational rather than binding, with no enforcement mechanism, and that Harper cannot sue under the UNGPs.
In support of the reactive-remedy critique, the essay cites an observation attributed to Dobbin and Kalev: "When victims of discrimination sue, the legal system rarely protects them from retaliation and rarely rights employers' wrongs." Farahany characterizes civil rights enforcement as a reactive remedy available to those who can afford the journey.
Three national theories of governance
The class also contrasts three country-level approaches. The United States is characterized as reactive individual remedy: free markets drive innovation, harm is awaited and then individually remedied, and Harper's suffering is tolerated as a cost of innovation. The European Union is characterized as proactive prevention: fundamental rights require protection before violation, the EU AI Act mandates pre-deployment risk assessments and conformity assessments, and slower innovation and regulatory burden are tolerated. China is characterized as state control: social stability and harmony take priority over individual rights, state expertise determines acceptable use, and national interests take priority over commercial ones.
Conclusion
Farahany closes on what she calls an intellectual-honesty point: "There is no objectively correct framework. Each reflects different beliefs about the role of law (reactive vs. proactive), the nature of rights (individual vs. collective), the balance between innovation and protection, and whether prevention or remedy should be prioritized." She frames the choice as a question not of which framework is right but of what one is willing to sacrifice.
Relationships
- part-of: Nita Farahany intro course series (Class 19 of 27)
- related: Harper V Sirius Xm (planned), AI Bias and Discrimination, Three Frameworks Ai Discrimination (planned), Dynamic Pricing Ai (planned)
- previous: Inside My AI Law & Policy Class 18: Data Privacy in an AI World (Farahany, November 2025) next: Inside My AI Law & Policy Class 20: The EU AI Act's Reality Check (Farahany, November 2025)