AI Ethics

Algorithmic bias occurs when a computational system produces outcomes that systematically disadvantage particular groups of people, often along lines of race, gender, age, or other protected characteristics, even when the system was not explicitly designed to discriminate. Bias can enter a system at many points: through historical data that encodes past human prejudice, through the choice of which variables to include as predictors, through the way a target outcome is defined, or through the statistical tradeoffs a model makes when it is optimized for overall accuracy rather than equitable treatment across groups. A model can be highly accurate in aggregate while still treating different subpopulations very differently, which is why researchers evaluate fairness using multiple distinct metrics rather than a single accuracy score. Because algorithmic bias is often invisible inside proprietary software, independent auditing and investigative journalism have played an important role in surfacing it to the public. Understanding algorithmic bias requires distinguishing between bias in the input data, bias in the model's design, and bias in how a system's outputs are ultimately used by human decision-makers.

The most widely cited real-world illustration of algorithmic bias in a high-stakes domain is the COMPAS system, an acronym for Correctional Offender Management Profiling for Alternative Sanctions. COMPAS was developed by the company Northpointe and used by courts across the United States to generate a recidivism risk score intended to help judges make decisions about bail, sentencing, and parole by estimating the likelihood that a given defendant would reoffend. In May 2016, journalists Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner published an investigation for ProPublica titled Machine Bias, which examined COMPAS risk scores assigned to roughly seven thousand people arrested in Broward County, Florida, and then checked which of those people were actually charged with new crimes over the following two years. Their methodology compared the risk scores the algorithm had assigned against actual reoffense outcomes, broken down by the defendants' race, in order to test whether the tool's errors were evenly distributed between Black and white defendants. This approach allowed the reporters to move beyond simply asking whether COMPAS was accurate on average and instead ask whether it was wrong in different ways for different groups.

The ProPublica investigation found a clear racial disparity in the pattern of the algorithm's mistakes. Among defendants who did not go on to reoffend, the tool mislabeled Black defendants as higher risk at a rate of 44.9 percent, compared to 23.5 percent for white defendants, meaning Black defendants who did not reoffend were nearly twice as likely to be incorrectly flagged as future criminals. Conversely, among defendants who did go on to reoffend, white defendants were more often mislabeled as low risk, at a rate of 47.7 percent compared to 28.0 percent for Black defendants who reoffended. After statistically controlling for prior crimes, age, and gender, the reporters found that Black defendants were still 77 percent more likely to be assigned a higher risk score for future violent crime and 45 percent more likely to be predicted to commit any future crime compared to white defendants with similar profiles. Northpointe disputed the analysis, arguing that COMPAS was equally well calibrated for both groups in the sense that a given risk score corresponded to similar reoffense rates regardless of race, which illustrates a deeper problem in algorithmic fairness: a model can satisfy one mathematical definition of fairness, such as calibration, while simultaneously violating another, such as equal false positive rates, and it is mathematically impossible in general to satisfy both simultaneously when base rates differ between groups. This tension between competing, individually reasonable fairness definitions is now a central topic in the technical study of algorithmic bias, and the COMPAS case remains a foundational example used to teach why fairness cannot be reduced to a single number.

As artificial intelligence systems have been deployed across borders in domains ranging from criminal justice to hiring to healthcare, national laws alone have proven insufficient to address the ethical risks these systems pose, because a system built or trained in one country can be deployed to affect people anywhere in the world. This has created demand for international, standard-setting instruments that establish shared ethical expectations for how AI should be designed, deployed, and governed, independent of any single national jurisdiction. Such instruments are typically nonbinding in a strict legal sense but carry significant normative weight because they represent a negotiated consensus among many governments, and they often shape subsequent national legislation, corporate self-regulation, and technical standards. Global governance frameworks for AI generally try to balance encouraging innovation and its benefits against the need to prevent concrete harms such as discrimination, mass surveillance, and the erosion of human autonomy. Because AI ethics touches culture, law, and human rights simultaneously, effective global frameworks require input from an unusually broad range of stakeholders, including scientists, civil society, industry, and governments from every region of the world.

In November 2021, member states of UNESCO, the United Nations Educational, Scientific and Cultural Organization, adopted the Recommendation on the Ethics of Artificial Intelligence, which UNESCO describes as the first global standard-setting instrument on the ethics of artificial intelligence. The Recommendation was adopted by all 193 UNESCO member states, giving it a uniquely broad base of international agreement compared to earlier, more limited AI ethics guidelines that had been issued by individual companies, professional associations, or smaller coalitions of countries. The document was developed through a multi-year, multi-stakeholder drafting process intended to ensure that its principles would be applicable across very different legal systems, economic circumstances, and cultural contexts rather than reflecting the assumptions of any single region. Rather than proposing detailed technical rules, the Recommendation sets out values and principles meant to guide national policy-making, corporate practice, and international cooperation on AI, leaving member states responsible for translating those values into concrete domestic laws, regulations, and institutional practices suited to their own circumstances.

The Recommendation is organized around a small set of foundational values that member states agreed should underpin the ethical development and use of AI everywhere. These include respecting and promoting human dignity and human rights, fostering the flourishing of the environment and ecosystems, ensuring diversity and inclusiveness, and enabling peaceful, just, and interconnected societies. Building on these values, the document sets out a wider set of guiding principles covering areas such as proportionality and preventing harm, safety and security, the right to privacy and data protection, multi-stakeholder and adaptive governance, accountability and responsibility for AI-driven outcomes, transparency and explainability, ensuring meaningful human oversight of automated decisions, environmental sustainability, and fairness together with non-discrimination. Taken together, these values and principles are meant to give governments, companies, and international bodies a common ethical vocabulary for evaluating AI systems, one that echoes concerns raised by cases like COMPAS, such as the need for accountability, transparency, and non-discrimination when automated systems make or inform consequential decisions about people's lives. Because the Recommendation is nonbinding, its practical influence depends heavily on how faithfully individual member states incorporate its values into enforceable national policy, which makes ongoing monitoring and reporting on implementation an important part of its long-term impact.