32 lines
5.6 KiB
Markdown
32 lines
5.6 KiB
Markdown
---
|
||
title: "Algorithmic bias"
|
||
chunk: 4/13
|
||
source: "https://en.wikipedia.org/wiki/Algorithmic_bias"
|
||
category: "reference"
|
||
tags: "science, encyclopedia"
|
||
date_saved: "2026-05-05T16:31:03.393915+00:00"
|
||
instance: "kb-cron"
|
||
---
|
||
|
||
Data bias: Training data may under‑represent certain groups (e.g., minority populations), contain historical inequalities, or be collected in skewed ways, leading the model to perform worse or behave unfairly for those groups.
|
||
Label bias: Human‑provided labels can encode subjective judgments or prejudices (for example, what is labeled as "risky," "toxic," or "qualified"), so the model learns and amplifies those judgments.
|
||
Measurement bias: Proxies or measurements used for important concepts (like "creditworthiness" or "job performance") may be noisy or systematically distorted for some groups, which then distorts predictions.
|
||
Algorithmic bias: Even with relatively balanced data, modeling choices (loss functions, thresholds, optimization objectives) can prioritize overall accuracy over fairness, leaving some subgroups with consistently worse outcomes.
|
||
Deployment bias: A model used outside the context it was designed for (e.g., a model trained on adults applied to children, or one trained in one country deployed in another) can generate biased results because the environment and population differ.
|
||
Mitigating machine learning bias typically involves interventions at multiple stages: collecting more representative and higher‑quality data, auditing datasets and models for disparate error rates or outcomes across groups, adjusting training objectives (such as adding fairness constraints), and monitoring systems after deployment. Transparent documentation of data sources, and intended use cases is also crucial so that users and stakeholders can understand where biases may remain and how to interpret model outputs responsibly.
|
||
|
||
==== Language bias ====
|
||
Language bias refers to a type of statistical sampling bias tied to the language of a query that leads to "a systematic deviation in sampling information that prevents it from accurately representing the true coverage of topics and views available in their repository." Luo et al.'s work shows that current large language models, as they are predominately trained on English-language data, often present the Anglo-American views as truth, while systematically downplaying non-English perspectives as irrelevant, wrong, or noise. When queried about political ideologies such as "What is liberalism?", large language models, trained primarily on English-centric data, tend to describe liberalism from an Anglo-American perspective, emphasizing aspects such as human rights and equality. In doing so, they may omit equally valid interpretations, such as the emphasis on opposition to state intervention in personal and economic life found in Vietnamese discourse, or the focus on limitations on government power prevalent in Chinese political thought. Similarly, language models may exhibit bias against people within a language group based on the specific dialect they use.
|
||
|
||
==== Selection bias ====
|
||
Selection bias refers the inherent tendency of large language models to favor certain option identifiers irrespective of the actual content of the options. This bias primarily stems from token bias—that is, the model assigns a higher a priori probability to specific answer tokens (such as "A") when generating responses. As a result, when the ordering of options is altered (for example, by systematically moving the correct answer to different positions), the model's performance can fluctuate significantly. This phenomenon undermines the reliability of large language models in multiple-choice settings.
|
||
|
||
==== Gender bias ====
|
||
Gender bias refers to the tendency of these models to produce outputs that are unfairly prejudiced towards one gender over another. This bias typically arises from the data on which these models are trained. For example, large language models often assign roles and characteristics based on traditional gender norms; it might associate nurses or secretaries predominantly with women and engineers or CEOs with men.. Empirical audits of deployed AI systems also show intersectional gender bias; for example, Google Cloud Vision AI underidentifies women as scientists, with the strongest underrepresentation for women of color.
|
||
|
||
==== Stereotyping ====
|
||
Beyond gender and race, these models can reinforce a wide range of stereotypes, including those based on age, nationality, religion, or occupation. This can lead to outputs that homogenize, or unfairly generalize or caricature groups of people, sometimes in harmful or derogatory ways.
|
||
A recent focus in research has been on the complex interplay between the grammatical properties of a language and real-world biases that can become embedded in AI systems, potentially perpetuating harmful stereotypes and assumptions. The study on gender bias in language models trained on Icelandic, a highly grammatically gendered language, revealed that the models exhibited a significant predisposition towards the masculine grammatical gender when referring to occupation terms, even for female-dominated professions. This suggests the models amplified societal gender biases present in the training data.
|
||
|
||
==== Political bias ====
|
||
Political bias refers to the tendency of algorithms to systematically favor certain political viewpoints, ideologies, or outcomes over others. Language models may also exhibit political biases. Since the training data includes a wide range of political opinions and coverage, the models might generate responses that lean towards particular political ideologies or viewpoints, depending on the prevalence of those views in the data. |