From Prejudice to Parity: Closing the AI Fairness Gap#

2.1 Formal Fairness Definitions#

Here are a few commonly discussed fairness definitions:

1. Demographic Parity (Statistical Parity)
   A classifier achieves demographic parity if the probability of a positive outcome is the same across demographic groups. Formally:
   P(Ŷ = 1 | A = a�? = P(Ŷ = 1 | A = a�?,
   for protected attribute A belonging to groups a�? a�?

2. Equalized Odds
   This criterion requires that the model’s false positive rate (FPR) and true positive rate (TPR) are equal across groups.

   • P(Ŷ = 1 | Y = 1, A = a�? = P(Ŷ = 1 | Y = 1, A = a�?
   • P(Ŷ = 1 | Y = 0, A = a�? = P(Ŷ = 1 | Y = 0, A = a�?

3. Predictive Parity
   This metric demands that the probability of the actual outcome (Y) given a predicted outcome (Ŷ = 1) is the same across groups:

   • P(Y = 1 | Ŷ = 1, A = a�? = P(Y = 1 | Ŷ = 1, A = a�?

4. Equal Opportunity
   A special case of equalized odds focusing only on the true positive rate:

   • P(Ŷ = 1 | Y = 1, A = a�? = P(Ŷ = 1 | Y = 1, A = a�?

In practice, these fairness definitions can be in conflict. Selecting which one to prioritize is a domain-specific judgment that balances societal values, legal requirements, and logistical constraints.

3.1 Biased Hiring Algorithms#

Several large tech companies once experimented with automated resume screening tools that showed bias toward male candidates. Historically, the training data came from previous hiring decisions in a male-dominated environment, meaning the AI algorithm learned to favor keywords more frequently associated with men.

3.2 Discriminatory Lending Decisions#

Financial institutions use AI for credit scoring. If the underlying data includes historical discrimination against a certain zip code (often correlated with race or socioeconomic status), the model might determine that applicants from that area have lower creditworthiness—even if the individual applicant’s financial behavior is otherwise solid.

3.3 Facial Recognition Errors#

Studies have shown facial recognition systems often perform worse on darker-skinned faces, leading to higher misidentification rates. This discrepancy arises from training data that skews heavily toward lighter-skinned faces, coupled with potential flaws in image processing pipelines.

Table: Common Consequences of AI Bias#

5.1 Group Fairness Metrics#

• Statistical Parity Difference:
  Measures the difference in the rate of positive classifications between protected and unprotected groups.
  SPD = P(Ŷ = 1 | A = a�? - P(Ŷ = 1 | A = a�?

• Disparate Impact (DI):
  Ratio of the rate of positive outcomes in one group to that in another.
  DI = (P(Ŷ = 1 | A = a�?) / (P(Ŷ = 1 | A = a�?)
  A rule of thumb in some legal contexts is that if DI < 0.8, it may be indicative of discriminatory impact.

• Equal Opportunity Difference:
  Difference in true positive rate across groups.
  EOD = TPR(a�? - TPR(a�?

5.2 Individual Fairness Metrics#

• Consistency:
  Measures how similar individuals in the dataset (by their features excluding the protected attribute) are treated similarly by the model.

• Counterfactual Fairness:
  Evaluates whether a model’s outcome would remain unchanged if a protected attribute were hypothetically altered.

5.3 Calibration Approaches#

• Calibration:
  For well-calibrated models, predicted probabilities correspond to actual outcome frequencies. If a model says 70% probability, then about 70% of the time the event should occur. Calibration should also ideally hold across subgroups, ensuring the model is equally reliable for all groups.

7.1 Data Preparation#

We’ll use some synthetic data to mimic the structure of a real dataset.

1
import numpy as np
2
import pandas as pd
3
from sklearn.model_selection import train_test_split
4
from sklearn.linear_model import LogisticRegression
5
from sklearn.metrics import accuracy_score
6

7
# Generate synthetic data
8
np.random.seed(42)
9
size = 2000
10

11
# Protected attribute: 0 or 1 (e.g., female=0, male=1)
12
gender = np.random.binomial(1, 0.5, size)
13

14
# Other features
15
income = np.random.normal(50000, 15000, size)
16
education = np.random.randint(1, 5, size)  # 1 to 4 representing education levels
17
age = np.random.normal(35, 10, size)
18

19
# Target (loan approval) with some bias introduced
20
# We'll assume that if gender=1 (male), there's slightly higher chance of approval
21
loan_approval = (0.3 * (income/1000) + 0.8 * education + 0.1 * age +
22
                 5 * gender + np.random.normal(0, 5, size)) > 65
23
loan_approval = loan_approval.astype(int)
24

25
data = pd.DataFrame({
26
    'gender': gender,
27
    'income': income,
28
    'education': education,
29
    'age': age,
30
    'loan_approval': loan_approval
31
})
32

33
X = data.drop('loan_approval', axis=1)
34
y = data['loan_approval']
35

36
# Train/Test split
37
X_train, X_test, y_train, y_test = train_test_split(X, y,
38
                                                    test_size=0.3,
39
                                                    stratify=y,
40
                                                    random_state=42)

7.2 Initial Model Training#

1
clf = LogisticRegression(solver='liblinear')
2
clf.fit(X_train, y_train)
3

4
y_pred = clf.predict(X_test)
5
accuracy = accuracy_score(y_test, y_pred)
6
print("Accuracy:", accuracy)

At this stage, our model might show reasonable accuracy, but we need to check the fairness metrics. Let’s measure some bias-related statistics for the protected attribute gender.

7.3 Fairness Evaluation (Statistical Parity Example)#

1
# Subset predictions by gender
2
male_idx = X_test['gender'] == 1
3
female_idx = X_test['gender'] == 0
4

5
male_positive_rate = np.mean(y_pred[male_idx])
6
female_positive_rate = np.mean(y_pred[female_idx])
7

8
print("Male Positive Rate:", male_positive_rate)
9
print("Female Positive Rate:", female_positive_rate)
10
print("Statistical Parity Difference:", male_positive_rate - female_positive_rate)

If you see that the positive rate (loan approval) is significantly higher for males than for females, that indicates potential bias in the model.

7.4 Mitigating Bias (Pre-processing: Reweighing)#

One approach is to reweigh or oversample the underrepresented group. Here’s a simplified illustration:

1
# Separate training data by gender
2
X_train_male = X_train[X_train['gender'] == 1]
3
y_train_male = y_train[X_train['gender'] == 1]
4
X_train_female = X_train[X_train['gender'] == 0]
5
y_train_female = y_train[X_train['gender'] == 0]
6

7
# Oversample the minority group if needed
8
ratio = len(X_train_male) / len(X_train_female)
9
X_train_female_oversampled = pd.concat([X_train_female]*int(ratio+1))
10
y_train_female_oversampled = pd.concat([y_train_female]*int(ratio+1))
11

12
X_train_balanced = pd.concat([X_train_male, X_train_female_oversampled]).sample(frac=1)
13
y_train_balanced = pd.concat([y_train_male, y_train_female_oversampled]).sample(frac=1)
14

15
# Re-train the model
16
clf_balanced = LogisticRegression(solver='liblinear')
17
clf_balanced.fit(X_train_balanced.drop('gender', axis=1), y_train_balanced)  # Dropping gender?

After re-training, evaluate the same metrics to see if the model’s parity has improved.

7.5 In-processing and Post-processing#

In reality, you might use more advanced methods such as:

• In-processing: Adding a fairness constraint in the loss function (e.g., adversarial debiasing).
• Post-processing: Adjusting decision thresholds for different subgroups to ensure a desired level of fairness.

9.1 Societal Considerations#

• Representational Harm: Even if a model does not “physically�?harm users, an algorithm that consistently associates certain demographic groups with negative connotations can reinforce stereotypes.
• Marginalized Communities: Overlooking the perspectives of underrepresented minorities can perpetuate social inequities.

9.2 Policy and Governance#

• Regulative Standards: The EU’s General Data Protection Regulation (GDPR), the proposed EU AI Act, and other emerging legal frameworks demand transparency and accountability in automated decision-making.
• Internal Governance: Organizations are establishing ethics boards and guidelines to ensure that AI initiatives consider fairness from the outset.
• Audits and Accountability: Independent audits, either voluntary or mandated, can uncover systemic biases before they cause harm.

9.3 Transparency vs. Privacy Trade-offs#

Pursuing fairness often requires collecting data on protected attributes such as race, gender, or religion. Yet, these attributes are sensitive and raise privacy concerns. Balancing the need for transparency with privacy safeguards requires careful planning and robust data governance.

10.1 Interdisciplinary Collaboration#

As we discover that purely technical solutions are not enough, collaboration between social scientists, ethicists, legal scholars, and technologists becomes increasingly important. Ethical AI needs a holistic approach that integrates legal constraints, cultural norms, and technical feasibility.

10.2 Dynamic Fairness#

Many current fairness definitions and tools focus on static datasets. However, real-world systems are dynamic: user behavior changes, and socio-political contexts evolve. Future research is exploring how to maintain fairness in models that continuously learn from new data streams.

10.3 Multi-Attribute Fairness#

Organizations often need to ensure fairness across multiple attributes simultaneously (e.g., race, gender, income, age). Handling overlapping identities—also known as intersectionality—remains a complex challenge in fairness research.

10.4 Explanation and Interpretability#

Transparent models (e.g., decision trees, rule-based systems) might be more interpretable, making it easier to detect and correct bias. Additionally, tools like SHAP (SHapley Additive exPlanations) or LIME (Local Interpretable Model-agnostic Explanations) help stakeholders see why a specific decision was made.