Rewriting the Code: Reducing Discrimination in Machine Learning#

Defining Discrimination and Bias in Machine Learning#

• Bias: An inclination or prejudice toward some outcome, often due to flawed data collection, labeling, or modeling processes. In machine learning, bias can cause algorithms to systematically favor or disfavor certain groups.
• Discrimination: When a model’s predictions or decisions treat an individual or group differently based on characteristics such as race, gender, or age. Discrimination in machine learning typically manifests as imbalanced error rates or systematically lower or higher predictions for protected groups.

Common Terminology#

• Protected Attribute: An attribute such as race, gender, ethnicity, or religion protected by law or recognized as a sensitive characteristic.
• Unprotected Attribute: Attributes not specifically shielded by anti-discrimination laws (e.g., height, weight, location). However, these can still cause bias and must be carefully monitored.
• Parity: The notion that performance or outcomes should be consistent (or “on par�? across different demographic groups.

Data Collection Issues#

Machine learning models are only as good as the data they learn from. If the dataset is incomplete, lacks diversity, or reflects historical prejudices, the resulting model may perpetuate these inequities.

• Limited Demographic Representation: If only one area or demographic is well represented, the model’s performance will suffer on underrepresented groups.
• Incorrect Labels: Data labels might be impacted by societal biases. For instance, in judicial contexts, certain crimes might be under-reported in specific communities, leading to skewed data.

Sampling Bias#

Sampling bias occurs when the data used to train a model is not representative of real-world populations. Systems that rely on crowdsourcing, for instance, might over-sample certain internet-savvy demographics, leaving out certain socioeconomic groups.

Historical Biases#

Historical biases reflect patterns deeply ingrained in existing systems. For instance, if in the past certain jobs were disproportionately offered to one demographic, training data for a hiring model carries that imbalance forward. The model “inherits�?historical inequities, reinforcing them if not corrected.

Predictive Policing#

Predictive policing uses algorithms to forecast where crimes are likely to occur. Biased datasets—often overemphasizing crimes reported in heavily policed neighborhoods—can cause these algorithms to recommend additional policing in those neighborhoods, creating a vicious cycle.

Hiring Systems#

As companies scale hiring, they often turn to automated resume screening tools. If a tool’s underlying training data is predominantly male, or historically favored certain backgrounds, it may systematically rank male resumes higher or filter out candidates from minority groups.

Healthcare#

Healthcare systems rely heavily on risk prediction models to recommend treatments. Biased models might underestimate health risks for underrepresented patient groups, resulting in subpar treatment recommendations and potential harm.

Group Fairness#

Group fairness requires that different demographic groups (e.g., males vs. females) receive equitable treatment. Formal metrics such as “demographic parity�?(equal probability of a positive prediction) ensure that a protected group is not denied benefits systematically.

Example principle:

The model predicts the same proportion of positive outcomes for men and women.

Individual Fairness#

Individual fairness focuses on ensuring that “similar individuals�?receive similar outcomes. Rather than looking at the group level, these methods treat each person on a case-by-case basis, to avoid scenarios where two candidates with similar qualifications receive dramatically different predictions solely because of membership in different demographic groups.

Calibration and Interpretability#

• Calibration: A well-calibrated model outputs probabilities that match real-world frequencies. For example, among people predicted to have a 70% risk of some outcome, about 70% should actually experience it.
• Interpretability: The ability to understand and explain a model’s reasoning. Interpretability helps detect unfair treatment by revealing which features inform the model’s predictions the most.

Data Cleaning and Preprocessing#

One of the simplest ways to reduce bias is to ensure your data is as clean and complete as possible. Common steps include:

• Removing Identifiers: Directly removing sensitive attributes (when appropriate).
• Feature Engineering: Building features that capture relevant information without directly implicating protected attributes.
• Data Balancing: Re-sampling or synthesizing minority examples to balance out the distribution.

Re-sampling (Under/Over-Sampling)#

If your dataset shows underrepresentation in a particular demographic subgroup, you can:

• Under-sample the dominant group.
• Over-sample the minority group.
• Synthesizing new minority group samples using techniques like SMOTE (Synthetic Minority Over-sampling Technique).

These tasks should be carried out carefully to avoid artificially inflating your dataset or blindly removing critical data.

Adversarial Debiasing#

Adversarial debiasing uses a setup in which one model (the predictor) tries to make accurate predictions, while another model (the adversary) attempts to determine the protected attribute from the predictor’s outputs. Over time, the predictor learns to mitigate signs of the protected attribute, effectively “scrubbing�?it from the process.

Demographic Parity#

Demographic parity is reached if the probability of a positive outcome is the same for every group. It addresses scenarios where we want the model to be “group blind�?when distributing outcomes (e.g., loans, approvals, or diagnoses).

Example formula:
P(Ŷ = 1 | A = 0) = P(Ŷ = 1 | A = 1),
where Ŷ is the predicted label, and A is the protected attribute (with values 0 or 1).

Equality of Opportunity#

A less stringent requirement than demographic parity, equality of opportunity ensures that true positive rate is the same across groups. For instance, if one group is identified as having a disease at a certain rate, the model should correctly identify that disease for individuals in that group with the same success rate as for other groups.

Example formula:
TPR(A = 0) = TPR(A = 1).

Equalized Odds#

Equalized odds expands equality of opportunity by requiring not just TPR to be equal across groups, but also FPR (false positive rate). Thus, the model must be equally likely to correctly classify positives and not misclassify negatives across all groups.

AI Fairness 360 (AIF360)#

Developed by IBM, AI Fairness 360 is a comprehensive Python toolkit providing:

• Preprocessing algorithms (like reweighing).
• In-processing algorithms (like adversarial debiasing).
• Postprocessing algorithms (like calibrated equalized odds).
  It also includes fairness metrics like demographic parity and equalized odds.

Themis#

Themis is a testing-based tool for measuring discrimination in software systems. It systematically generates test inputs to evaluate potential discriminatory behavior in your model.

Fairlearn#

Fairlearn (developed by Microsoft) offers a variety of algorithms and metrics to mitigate and measure fairness in models. It has a user-friendly Python API with advanced visualizations.

Data Preparation#

Assume we have a dataset for a loan approval model. Each row represents an applicant and includes:

• Demographic attributes (e.g., gender).
• Financial history (e.g., credit score, annual income).
• The final “Approved�?or “Not Approved�?label.

Below is a small snippet demonstrating how you might load and preprocess the data using pandas and scikit-learn.

1
import pandas as pd
2
from sklearn.model_selection import train_test_split
3

4
# Example dataset: columns = ['gender', 'credit_score', 'annual_income', 'loan_approved']
5
data = pd.read_csv('loan_applications.csv')
6

7
# Separate features and labels
8
X = data.drop(columns=['loan_approved'])
9
y = data['loan_approved']
10

11
# Identify the protected attribute
12
protected_attribute = 'gender'
13
X_protected = X[protected_attribute]
14

15
# Encode categorical variables (if any)
16
X = pd.get_dummies(X, drop_first=True)
17

18
# Train-test split
19
X_train, X_test, y_train, y_test = train_test_split(X, y, stratify=y, random_state=42)

Baseline Model Training#

Let’s train a baseline classifier without performing any fairness-oriented steps. For demonstration, we’ll use a random forest. In practice, the choice of model can heavily influence fairness outcomes.

1
from sklearn.ensemble import RandomForestClassifier
2
from sklearn.metrics import accuracy_score
3

4
model = RandomForestClassifier(random_state=42)
5
model.fit(X_train, y_train)
6

7
y_pred = model.predict(X_test)
8

9
# Evaluate accuracy
10
accuracy = accuracy_score(y_test, y_pred)
11
print(f"Baseline model accuracy: {accuracy:.3f}")

At this point, we have a baseline model. It might have high accuracy overall but still exhibit discriminatory behavior for certain subgroups.

Evaluating Fairness#

Let’s say “gender_Female�?is a column that indicates female applicants. We can compute common metrics split by protected groups.

1
import numpy as np
2
from sklearn.metrics import confusion_matrix
3

4
# Predict probas for advanced metrics
5
y_proba = model.predict_proba(X_test)[:, 1]  # Probability of approval
6

7
# Subsetting test data into protected groups
8
female_idx = X_test['gender_Female'] == 1
9
male_idx = X_test['gender_Female'] == 0
10

11
# Confusion Matrix for each group
12
def get_rates(y_true, y_pred):
13
    tn, fp, fn, tp = confusion_matrix(y_true, y_pred).ravel()
14
    tpr = tp / (tp + fn)
15
    fpr = fp / (fp + tn)
16
    return tpr, fpr
17

18
tpr_female, fpr_female = get_rates(y_test[female_idx], y_pred[female_idx])
19
tpr_male, fpr_male = get_rates(y_test[male_idx], y_pred[male_idx])
20

21
print(f"TPR - Female: {tpr_female:.3f}, Male: {tpr_male:.3f}")
22
print(f"FPR - Female: {fpr_female:.3f}, Male: {fpr_male:.3f}")

If there is a significant difference in TPR or FPR across groups, your model might violate criteria like equalized odds.

Mitigating Bias#

To demonstrate a simple in-processing technique, consider “reweighing.�?If you have a training dataset that under-represents female applicants, you could assign a higher weight to them in the training process. Alternatively, you can use a specialized tool like AIF360 for advanced techniques. Here’s how you might manually implement a basic reweighing approach:

1
# Suppose "weight" is computed based on the proportion of each group in the training set
2
group_counts = X_train['gender_Female'].value_counts()
3
female_count = group_counts[1]
4
male_count = group_counts[0]
5
n_samples = len(X_train)
6

7
# For illustration: equal weight to each group
8
weight_female = n_samples / (2 * female_count)
9
weight_male = n_samples / (2 * male_count)
10

11
train_weights = X_train['gender_Female'].apply(lambda x: weight_female if x == 1 else weight_male)
12

13
# Retrain with sample weights
14
model_reweighed = RandomForestClassifier(random_state=42)
15
model_reweighed.fit(X_train.drop(columns=['gender_Female']), y_train, sample_weight=train_weights)
16

17
y_pred_reweighed = model_reweighed.predict(X_test.drop(columns=['gender_Female']))
18

19
tpr_female_rw, fpr_female_rw = get_rates(y_test[female_idx], y_pred_reweighed[female_idx])
20
tpr_male_rw, fpr_male_rw = get_rates(y_test[male_idx], y_pred_reweighed[male_idx])
21

22
print(f"After Reweighing - TPR Female: {tpr_female_rw:.3f}, TPR Male: {tpr_male_rw:.3f}")
23
print(f"After Reweighing - FPR Female: {fpr_female_rw:.3f}, FPR Male: {fpr_male_rw:.3f}")

While this is a simple example, there are many sophisticated algorithms (like adversarial debiasing, reject option classification, or calibration-based methods) that can further ameliorate discrimination.

Interpretability Techniques#

• LIME (Local Interpretable Model-Agnostic Explanations): Provides localized explanations for a model’s predictions, making it easier to see how each feature affects an individual decision.
• SHAP (SHapley Additive exPlanations): Based on cooperative game theory, it attributes each feature’s “contribution�?to the final prediction.

Deep Learning and Fairness#

Deep neural networks can learn complex relationships but are also prone to inadvertently capturing biases. Techniques include:

• Designing architectures to reduce overfitting on majority groups.
• Using adversarial training to remove sensitive-attribute-related signals within hidden layers.

Privacy and Fairness#

Privacy and fairness intersect in many ways. Privacy-preserving methods like differential privacy can limit the visibility of sensitive attributes but can also complicate fairness interventions. A balanced approach is crucial to meet both objectives.

Ongoing Monitoring#

Even a well-designed system may drift into unfairness over time. Continuous monitoring of:

• Predictions grouped by demographic categories.
• Model performance metrics that capture fairness constraints.
• Shifts in data distribution (a phenomenon known as dataset shift).

Feedback Loops#

In many real-world systems, the model’s predictions influence future data collection (e.g., recommended policing in certain neighborhoods increases policing data from those neighborhoods). This feedback loop can intensify bias. Deploying “fairness-aware�?policies and regularly retraining on balanced data can help.