When Are Learning Biases Equivalent? A Unifying Framework for Fairness, Robustness, and Distribution Shift

Basic Information

• Paper ID: 2511.07485
• Title: When Are Learning Biases Equivalent? A Unifying Framework for Fairness, Robustness, and Distribution Shift
• Author: Sushant Mehta
• Classification: cs.LG cs.AI stat.ML
• Conference: NeurIPS 2025 (39th Conference on Neural Information Processing Systems)
• Paper Link: https://arxiv.org/abs/2511.07485

Abstract

Machine learning systems exhibit multiple failure modes: unfairness toward protected groups, vulnerability to spurious correlations, and poor performance on minority subgroups. These issues are typically studied independently by different research communities. This paper proposes a unified theoretical framework that characterizes when different bias mechanisms produce quantitatively equivalent effects on model performance. By formalizing biases as violations of conditional independence (using information-theoretic measures), the authors prove formal equivalence conditions between spurious correlations, subgroup shifts, class imbalance, and fairness violations. The theory predicts that spurious correlations of strength α produce worst-group accuracy drops equivalent to subgroup imbalance ratios r ≈ (1+α)/(1-α). Empirical validation on six datasets and three architectures confirms that predicted equivalences hold within 3% error on worst-group accuracy, enabling principled transfer of debiasing methods across problem domains.

Research Background and Motivation

Problems to Address

Deep learning systems frequently exhibit systematic failures with degraded performance on specific subgroups despite high average accuracy. Specifically:

1. Algorithmic Unfairness: Medical diagnostic models accurate for majority populations but catastrophically fail for minority groups
2. Shortcut Learning: Image classifiers exploit spurious background correlations rather than learning robust features
3. Subgroup Shift: Recommendation systems amplify existing societal biases

Problem Importance

Current research lacks a formal framework to compare different bias mechanisms:

• The fairness community uses demographic parity and equalized odds metrics
• Robustness researchers optimize worst-group accuracy on spurious correlation benchmarks
• Distribution shift literature analyzes covariate and label shifts

These parallel research efforts use incompatible formalizations, preventing direct comparison and unified understanding.

Core Research Questions

1. Quantitative Equivalence: When are different biases quantitatively equivalent?
2. Performance Prediction: Does 90% spurious correlation produce the same worst-case performance as 9:1 class imbalance?
3. Method Transfer: Can fairness techniques mitigate spurious correlations? Can robust optimization address class imbalance?

Research Motivation

Answering these questions would enable:

• Predicting worst-group performance from distribution diagnostics
• Transferring verified debiasing methods across problem domains
• Selecting appropriate interventions based on which bias type has the most mature mitigation toolkit

Core Contributions

1. Unified Theoretical Framework: Treats all biases as violations of conditional independence between predictions and protected/spurious attributes given true labels, formalized through information-theoretic measures
2. Formal Equivalence Conditions: Proves when spurious correlations, subgroup shifts, and fairness violations produce quantitatively equivalent effects (Theorem 2)
3. Predictive Theory: Framework predicts worst-group performance from distribution properties, empirically validated on 18 problem configurations
4. Method Transfer Verification: Successfully demonstrates transfer of debiasing techniques across theoretically equivalent problems, achieving within 5% of from-scratch training performance
5. Literature Bridging: Establishes unified perspective across fairness, robustness, and generalization research communities

Methodology Details

Task Definition

Consider a learning problem:

• Input: X ∈ X
• Label: Y ∈ {0,1} (binary classification)
• Attribute: A ∈ {0,1}, representing protected group, spurious feature, or domain indicator
• Model: f_θ : X → {0,1}, producing prediction Ŷ = f_θ(X)

Core Definition: Information-Theoretic Formalization of Bias

Definition 1 (Bias): The bias of model f regarding attribute A on distribution D is:

B(f; D) = I(Ŷ; A | Y)

where I(·; · | ·) denotes conditional mutual information.

Unified Perspective:

• B > 0 indicates model predictions depend on A even given true label Y, violating conditional independence
• When A represents protected attributes, measures fairness violations
• When A represents spurious features, quantifies shortcut learning
• When A represents domain membership, captures distribution shift sensitivity

Theoretical Framework

Theorem 2 (Bias Equivalence): Consider two learning problems (D₁, A₁) and (D₂, A₂) with identical feature space X and label space Y but different attributes A₁, A₂. Under smoothness assumptions on loss function ℓ and feature overlap condition:

η = min_y ∫ min(p₁(x|y), p₂(x|y))dx > τ

If bias mechanisms satisfy ϵ-equivalence:

|B(f; D₁) - B(f; D₂)| ≤ ϵ

then worst-group accuracy difference is at most δ(ϵ, η), where:

δ(ϵ, η) = O(√ϵ/η)

Corollary 3 (Spurious Correlation ↔ Imbalance): Spurious correlation of strength α is equivalent to subgroup imbalance ratio r when:

r ≈ (1 + α)/(1 - α) · P(Y=1)/P(Y=0)

where:

• α = P(A=1|Y=1) - P(A=1|Y=0) (correlation strength)
• r = P(Y=1, A=1)/P(Y=0, A=1) (imbalance ratio)

Proof Strategy (Appendix A)

Step 1: Relating Bias to Worst-Group Loss Via Fano's inequality, worst-group error rate satisfies:

Err_worst ≤ [H(Y|A) + B(f; D)] / log 2

Step 2: Feature Overlap and Loss Distribution Under feature overlap condition η > τ, via coupling lemma and Lipschitz continuity, Wasserstein-1 distance satisfies:

|B(f; D₁) - B(f; D₂)| ≤ ϵ ⟹ W₁(L₁, L₂) ≤ C√ϵ/η

Step 3: Bounding Accuracy Difference Via Kantorovich-Rubinstein duality:

|Acc₁ - Acc₂| ≤ W₁(L₁, L₂) ≤ δ(ϵ, η) = O(√ϵ/η)

Technical Innovations

1. Information-Theoretic Unified View: First uses conditional mutual information I(Ŷ; A | Y) to uniformly characterize fairness, robustness, and distribution shift
2. Quantitative Equivalence Prediction: Provides computable formulas predicting equivalent bias configurations, rather than merely qualitative analysis
3. Feature Overlap Conditions: Explicitly identifies boundary conditions for equivalence (η > τ), explaining when equivalence fails
4. Operationality: Theory predictions directly applicable by measuring α and label marginals without complex computation

Experimental Setup

Datasets

Six benchmarks spanning spurious correlations, fairness, and distribution shift:

1. Waterbirds: Bird classification with background spurious correlation (95% training correlation)
2. CelebA: Hair color prediction with gender spurious correlation
3. ColoredMNIST: Synthetic dataset with controllable color-digit correlation
4. Adult Income: Income prediction with gender as protected attribute
5. CivilComments-WILDS: Toxicity detection across demographic groups
6. MetaShift: Visual domain adaptation with natural distribution shift

Model Architectures

Three architectures tested to evaluate whether equivalence depends on architecture choice:

• ResNet-50: Strong convolutional inductive bias
• ViT-B/16: Attention-based mechanism
• MLP-4L: Minimal structure

Comparison Methods

• ERM (Empirical Risk Minimization): Baseline
• GroupDRO: Group distribution robust optimization
• DFR (Deep Feature Reweighting): Last-layer retraining
• JTT (Just Train Twice): Two-stage training
• SPARE: Early spurious bias identification

Evaluation Metrics

• Primary Metric: Worst-group accuracy (minimum across (Y,A) groups)
• Auxiliary Metrics: Average accuracy, conditional mutual information B(f; D), fairness metrics (demographic parity gap, equalized odds violation)

Implementation Details

• Optimizer: SGD with learning rate 0.001 (decay 0.1 at epochs 30 and 60)
• Momentum: 0.9
• Weight Decay: 0.0001
• Batch Size: 128
• Training Epochs: 80 with early stopping based on validation worst-group accuracy
• Pretraining: ResNet-50 pretrained on ImageNet (Waterbirds, CelebA, MetaShift)
• Mutual Information Estimation: MINE estimator with 5-layer MLP, 1000 training iterations
• Random Seeds: 3 seeds (42, 123, 456)
• Computational Resources: 4 NVIDIA A100 GPUs (40GB), ~150 GPU hours total

Experimental Results

Main Results: Baseline Performance (Table 1)

Key Findings:

• ERM exhibits large gaps between average and worst-group accuracy (e.g., Waterbirds: 97.2% vs 62.3%)
• Debiasing methods significantly improve worst-group performance
• SPARE and DFR achieve best results on most benchmarks
• All entries have standard deviation < 1.2%

Equivalence Verification (Table 2)

Key Findings:

• Predicted accuracy differences match observations within 1% (all 6 problem pairs successful)
• Correlation between |B₁-B₂| and observed worst-group accuracy difference: ρ = 0.94 (p < 0.01)
• Validates that information-theoretic characterization captures essential relationships

Method Transfer Experiments (Table 3)

Key Findings:

• Transfer performance within 2.6% of from-scratch training (average degradation: 1.8%)
• Validates that theoretically equivalent problems share sufficient structure for direct method application
• Significant computational savings: transfer requires only forward pass vs. full optimization for from-scratch

Ablation Studies

Feature Overlap Dependency (Table 4)

Finding: Equivalence tightness improves with overlap, matching theoretical prediction δ ∝ 1/η

Architecture Sensitivity (Table 5)

Finding: Consistent equivalence across architectures (average variation 0.8%), indicating phenomenon is fundamentally distributional

Correlation Strength: Systematically vary spurious correlation strength α from 0.7 to 0.99, observing predicted equivalent imbalance ratios from 5.7:1 to 199:1, with all predictions verified within 4% worst-group accuracy, confirming Corollary 3 across entire correlation strength range.

Related Work

Spurious Correlations

• Deep networks readily exploit training-time features correlated with labels but non-generalizing
• Standard benchmarks: Waterbirds (bird species vs. background correlation), CelebA (hair color vs. gender correlation)
• Mitigation strategies: Two-stage training, last-layer retraining, early group separation

Fairness in Machine Learning

• Requires equal treatment across protected groups
• Common standards: demographic parity, equalized odds, individual fairness
• Impossibility results: multiple standards cannot simultaneously hold

Distribution Shift

• Models trained on one distribution often fail when deployed on shifted distributions
• Subgroup shift: group proportions change between training and testing
• Class imbalance: training data dominated by majority class

Implicit Bias

• Optimization algorithms introduce implicit biases determining which solutions appear during training
• Gradient descent converges to maximum ℓ₂-margin solutions
• Adam exhibits ℓ∞-margin bias

This Work's Contribution

Prior work addresses these phenomena separately. This paper provides the first formal framework characterizing their equivalence.

Conclusions and Discussion

Main Conclusions

1. Unified Perspective: Fairness, robustness, and generalization are different viewpoints on shared distributional challenges
2. Quantitative Prediction: Worst-group performance can be predicted from distribution measurements without expensive training
3. Method Transfer Feasibility: Verified debiasing techniques can transfer across theoretically equivalent problems
4. Empirical Validation: Worst-group accuracy differences in theoretically equivalent problems < 3% across 18 problem configurations

Limitations

Theoretical Limitations:

1. Binary Classification Assumption: Current theory limited to binary classification, though naturally extends to multi-class via one-vs-rest decomposition
2. Bound Looseness: δ(ϵ, η) bound may be loose in practice; tighter characterization via concentration inequalities remains open
3. Worst-Group Metric: Focuses on worst-group metrics; connections to calibration and individual fairness merit exploration

Practical Boundary Conditions (when equivalence fails):

1. Insufficient Feature Overlap: η < τ (typically 0.2), when groups occupy completely disjoint feature space regions
2. Non-Smooth Loss: 0-1 loss violates continuity assumptions (though cross-entropy used in practice satisfies requirements)
3. Architecture Bias Dominance: Overwhelms distributional effects (ablation studies show this rare)
4. Conditional Independence Assumption Violation: E.g., spurious feature actually causal

Future Directions

1. Multi-Class Extension: Complete theory for multi-class settings
2. Tighter Bounds: Improve δ(ϵ, η) characterization via concentration inequalities
3. Architecture-Data Interaction: Study whether architecture modifications constructively offset data biases
4. Causal Perspective: Integrate causal inference to distinguish true causality from spurious correlation
5. Calibration Fairness: Explore connections to calibration and individual fairness

Broader Impact

Positive Impact:

• Promotes more efficient research by revealing fundamental equivalences between bias types
• Techniques developed in one domain immediately suggest applications in others
• Likely accelerates progress in fairness and robustness

Potential Risks:

• Equivalence predictions assume correct attribute specification
• Misidentifying attributes (e.g., labeling spurious features as protected attributes) may cause practitioners to incorrectly transfer methods
• May amplify rather than mitigate biases

Recommendations: Conduct careful distribution analysis before applying transfers

In-Depth Evaluation

Strengths

1. Theoretical Innovation
   • First uses conditional mutual information to uniformly characterize multiple bias types
   • Provides computable quantitative equivalence prediction formulas
   • Rigorous proofs with explicit assumptions (smoothness, feature overlap)
2. Experimental Sufficiency
   • 6 datasets × 3 architectures = 18 configurations comprehensively validate theory
   • Multiple ablation studies verify theoretical predictions (feature overlap, architecture, correlation strength)
   • 3 random seeds with standard deviation reporting and statistical significance testing
3. Result Convincingness
   • Predictions match observations within 1% (Table 2)
   • Correlation ρ = 0.94 (p < 0.01) strongly supports theory
   • Successful method transfer (average degradation only 1.8%)
4. Practical Value
   • Provides actionable diagnostic tools
   • Significant computational savings (transfer vs. from-scratch)
   • Principled guidance for cross-community method transfer
5. Writing Clarity
   • Clear motivation and problem definition
   • Progressive theoretical framework development
   • Complete appendix with proofs and implementation details
   • Comprehensive NeurIPS checklist

Weaknesses

1. Method Limitations
   • Binary Classification Restriction: Despite authors' extensibility claims, lacks complete theory and experiments for multi-class
   • Bound Looseness: δ(ϵ, η) = O(√ϵ/η) may be loose in practice, limiting prediction precision
   • Binary Attribute Assumption: A ∈ {0,1} oversimplifies many real scenarios
2. Experimental Setup Deficiencies
   • Limited Transfer Verification: Only 3 problem pairs (Table 3) vs. 18 configurations for equivalence validation
   • Limited Architecture Coverage: Only 3 architectures tested, missing diverse inductive biases (Transformer variants, graph neural networks)
   • Missing Failure Cases: No demonstration of equivalence prediction failures and root causes
3. Insufficient Analysis
   • Feature Overlap Threshold τ: Theory requires η > τ but provides no practical guidance for selecting τ
   • Causality vs. Correlation: Insufficient discussion of distinguishing true causal features from spurious correlations
   • Mutual Information Estimation Error: Uses MINE estimator but doesn't quantify estimation error impact on predictions
4. Reproducibility Issues
   • Code promised after publication, unverifiable during review
   • Some experimental details missing (e.g., specific MINE hyperparameters)

Impact

1. Field Contributions
   • Pioneering Work: First establishes formal equivalence relationships between fairness, robustness, and distribution shift
   • Bridging Role: Connects three independent research communities, promoting cross-domain collaboration
   • Methodological Contribution: Information-theoretic perspective may inspire unified analysis of other ML problems
2. Practical Value
   • Diagnostic Tool: Practitioners can diagnose bias types by measuring B(f; D)
   • Method Selection Guidance: Choose mature mitigation techniques based on equivalence
   • Computational Efficiency: Method transfer significantly reduces computational costs
3. Reproducibility
   • Detailed experimental setup (Appendix B)
   • Uses standard public datasets
   • Code release promised
   • Unverifiable during review period
4. Citation Potential
   • Theoretical framework likely becomes foundation for subsequent research
   • Equivalence prediction formulas widely citable
   • Method transfer paradigm may inspire new research directions

Applicable Scenarios

Suitable Scenarios:

1. Bias Diagnosis: When models show worst-group performance degradation, determining root causes
2. Method Selection: Multiple debiasing techniques available, selecting most mature based on equivalence
3. Rapid Prototyping: Resource-constrained settings, validating ideas via transfer rather than from-scratch
4. Cross-Domain Application: Applying existing fairness/robustness techniques to new domains

Unsuitable Scenarios:

1. Complex Multi-Class Problems: Beyond binary classification with complex inter-class relationships
2. Extreme Feature Separation: Subgroups completely disjoint in feature space (η < 0.2)
3. Causal Structure Critical: Distinguishing causality from correlation essential
4. Non-Standard Loss: Using non-smooth loss functions (e.g., certain ranking losses)

Application Recommendations:

1. First measure feature overlap η and conditional mutual information B(f; D)
2. Verify smoothness assumptions hold for target problem
3. Carefully specify attribute A (distinguish protected attributes, spurious features, domain indicators)
4. Validate equivalence predictions on small-scale experiments before large-scale application
5. Monitor post-transfer performance, fine-tune if necessary

References

Key cited literature includes:

1. Sagawa et al. (2020) - GroupDRO method and Waterbirds benchmark
2. Geirhos et al. (2020) - Shortcut learning in deep networks
3. Hardt et al. (2016) - Equalized odds in supervised learning
4. Koh et al. (2021) - WILDS wild distribution shift benchmark
5. Kirichenko et al. (2022) - Deep Feature Reweighting (DFR)
6. Liu et al. (2021) - Just Train Twice (JTT) method

Overall Assessment: This is a high-quality theory-and-empirics combined work with pioneering contributions to machine learning bias research. The theoretical framework is elegant and practical, with sufficient experimental validation. Main limitations are binary classification assumptions and missing multi-class extensions. For a top-tier venue like NeurIPS, this is a strong paper meriting acceptance, with anticipated significant impact and inspiration for subsequent research. Recommend authors supplement final version with additional method transfer experiments, failure case analysis, and practical guidance for feature overlap threshold selection.