Evidence Without Injustice: A New Counterfactual Test for Fair Algorithms

Basic Information

• Paper ID: 2510.12822
• Title: Evidence Without Injustice: A New Counterfactual Test for Fair Algorithms
• Authors: Michele Loi, Marcello Di Bello, Nicolò Cangiotti
• Categories: cs.CY (Computers and Society), cs.AI (Artificial Intelligence), cs.LG (Machine Learning)
• Publication Date: October 10, 2025
• Paper Link: https://arxiv.org/abs/2510.12822

Abstract

This paper explores an overlooked but important dimension of algorithmic fairness: whether the evidential value of algorithmic outputs itself depends on structural injustice. Through contrasting two policing algorithm cases—predictive policing algorithms based on historical crime data and systems based on camera recordings of crimes in progress—the authors propose a new moral evaluation standard. This standard requires us to consider not only whether evidence has probative force in the actual world, but also whether it retains probative force in nearby possible worlds without relevant injustice. Predictive policing algorithms fail this test, while camera-based systems pass it.

Research Background and Motivation

Core Problem

Existing algorithmic fairness research primarily focuses on statistical standards (such as equalized odds and calibration), causal and counterfactual approaches, and the role of structural and compound injustice, but overlooks a critical dimension: whether the evidential value of algorithmic outputs itself depends on structural injustice.

Problem Significance

1. Divergence in Moral Intuitions: Although both types of policing algorithms produce disproportionate impacts on minority communities, people have different moral intuitions about their acceptability
2. Theoretical Gap: Existing fairness theories cannot explain why camera-based algorithms are morally more acceptable than predictive policing algorithms
3. Practical Application Needs: A more nuanced framework is needed to guide algorithm deployment in sensitive domains such as criminal justice and healthcare

Limitations of Existing Approaches

1. Statistical Standards: Cannot capture the moral difference between the two algorithms, as both produce different error rates across racial groups
2. Causal Approaches: Treat race as a manipulable discrete variable, overlooking race's embeddedness in complex structural phenomena
3. Compound Injustice Theory: Applies equally to both algorithms, failing to distinguish their moral differences

Core Contributions

1. Proposes the Counterfactual Independence Principle (CIP): A new moral evaluation framework for assessing the acceptability of evidence-based algorithms
2. Establishes CIP Testing Methodology: Systematic steps to evaluate whether evidence retains probative force in worlds without structural injustice
3. Provides Cross-Domain Application Guidance: Applies the principle to criminal justice and healthcare domains, demonstrating its practical utility
4. Distinguishes Punitive and Supportive Applications: Provides differentiated moral guidance for different types of algorithmic applications

Methodology in Detail

Definition of Counterfactual Independence Principle (CIP)

CIP comprises two core components:

CIP Test

For evidence E regarding outcome O (such as crime location), examine whether E retains its probative value for O in nearby possible worlds where relevant structural injustice does not exist. If yes, E passes the test; otherwise, it fails.

Downstream Use Guidance

Whether E passes the CIP test is a morally relevant consideration for determining its acceptable downstream uses. For punitive uses of evidence (such as dispatching police or restricting individual liberty), greater caution should be exercised when evidence fails the CIP test than when it passes.

Specific Steps of CIP Testing

1. Identify Mechanisms: Determine the mechanism that makes evidence E relevant to outcome O
2. Assess Injustice: Determine whether this mechanism is part of structural injustice
3. Evaluate Counterfactual Worlds: If so, evaluate nearby counterfactual worlds where the unjust mechanism is removed, asking whether the evidential connection still holds
4. Draw Conclusions: If it holds, E passes the CIP test; otherwise, it fails

Causal Chain Analysis Examples

Causal Chain in Predictive Policing:

Minority Communities → Redlining Policies → Racial Segregation → Underinvestment in Communities 
→ Underfunded Schools → Limited Employment Opportunities → Concentrated Poverty → More Crime

Causal Chain in Camera Evidence:

Crime in Minority Communities → Camera Captures Perpetrator → Crime Location Information

Distinction from Existing Methods

CIP avoids problems with traditional causal approaches:

• Does Not Require Manipulating Race Variables: Instead asks whether the evidential relationship still holds without unjust mechanisms
• Focuses on Epistemic Relevance of Evidence: Rather than algorithm output sensitivity to racial variation
• Acknowledges Structural Embeddedness of Race: Does not treat race as an independently manipulable variable

Experimental Setup and Application Cases

Applications in Criminal Justice

Traditional Evidence Evaluation

• DNA Samples, Fingerprints, Video Recordings: Pass the CIP test, as their evidential value is based on causal connections to crime, not dependent on unjust social conditions
• Character Evidence and Criminal History: May fail the test if their predictive value depends on crime environments created by structural injustice

Algorithm Tool Evaluation

• DNA Matching, Surveillance Video Authentication Algorithms: Pass the CIP test
• PredPol, HunchLab and Other Predictive Policing Systems: Fail the test, as they depend on crime spatial clustering caused by structural injustice
• Prediction Tools Based on Immediate Repeat Victimization: May pass the test

Applications in Healthcare

Three Scenario Analysis

1. Genetic Factor Scenario:
   • Patient has reduced drug metabolism capacity due to genetic variation
   • CIP Result: Passes the test, as genetic variation remains relevant in a just world
2. Comorbidity Scenario:
   • Type 2 diabetes and other comorbidities reduce drug effectiveness, but their prevalence is affected by structural injustice
   • CIP Result: Passes the test, as the biological relationship still holds in a just world
3. Social Variable Scenario:
   • Algorithm predicts treatment outcomes based on social variables such as low income and housing instability
   • CIP Result: Fails the test, as these variables would no longer predict treatment outcomes in a more equitable world

Experimental Results and Findings

Main Findings

1. Explanation of Moral Differences: CIP successfully explains why camera-based algorithms are morally more acceptable than predictive policing algorithms
2. Cross-Domain Applicability: The principle demonstrates practical value in both criminal justice and healthcare domains
3. Differentiation of Application Types: Punitive vs. supportive applications require different moral considerations

Reversal Effect in Punitive vs. Supportive Applications

• Punitive Applications: Evidence failing the CIP test should be used cautiously
• Supportive Applications: Failing the CIP test actually provides additional reasons for using evidence to guide remedial interventions

Specific Case Analysis

Diabetes Prevention Program

• Historical data shows higher diabetes risk in certain minority communities
• Although evidence fails the CIP test, this reinforces the rationale for using it to guide preventive care

Community Policing

• Historical crime data can be used for supportive interventions: improving street lighting, community mediation programs
• Rather than solely for punitive measures: increased patrols, expanded stop-and-frisk

Related Work

Algorithmic Fairness Research Context

1. Statistical Fairness Standards: Equalized odds, calibration, etc.
2. Causal Fairness: Chiappa & Gillam's path-specific counterfactual fairness
3. Counterfactual Fairness: Kusner et al.'s individual counterfactual approach
4. Structural Injustice: Hellman's compound injustice theory
5. Predictive Justice: Lazar & Stone's predictive parity principle

Unique Contributions of This Paper

• Evidential Turn: Shifts focus from algorithmic outputs to the moral status of evidence itself
• Mechanism-Level Analysis: Deeply analyzes causal mechanisms that give evidence probative force
• Application-Oriented Differentiation: Provides differentiated guidance for different types of applications

Conclusions and Discussion

Main Conclusions

1. CIP Provides a New Moral Evaluation Dimension: Supplements existing algorithmic fairness theories
2. The Moral Status of Evidence Matters: Must consider not only statistical performance but also the moral acceptability of the evidence basis
3. Application Type Determines Moral Requirements: Punitive and supportive applications require different moral standards

Limitations

1. Epistemological Challenges of Counterfactual Reasoning: Evaluating worlds without structural injustice requires robust causal theory support
2. Difficulty in Identifying Complex Causal Mechanisms: In practice, it may be difficult to accurately identify all relevant causal mechanisms
3. Ambiguity in Application Boundaries: The boundary between punitive and supportive applications may be unclear in some cases

Future Research Directions

1. Operationalization of CIP: Develop more concrete tools and guidelines for applying the CIP test
2. Cross-Domain Extension: Apply the principle to other sensitive domains such as education and employment
3. Integration with Existing Fairness Standards: Explore coordination mechanisms between CIP and statistical fairness standards

In-Depth Evaluation

Strengths

1. Strong Conceptual Innovation: Introduces a new dimension to algorithmic fairness research, filling an important theoretical gap
2. Solid Philosophical Foundation: Successfully integrates insights from epistemology and moral philosophy into algorithmic fairness discussion
3. High Practical Value: Provides concrete moral guidance frameworks for actual algorithm deployment
4. Rigorous Argumentation: Supports core arguments through detailed case analysis and comparative reasoning
5. Interdisciplinary Integration: Successfully integrates perspectives from computer science, philosophy, law, and public policy

Shortcomings

1. Operationalization Challenges: CIP testing may face practical difficulties in complex real-world scenarios
2. Value Judgment Dependency: What constitutes "structural injustice" itself involves value judgments that may be controversial
3. Computational Complexity: Evaluating counterfactual scenarios in complex causal networks may be computationally challenging
4. Cultural Relativity: Different societies and cultural contexts may have different understandings of justice

Impact Assessment

1. Academic Contribution: Opens new theoretical directions for algorithmic fairness research
2. Policy Value: Provides important reference for government and organizational algorithm use policies
3. Practical Guidance: Provides moral decision-making frameworks for algorithm developers and deployers
4. Cross-Domain Impact: May influence research in law, public policy, ethics, and other related fields

Applicable Scenarios

1. High-Risk Algorithm Applications: Particularly suitable for sensitive domains such as criminal justice, medicine, and education
2. Policy Making: Provides theoretical support for government algorithm governance policies
3. Algorithm Auditing: Provides evaluation frameworks for third-party algorithm audits
4. Ethics Committee Decisions: Provides tools for institutional ethics committees to evaluate algorithm projects

References

This paper cites important literature from multiple fields including algorithmic fairness, structural injustice, and epistemology, including:

• Algorithmic Fairness: Kusner et al. (2017) on counterfactual fairness, Chiappa & Gillam (2018) on path-specific fairness
• Structural Injustice: Young (2011) on structural injustice theory, Hellman (2023) on compound injustice
• Empirical Research: Extensive empirical research on redlining policies, racial segregation's effects on crime and health
• Legal Theory: Related literature on the use of statistical evidence in legal proceedings

This paper makes important theoretical contributions to algorithmic fairness research. The proposed Counterfactual Independence Principle not only offers profound philosophical insights but also possesses significant practical value. Although it still faces operationalization challenges, it points to important directions for future development in this field.