A Data-Centric Approach to Detecting and Mitigating Demographic Bias in Pediatric Mental Health Text: A Case Study in Anxiety Detection

Basic Information

• Paper ID: 2501.00129
• Title: A Data-Centric Approach to Detecting and Mitigating Demographic Bias in Pediatric Mental Health Text: A Case Study in Anxiety Detection
• Authors: Julia Ive, Paulina Bondaronek, Vishal Yadav, Daniel Santel, Tracy Glauser, Tina Cheng, Jeffrey R. Strawn, Greeshma Agasthya, Jordan Tschida, Sanghyun Choo, Mayanka Chandrashekar, Anuj J. Kapadia, John Pestian
• Classification: cs.CL cs.AI
• Institutions: University College London, Queen Mary University of London, Cincinnati Children's Hospital Medical Center, Oak Ridge National Laboratory, et al.
• Paper Type: Research Paper

Abstract

This study addresses demographic bias in AI models for pediatric mental health by proposing a data-centric debiasing approach. The research identifies systematic underdiagnosis in female adolescent patients, with 4% lower accuracy and 9% higher false negative rates compared to male patients. Through information density filtering and gender-neutral word substitution debiasing methods, the study successfully reduces diagnostic bias by 27%, providing an effective solution for fairness in medical AI.

Research Background and Motivation

Problem Identification

1. Prevalence of AI Bias: Medical AI models frequently inherit biases from training data, potentially exacerbating healthcare disparities, particularly affecting minority populations
2. Specificity of Mental Health: Mental health diagnosis heavily relies on unstructured text data (clinical notes), while existing debiasing research primarily focuses on structured data
3. Pediatric Mental Health Crisis: Post-COVID-19, the prevalence of anxiety symptoms in children has doubled, particularly among female adolescents

Significance

• Complexity and challenges in pediatric mental health screening
• Enormous potential of AI in expanding mental health diagnosis
• Urgent need to ensure AI tools are fair and effective across diverse populations

Limitations of Existing Approaches

• Traditional debiasing techniques (e.g., word embedding debiasing, adversarial training) are not applicable to medical domains
• Heterogeneity of medical data (from different healthcare institutions) has not been adequately addressed
• Lack of specialized debiasing frameworks for medical text

Core Contributions

1. Systematic Bias Identification: First to identify and quantify gender bias in pediatric anxiety detection, with significantly higher false negative rates for female patients
2. Data-Centric Debiasing Framework: Proposes debiasing methods specifically tailored for medical text, including information density filtering and gender word neutralization
3. Effectiveness Validation: Validates the method's effectiveness on real clinical data, reducing diagnostic bias by up to 27%
4. Interpretability Analysis: Uses LIME technology to analyze vocabulary on which model decisions depend, revealing sources of bias

Methodology Details

Task Definition

Input: Sequence of clinical note text from pediatric patients Output: Binary classification prediction (anxiety/non-anxiety) Objective: Reduce performance disparities across gender groups while maintaining prediction accuracy

Bias Detection Framework

1. Classification Fairness Assessment

Multiple metrics are used to evaluate model bias:

• Balanced Error Rate (BER): $BER = \frac{(\frac{FP}{FP+TN}) + (\frac{FN}{FN+TP})}{2}$
• False Negative Rate (FNR): Measures underdiagnosis rate
• False Positive Rate (FPR): Measures misdiagnosis rate
• BER Ratio: Ratio of underprivileged to privileged group BER; >1.25 indicates significant bias

2. Text Distribution Analysis

Analyzes differences in text characteristics across demographic groups:

• Average note length
• Medical terminology percentage
• Gender-biased vocabulary percentage
• Jaccard distance and familiarity scores

Debiasing Methods

1. Information Density Filtering (tf-idf_filt)

• Calculates sentence importance using TF-IDF scores
• Removes 20% of sentences with lowest information content
• Balances information density across different groups

2. Gender Word Debiasing (gen_sub)

• Automatically detects gender-biased vocabulary such as names and pronouns
• Extracts proper nouns using Stanza tool
• Replaces gender-specific vocabulary with neutral alternatives
  • Names → "person1", "person2", etc.
  • Pronouns → "he/she" → "they"

3. Combined Method (tf-idf_filt+gen_sub)

Combines information density filtering and gender word substitution to leverage synergistic effects

Model Architecture

• Transformer model based on Clinical-BigBird
• Pre-trained specifically on clinical text
• Supports long sequence input (up to 4,096 tokens)
• Fine-tuning parameters: 2 epochs, learning rate 1e-5, batch size 8

Experimental Setup

Dataset

• Source: Cincinnati Children's Hospital Medical Center
• Scale: 1.3 million patients, 63 million clinical notes
• Time Span: January 2009 - March 2022
• Anxiety Cases: 84,426 cases meeting screening criteria
• Final Data: 73,288 patients, 7.81 million notes

Age Grouping Strategy

• Divided into 5 age groups: 5, 8, 10, 12, 15 years
• 3,700-5,064 training samples per group
• 852-1,278 test samples per group
• 1:1 case-control matching (by age and gender)

Data Preprocessing

• Deduplication: notes with cosine similarity ≥0.8
• Selection of most recent 25 notes
• Input length limited to 1,000 tokens

Evaluation Metrics

• Accuracy
• False Negative Rate (FNR) - primary metric of focus
• False Positive Rate (FPR)
• Balanced Error Rate (BER)
• Percentage of uncertain predictions (probability in 0.4, 0.6 interval)

Experimental Results

Main Findings

1. Systematic Gender Bias

2. Text Distribution Differences

• Vocabulary Similarity: Jaccard index 0.54 (male-female)
• Terminology Distribution: Jaccard index 0.34 (significant difference)
• Lowest Similarity: Age 5 and 15 groups (Jaccard 0.43)

3. Debiasing Effects

Best Method (tf-idf_filt):

• FNR gap reduction of 0.024 (27% improvement)
• Bin 5: FNR gap reduced from 0.13 to 0.02
• Bin 15: FNR gap reduced from 0.13 to 0.07
• BER ratio decreased from 1.33 to 0.98 (Bin 10)

Ablation Study

Interpretability Analysis

Using LIME to analyze vocabulary on which model decisions depend:

• Original Model: 10% of cases rely on biased vocabulary for prediction
• tf-idf_filt: Reduced to 3%
• Combined Method: 50% reduction in biased vocabulary frequency

Cross-Race Validation

• Other racial groups show average FNR 0.05 higher
• Combined method reduces FNR gap by 0.034
• Demonstrates method's generalizability

Related Work

Machine Learning Fairness

• Preprocessing techniques: resampling, data augmentation
• Algorithm modification: adversarial debiasing, objective function modification
• Post-processing techniques: calibration, embedding transformation

NLP Debiasing Methods

• Attribute swapping: exchanging sensitive attribute vocabulary
• Embedding debiasing: removing gender components from word embeddings
• Adversarial training: penalizing predictions influenced by protected attributes

Medical AI Bias

• Racial bias in commercial prediction algorithms
• Group disparities in suicide risk prediction
• Demographic bias in pathology models

Conclusions and Discussion

Main Conclusions

1. Bias is Pervasive: Pediatric anxiety detection models exhibit systematic underdiagnosis for female patients
2. Text Differences are Root Cause: Significant differences exist in information density and language distribution between notes of different genders
3. Data-Centric Approach is Effective: Significant bias reduction can be achieved through information density balancing and language neutralization
4. Clinical Significance: 27% bias reduction is clinically meaningful for improving female patient diagnosis

Limitations

1. Data Quality Dependency: Method effectiveness is limited by EHR text quality and consistency
2. Single Bias Type: Focuses only on gender bias, does not address other demographic characteristics
3. Generalization Capacity: Generalizability across different clinical settings requires further validation
4. Biological Differences: Difficult to completely distinguish biological differences from sociocultural differences

Future Directions

1. Extend to other mental health conditions and populations
2. Develop more refined bias detection and mitigation techniques
3. Incorporate multimodal data (text + structured data)
4. Establish standardized fairness assessment frameworks for medical AI

In-Depth Evaluation

Strengths

1. Problem Importance: Focuses on pediatric mental health, a critical domain with significant social value
2. Methodological Innovation: Proposes a data-centric debiasing framework specifically tailored for medical text
3. Experimental Rigor: Validated on large-scale real clinical data with multidimensional bias analysis
4. Practical Value: Simple and effective methods easily deployable in clinical environments
5. Interpretability: Provides interpretable model decision analysis using techniques like LIME

Weaknesses

1. Theoretical Depth: Lacks deep theoretical analysis of bias generation mechanisms
2. Method Limitations: Debiasing methods are relatively simple, potentially oversimplifying the problem
3. Single Evaluation Dimension: Primarily focuses on classification fairness, lacks other fairness dimensions such as calibration
4. Long-term Impact: Does not evaluate the impact of debiasing on long-term model performance and generalization

Impact

1. Academic Contribution: Provides important case studies and methodological references for medical NLP bias research
2. Practical Value: Offers concrete solutions for improving fairness in clinical AI systems
3. Policy Significance: Provides technical support for medical AI regulation and standard-setting
4. Reproducibility: Detailed method descriptions ensure good reproducibility

Applicable Scenarios

1. Clinical Decision Support: Mental health screening and diagnostic assistance systems
2. Healthcare Quality Improvement: Identifying and mitigating bias in existing medical AI systems
3. Regulatory Compliance: Meeting fairness and ethical requirements for medical AI
4. Research Tool: Providing methodological foundation for other medical AI bias research

References

This paper cites important literature from fairness in machine learning, NLP debiasing, and medical AI, including:

• Feldman et al. (2015) - Fairness metrics
• Bolukbasi et al. (2016) - Word embedding debiasing
• Obermeyer et al. (2019) - Racial bias in medical algorithms
• Ribeiro et al. (2016) - LIME interpretability method

Overall Assessment: This is an important research paper in the field of medical AI fairness that not only identifies gender bias issues in pediatric mental health AI but also proposes practical solutions. While there is room for improvement in theoretical depth and methodological complexity, its practical value and social significance make it an important contribution to the field.