A comparison of approaches to incorporate patient-selected and patient-ranked outcomes in clinical trials

Basic Information

• Paper ID: 2510.11578
• Title: A comparison of approaches to incorporate patient-selected and patient-ranked outcomes in clinical trials
• Authors: David S. Robertson, Thomas Jaki
• Classification: stat.ME (Statistical Methodology)
• Publication Date: October 13, 2025 (arXiv preprint)
• Paper Link: https://arxiv.org/abs/2510.11578

Abstract

A critical aspect of patient-directed drug development is identifying and measuring outcomes that matter to patients in clinical trials. Many diseases affect multiple symptom domains, and consensus approaches for determining the relative importance of relevant multiple outcomes overlook the heterogeneity of individual patient preferences. Patient-selected outcomes provide a method for incorporating individual patient preferences, as proposed in the FDA's recently released guidance for migraine treatment, where each patient selects the most bothersome migraine-related symptom in addition to pain. Patient-ranked outcomes have also been recently proposed, further considering complete rankings of the relative importance of all outcomes, which can be assessed through composite DOOR (Desirability of Outcome Ranking) endpoints. This paper compares the advantages and disadvantages of using patient-selected outcomes versus patient-ranked outcomes in the context of a two-arm randomized controlled trial for multiple sclerosis, comparing efficacy and Type I error rates through simulation, and discussing other important considerations when using both approaches.

Research Background and Motivation

Importance of the Problem

1. Need for patient-directed drug development: Traditional clinical trial endpoint selection is often based on consensus among researchers and regulators, overlooking the heterogeneity of individual patient preferences
2. Multidimensional nature of complex diseases: Complex diseases such as multiple sclerosis and rheumatoid arthritis affect multiple symptom domains, with different patients placing varying emphasis on improvements in different symptoms
3. Regulatory endorsement: The FDA's 2018 guidance on migraine drug development formally recognized the use of patient-selected outcomes for the first time

Limitations of Existing Approaches

1. Traditional consensus methods: Overlook heterogeneity in patient preferences and may not truly reflect outcomes patients care about
2. Multiple co-primary endpoints: Require demonstrating effects across all outcomes, which may be overly stringent
3. Lack of individualization: Fail to adequately utilize individual patient preference information to optimize trial design and analysis

Research Motivation

This paper aims to systematically compare two approaches for incorporating patient preferences: patient-selected outcomes (considering only the most important outcome) and patient-ranked outcomes (considering complete importance rankings), providing methodological guidance for clinical trial design.

Core Contributions

1. Proposed analytical methods for patient-selected outcomes: Developed two statistical methods for analyzing patient-selected outcomes (mean comparison and proportion testing methods)
2. Systematically compared statistical performance of different methods: Through simulation studies, compared efficacy and Type I error rates of patient-selected outcomes, patient-ranked outcomes (composite DOOR), and traditional univariate analysis
3. Provided practical guidance for method selection: Based on patient preference distributions, outcome correlations, and other factors, offered specific recommendations for method selection
4. Extended patient-directed clinical trial methodology: Extended from binary outcomes in migraine to continuous outcomes, laying the foundation for broader application

Methodological Details

Task Definition

In two-arm randomized controlled trials, compare the effects of experimental (k=1) versus control (k=0) groups on multiple outcomes, where each patient has m continuous outcomes Yi = (Yi,1, ..., Yi,m) and corresponding preference information.

Patient-Selected Outcomes Methods

1. Mean Patient-Selected Outcomes Method

For outcome si selected by patient i, assuming Yi,j ~ N(μai,j, σ²ai), using Welch's t-test:

t = (Ȳ1 - Ȳ0) / √(d²0/n0 + d²1/n1)

where Ȳk = (1/nk)Σ1{ai = k}Yi,si

Key conditions: To ensure correct distribution under the null hypothesis:

• Condition 1: n1,j/n1 = n0,j/n0 (achieved through stratified randomization by selected outcome)
• Condition 2: μj = 0 (as in placebo-controlled trials)

2. Proportion Testing Method

Define binary outcome: Y*i = 1{Yi,si > λsi}, where λsi is the minimum clinically important difference (MCID) Use Wald test to compare proportions achieving clinically meaningful improvement between groups.

Patient-Ranked Outcomes Methods

Composite DOOR Method

For each pair of patients (i0, i1), define individual outcome DOOR indicator:

Zi0,i1(j) = {
  1    if Yi1,j - Yi0,j > MCIDj
  0    if Yi1,j - Yi0,j < MCIDj  
  0.5  if |Yi1,j - Yi0,j| ≤ MCIDj
}

Through iterative algorithm considering patient ranking hierarchy, from minimal common ranking set to maximal set, determine composite DOOR.

Estimator: θ̂ = (1/n0n1)ΣΣZi0,i1

Weighted Winning Probability (WWP) Method

Stratify by patients' preferred outcomes, calculate winning probability for each stratum, then compute weighted average: θ̂WT = Σp̂jθ̂j

where p̂j is the proportion of patients selecting outcome j as most important.

Technical Innovations

1. Theoretical foundation for stratified randomization: Clarified theoretical conditions for patient-selected outcomes analysis
2. Extension to continuous outcomes: Extended patient-selected outcomes from binary to continuous outcomes
3. Systematic comparison framework: Established statistical framework for fair comparison among different methods

Experimental Setup

Simulation Design

Based on multiple sclerosis clinical trial design, considering three outcomes: fatigue, pain, depression.

Data Generation Mechanism

• Sample size: 60 patients, 1:1 randomization
• Outcome distribution: Trivariate normal distribution with covariance matrix considering low (0.25), medium (0.55), high (0.75) correlations
• Patient preferences:
  • Unequal preferences: (p123, p132, p213, p231, p312, p321) = (0.42, 0.17, 0.24, 0.05, 0.08, 0.04)
  • Equal preferences: each ranking = 1/6

Simulation Scenarios

8 scenarios covering different treatment effect patterns:

• S1: No effect (null hypothesis)
• S2: Uniform improvement across all outcomes
• S3: Fatigue improvement only (59% of patients' first choice)
• S4: Depression improvement only (12% of patients' first choice)
• S5-S8: Various combinations of effects and interactions

Evaluation Metrics

• Type I error rate: Rejection rate under scenario 1
• Statistical power: Rejection rates under scenarios 2-8
• Monte Carlo standard error: <0.5% (based on 10⁴ replications)

Experimental Results

Main Results

Power Comparison Under Unequal Preferences

Key Findings

1. Type I error rate control: Mean patient-selected outcomes method adequately controls Type I error rate at 5% level
2. Power advantages: When outcomes preferred by most patients show substantial treatment effects (e.g., S3), patient-selected outcomes method shows comparable power to composite DOOR
3. Correlation impact: High correlation reduces composite DOOR power but has minimal impact on patient-selected outcomes method
4. Equal vs. unequal preferences: Composite DOOR shows greater advantages under equal preference conditions

Ablation Studies

Impact of Correlation Structure

• Low correlation: Composite DOOR power increases, patient-selected outcomes method remains essentially unchanged
• High correlation: Composite DOOR power decreases, patient-selected outcomes method remains stable

Impact of Marginal Effects

Setting MCID=0:

• Composite DOOR shows significantly increased power under unequal preferences
• WWP and proportion test show slight power reduction

Related Work

Patient-Focused Drug Development

• FDA guidance series 2018-2023 advancing PFDD
• Migraine treatment guidance formally recognizing patient-selected outcomes

Composite Endpoint Methodology

• DOOR method originally developed for infectious diseases
• PROOF method application in ALS
• Development of related methods such as win ratio

Multiple Outcomes Analysis

• Traditional multiple co-primary endpoints methods
• Hierarchical testing and gatekeeping procedures
• Methods incorporating patient preference weights

Conclusions and Discussion

Main Conclusions

1. Conditional advantages: Patient-selected outcomes method achieves comparable power to composite DOOR when outcomes preferred by most patients show treatment effects and outcomes have medium-to-high correlation
2. Implementation advantages: Patient-selected outcomes are easier to implement, requiring collection of only the most important outcome rather than complete rankings
3. Regulatory acceptance: Patient-selected outcomes have regulatory precedent, facilitating regulatory approval

Limitations

1. Theoretical condition requirements: Requires stratified randomization or specific null hypothesis conditions
2. Insufficient information utilization: May miss important treatment effects on non-preferred outcomes
3. Parametric method limitations: Current methods assume normal distribution, requiring extension to other distributions

Future Directions

1. Joint modeling: Develop joint analysis strategies borrowing information
2. Non-parametric methods: Extend analysis methods to different outcome types
3. Covariate incorporation: Consider effects of patient characteristics on preferences and outcomes
4. Continuous preference weights: Develop DOOR methods handling continuous preference weights

In-Depth Evaluation

Strengths

1. High practical value: Addresses real problems in clinical trials with clear application scenarios
2. Rigorous methodology: Provides complete theoretical foundation and statistical inference framework
3. Comprehensive comparison: Systematically compares multiple methods, considering various influencing factors
4. Reasonable simulation design: Based on realistic clinical trial scenarios with credible parameter settings

Limitations

1. Sample size constraints: Simulations based on relatively small sample size (60), requiring further verification for large-sample performance
2. Disease specificity: Primarily based on multiple sclerosis scenarios, requiring verification of applicability to other diseases
3. Preference stability assumption: Assumes patient preferences remain constant during trial, though actual changes may occur
4. MCID specification: Composite DOOR method depends on MCID specification, though its reasonableness is debatable

Impact

1. Methodological contribution: Provides important statistical methodology tools for patient-directed clinical trials
2. Regulatory significance: May influence development and updates of future regulatory guidance
3. Application prospects: Has broad application potential in clinical trials for complex diseases
4. Theoretical value: Advances application of personalized medicine in clinical trial design

Applicable Scenarios

1. Complex disease trials: Multiple sclerosis, rheumatoid arthritis, and other multi-symptom diseases
2. Large patient preference heterogeneity: Situations where different patients have markedly different priorities for symptom improvement
3. Continuous outcome measurement: Trials using standardized scales (e.g., PROMIS) for measurement
4. Flexible regulatory environment: Domains where regulators accept innovative endpoint designs

References

1. US Food and Drug Administration. FDA Patient-Focused Drug Development Guidance Series. 2018-2023.
2. Lu, Y., et al. A composite endpoint for treatment benefit according to patient preference. Statistics in Biopharmaceutical Research, 2022.
3. van Eijk, R.P., et al. Composite endpoint for ALS clinical trials based on patient preference. Journal of Neurology, Neurosurgery & Psychiatry, 2022.

This paper makes important methodological contributions to patient-directed drug development, particularly in balancing statistical efficiency with individualization of patient preferences. Despite certain limitations, its practical value and theoretical significance are considerable, with potential to advance clinical trial design toward a more patient-centered approach.