Response to Comment from Robert Cousins on Confidence intervals for the Poisson distribution

Basic Information

• Paper ID: 2510.25162
• Title: Response to Comment from Robert Cousins on Confidence intervals for the Poisson distribution
• Author: Frank C. Porter (California Institute of Technology)
• Classification: physics.data-an (Physics Data Analysis)
• Publication Date: November 3, 2025 (v2 version)
• Paper Link: https://arxiv.org/abs/2510.25162

Abstract

This is a response article to Robert Cousins' commentary. The core controversy concerns whether the likelihood function should include non-physical parameter values (such as negative signal strength) within the frequentist statistical framework. Cousins argues against this inclusion, while Porter advocates that such inclusion is useful when discussing descriptive statistics.

Research Background and Motivation

Core Problem

In constructing confidence intervals for the Poisson distribution, when the observed value n is smaller than the known background b, the signal strength parameter θ = n - b becomes negative. This raises a fundamental question: Within the frequentist statistical framework, should the likelihood function be defined in physically impossible parameter regions (θ < 0)?

Importance of the Problem

1. Theoretical Level: Involves the essential distinction between frequentist and Bayesian statistics, relating to the foundational theory of statistical inference
2. Practical Level: In high-energy physics experiments, negative signal estimates resulting from background fluctuations occur frequently; how to correctly report such measurements is a practical issue
3. Methodological Level: Affects the definition of sufficient statistics and the application of maximum likelihood estimation

Core of the Controversy

• Cousins' Position: The likelihood function should be considered undefined in non-physical regions because the physical model does not exist in that region
• Porter's Position: When describing measurement results, allowing the likelihood function to extend into non-physical regions is useful and does not violate physical constraints

Limitations of Existing Approaches

Traditional statistics textbooks largely remain silent on this issue, lacking explicit guiding principles. This leads to confusion in practice and has even caused some to mistakenly believe that frequentist statistical methods are "incorrect."

Core Contributions

1. Clarified the descriptive nature of frequentist statistics: Emphasizes that frequentist statistics provides a description of measurements rather than belief statements about parameters
2. Argued for the rationality of defining likelihood functions in non-physical regions: Through the concept of sufficient statistics, demonstrates that θ̂ = n - b contains more information than max(n - b, 0)
3. Clarified the relationship between descriptive and inferential statistics: Descriptive statistics can provide information for inference but do not themselves constitute degree-of-belief statements
4. Provided practical methodological guidance: Supports reporting negative signal estimates in experiments (e.g., θ̂ = -2 ± 1)

Detailed Methodology

Mathematical Formulation of the Problem

Consider sampling a random variable N = 0, 1, 2, ... from a Poisson distribution, where the mean μ equals the signal strength θ ≥ 0 plus the known background b (≥ 0):

$f(n; θ, b) = \frac{μ^n}{n!}e^{-μ} = \frac{(θ + b)^n}{n!}e^{-θ-b}$

where n is a possible value of N.

Definition of the Likelihood Function

Porter defines the likelihood function as:

$L(θ; n) = \frac{μ^n}{n!}e^{-μ} = \frac{(θ + b)^n}{n!}e^{-θ-b}$

Key Point: This function is mathematically defined for any μ ≥ 0, even when θ = μ - b < 0.

Sufficient Statistics Argument

Porter's core argument is based on the concept of sufficient statistics:

1. N - b is a sufficient statistic for θ: Contains all information in the data about θ
2. max(N - b, 0) is not a sufficient statistic: Forcing the estimate to be non-negative loses information
3. Therefore: θ̂ = n - b as a descriptive statistic is more informative than the truncated version

Descriptive Interpretation of Frequentist Statistics

Porter's position is:

• Frequentist statistical results are descriptive: They describe the measurement itself, not the true value of the parameter
• Should not be interpreted as degree-of-belief statements: Probability statements only refer to long-run frequency properties
• Can provide information for inference: When observing θ̂ = -2, one can infer (in the degree-of-belief sense) that θ is "likely" very small

Contrast with Bayesian Approach

Within the Bayesian framework, there is no controversy:

• The prior distribution naturally restricts parameters to the physical region
• Belief in non-physical regions is zero
• Physical constraints are implemented through the prior rather than the likelihood function

Technical Innovations

1. Conceptual Innovation

Distinguishing "Sampling Distribution" from "Domain of Definition of Likelihood Function":

• The sampling distribution always assumes θ ≥ 0 (physical constraint)
• The likelihood function as a mathematical object can be defined over a larger range
• This distinction allows using all mathematical tools without violating the physical model

2. Terminological Flexibility

Porter proposes that if statistical authorities prohibit defining "likelihood" in non-physical regions, it could be called "descriptionhood" and "maximum descriptionhood," emphasizing its nature as a descriptive tool.

3. Methodological Consistency

This approach remains consistent across different distributions:

• Poisson distribution: θ̂ = n - b can be negative
• Normal distribution: Similar considerations discussed in reference 4
• Maintains universality and consistency of statistical methods

Theoretical Analysis

Nature of Frequentist Statistics

Porter cites the classical work by Kendall et al. (Volume 2A, sections 26.58-26.78), particularly the statement in section 26.69:

"The failure of the frequency method to provide statements about the credibility of hypotheses is almost axiomatic, since frequentists are unwilling to accept any probability concept lacking a frequency interpretation."

Here "probability₁" refers to degree-of-belief probability.

Dangers of Confusion

Porter points out that misinterpreting frequentist statistics as inferential statements has caused considerable confusion:

Typical Case: In 2015, the editor of the psychology journal Basic and Applied Social Psychology claimed p-values were "invalid" and banned their use, a consequence of confusing the nature of frequentist statistics.

Status of Inference

While Porter emphasizes that frequentist statistics is descriptive, he acknowledges:

• When measurement yields θ̂ = n - b, it is reasonable to infer that θ is "more likely" to be close to θ̂
• This inference enters the degree-of-belief domain
• When observing negative θ̂, one can infer θ is "probably" small (but not negative)

Experimental/Application Scenarios

High-Energy Physics Experiments

Although the paper contains no specific experimental results, the background comes from high-energy physics:

• Searching for rare signals (e.g., new particles)
• Known background event count b
• Observed total event count n
• Need to estimate signal strength θ

Practical Situations

When background fluctuations cause n < b:

• Traditional Approach: May report θ̂ = 0 or not report at all
• Porter's Advocacy: Report θ̂ = n - b (negative value) with confidence interval
• Advantage: Preserves all information from the sufficient statistic

Reporting Example

Porter argues it is entirely acceptable to report:

"The measurement result is θ̂ = -2 ± 1, although it is known that θ > 0"

Such a report:

• Completely describes the measurement
• Allows readers to infer that θ is probably close to zero
• Does not claim that θ is actually negative

Related Work

Statistical Literature

1. Shao (2003): Mathematical Statistics textbook contains examples supporting Cousins' viewpoint, but appears in chapters with more Bayesian content, lacking detailed contextual discussion
2. Kendall et al. (1999): Kendall's Advanced Theory of Statistics recognizes problems and controversies in various estimation methods, with views consistent with Porter's position
3. Narsky & Porter (2014): Discusses similar issues in the case of normal distribution

Historical Background of the Controversy

• The philosophical debate between frequentist and Bayesian statistics has a long history
• Questions about the domain of definition of likelihood functions are often avoided in statistics textbooks
• Practical confusion leads to misunderstanding and misuse of statistical methods

Conclusions and Discussion

Main Conclusions

1. Within the frequentist statistical framework, allowing the likelihood function to be defined in non-physical parameter regions is useful and rational
2. This does not violate the physical model: The sampling distribution always assumes physical constraints
3. Consideration of sufficient statistics supports this approach: N - b contains more information than max(N - b, 0)
4. Frequentist statistics should be viewed as descriptive: Providing descriptions of measurements rather than beliefs about parameters

Porter's Position

• Acknowledges that his position may be more extreme than historical precedent
• But believes this position helps maintain conceptual clarity
• No disagreement exists within the Bayesian framework; controversy is limited to the interpretation of frequentist statistics

Practical Value

This methodological approach:

• Allows more complete reporting of experimental results
• Avoids information loss
• Maintains consistency of statistical methods
• Facilitates subsequent Bayesian analysis or meta-analysis

Future Directions

Not explicitly stated in the paper, but implicit directions include:

• Further clarification of the relationship between frequentist and Bayesian statistics
• Promotion of rational statistical reporting practices in experimental physics
• Education of statistical users to avoid common misconceptions

In-Depth Evaluation

Strengths

1. Conceptual Clarity:
   • Clearly distinguishes descriptive from inferential statistics
   • Clearly defines the nature and limitations of frequentist statistics
   • The sufficient statistics argument is persuasive
2. Practical Orientation:
   • Addresses real problems encountered in actual experiments
   • Provides actionable methodological guidance
   • Avoids information loss
3. Solid Theoretical Foundation:
   • Cites classical statistical literature to support viewpoints
   • Mathematical arguments for sufficient statistics are rigorous
   • Comparison with Bayesian methods is appropriate
4. Clear Writing:
   • Logical structure is clear
   • Acknowledges the reasonableness of different viewpoints
   • Professional and measured tone

Limitations

1. Potential Philosophical Controversy:
   • The question of how the likelihood function "should" be defined has inherent philosophical components
   • Different statistical schools may have irreconcilable differences
   • Porter acknowledges that consensus may be impossible
2. Potential Confusion in Practice:
   • Allowing negative estimates may be misunderstood by non-specialists
   • Requires additional explanation and education
   • May cause confusion in certain application domains
3. Limited Empirical Evidence:
   • Primarily conceptual and theoretical arguments
   • Lacks case studies with actual data analysis
   • Does not demonstrate application effects in specific experiments
4. Insufficient Discussion of Cousins' Position:
   • Mainly states own position
   • Fewer specific responses to opposing arguments
   • May not fully understand the other side's deeper concerns

Impact

Positive Aspects:

• Provides theoretical support for statistical practice in high-energy physics and related fields
• Helps clarify the nature of frequentist statistics
• May influence experimental result reporting standards

Limitations:

• May have direct impact only in specific fields (e.g., particle physics)
• Statistical community may continue to have disagreements
• Requires accompanying education and promotion to change practice

Reproducibility

• This is a theoretical/methodological paper not involving experimental reproducibility
• Concepts and arguments are clear and easy to understand and apply
• Mathematical derivations are straightforward

Applicable Scenarios

Most Suitable Scenarios:

1. High-Energy Physics Experiments: Signal searches, background estimation
2. Low Count Rate Experiments: Poisson statistics dominate
3. Occasions Requiring Sufficient Statistics Reporting: Complete information transmission is important

Less Suitable Scenarios:

1. Public Science Communication: Negative signal strength may cause misunderstanding
2. Regulatory Decision Environments: May require more conservative reporting approaches
3. Audiences Unfamiliar with Statistics: Requires additional explanation

Summary of Methodological Contributions

Porter's core contributions are:

1. Clarified the boundary between "description" and "inference" in frequentist statistics
2. Provided sufficient statistics theoretical support for defining likelihood functions in non-physical regions
3. Provided practical reporting guidelines: Allowing reporting of negative parameter estimates
4. Clarified the relationship with Bayesian methods: No controversy exists within the latter framework

This is an important methodological paper that, while unlikely to end the controversy, provides valuable perspective for statistical practice.

References

1 Robert D. Cousins. Comment on Frank Porter, "Confidence intervals for the Poisson distribution", 2025. arXiv:2509.17339

2 Frank C. Porter. Confidence intervals for the Poisson distribution, 2025. arXiv:2509.02852

3 Jun Shao. Mathematical Statistics, 2nd edition. Springer-Verlag, 2003

4 I. Narsky and F. C. Porter. Statistical analysis techniques in particle physics, 2014

5 Alan Stuart et al. Kendall's Advanced Theory of Statistics, Volume 2A, 1999

6 David Trafimow and Michael Marks. Editorial. Basic and Applied Social Psychology, 37(1):1–2, 2015

Overall Assessment: This is a methodologically sound paper with clear concepts and forceful arguments that provides valuable perspective on practical problems in statistical practice. While it may not convince everyone, its argument based on sufficient statistics and its clear exposition of the nature of frequentist statistics provide important guidance for experimental physicists. The paper's primary value lies in clarifying concepts and providing practical methodology rather than resolving a technical problem with a definitive answer.