Simultaneous Frequentist Calibration of Confidence Regions for Multiple Functionals in Constrained Inverse Problems

Basic Information

• Paper ID: 2510.11708
• Title: Simultaneous Frequentist Calibration of Confidence Regions for Multiple Functionals in Constrained Inverse Problems
• Authors: Pau Batlle, Pratik Patil, Michael Stanley, Javier Ruiz Lupon, Houman Owhadi, Mikael Kuusela
• Classification: math.ST stat.TH
• Publication Date: October 13, 2025
• Paper Link: https://arxiv.org/abs/2510.11708

Abstract

Many scientific analyses require simultaneous comparison of multiple functionals of an unknown signal, necessitating the construction of multidimensional confidence regions with guaranteed simultaneous frequentist coverage under structural constraints (such as non-negativity, shape, or physics-based constraints). This paper unifies and extends optimization-based confidence region construction methods for constrained linear inverse problems through the perspective of statistical hypothesis testing inversion. The paper first reviews the historical development of optimization-based confidence intervals in the single-functional setting, from "strict bounds" to the Burrus conjecture and its recent refutation through the hypothesis testing inversion framework. It then extends this framework to the multi-functional setting. The framework enables: (i) improved calibration constants over previous methods, yielding smaller confidence regions while maintaining frequentist coverage; (ii) tractable multidimensional confidence regions that need not be hyperrectangular, better capturing functional dependence structure; (iii) generalization from Gaussian error distributions to general log-concave error distributions.

Research Background and Motivation

Problem Definition

This paper studies the construction of simultaneous confidence regions for multiple functionals in linear inverse problems. Consider the linear inverse problem: $y = Kx^* + \varepsilon$ where $y \in \mathbb{R}^n$ is the observation, $x^* \in \mathbb{R}^p$ is the unknown parameter, $K \in \mathbb{R}^{n \times p}$ is the known forward operator, and $\varepsilon \in \mathbb{R}^n$ is random noise.

Research Motivation

1. Scientific Need: Many scientific analyses require simultaneous inference of multiple linear functionals $Hx^*$ of the unknown signal, rather than estimating the entire high-dimensional parameter $x^*$
2. Constraint Information: The true parameter $x^*$ typically satisfies constraints based on prior physical knowledge (such as non-negativity $x^* \geq 0$)
3. Simultaneous Coverage: Guaranteeing simultaneous frequentist coverage for all functionals, not merely marginal coverage

Limitations of Existing Methods

1. Conservatism: Traditional simultaneous strict bounds (SSB) methods are overly conservative, first constructing confidence sets for $x^*$ then mapping to functional space
2. Rectangular Restriction: Existing methods typically produce hyperrectangular confidence regions, failing to capture dependence structure between functionals
3. Calibration Issues: Historical methods such as the Burrus conjecture lack rigorous theoretical guarantees

Core Contributions

1. Unified Framework: Unifies single-functional and multi-functional constrained confidence region construction methods through the hypothesis testing inversion perspective
2. Theoretical Breakthroughs:
   • Proves convexity of quantile functions for $\lambda_u^2$ and $\lambda^1$ test statistics
   • Determines optimal solution locations for quantile optimization problems
   • Establishes stochastic dominance relationships between test statistics
3. Practical Algorithms:
   • Provides optimal calibration constants for non-negativity constraints
   • Develops TFM reduction methods for high-dimensional problems
   • Proposes row space/null space separation techniques
4. Performance Improvements: Significantly reduces confidence region volume compared to classical methods while maintaining nominal coverage rates

Methodology Details

Task Definition

Given matrix $H \in \mathbb{R}^{k \times p}$, the goal is to construct a finite-sample $1-\alpha$ frequentist confidence set $R_\alpha(y) \subseteq \mathbb{R}^k$ for the unknown vector $Hx^* \in \mathbb{R}^k$ such that: $P_{y \sim P_x}(Hx \in R_\alpha(y)) \geq 1-\alpha$ holds for all $x \in \mathcal{X}$ (the constraint set).

Hypothesis Testing Inversion Framework

Single-Functional Case Review

For each $\mu \in \mathbb{R}$, consider the hypothesis test: $H_0: x^* \in \Phi_\mu \cap \mathcal{X} \quad \text{vs} \quad H_1: x^* \in \mathcal{X} \setminus \Phi_\mu$ where $\Phi_\mu = \{x \in \mathbb{R}^p: h^T x = \mu\}$.

Multi-Functional Extension

For $\mu \in \mathbb{R}^k$, define $\Phi_\mu = \{x \in \mathbb{R}^p: Hx = \mu\}$, and the hypothesis test becomes: $H_0: x^* \in \Phi_\mu \cap \mathcal{X} \quad \text{vs} \quad H_1: x^* \in \mathcal{X} \setminus \Phi_\mu$

Three Test Statistics

The paper analyzes three test statistics:

1. Constrained Second Term $\lambda_c^2(\mu, y)$: $\lambda_c^2(\mu, y) = \min_{Hx=\mu, Ax \leq b} \|Kx - y\|_2^2 - \min_{Ax \leq b} \|Kx - y\|_2^2$
2. Unconstrained Second Term $\lambda_u^2(\mu, y)$: $\lambda_u^2(\mu, y) = \min_{Hx=\mu, Ax \leq b} \|Kx - y\|_2^2 - \min_{x \in \mathbb{R}^p} \|Kx - y\|_2^2$
3. Single Term $\lambda^1(\mu, y)$: $\lambda^1(\mu, y) = \min_{Hx=\mu, Ax \leq b} \|Kx - y\|_2^2$

Calibration Problem

For each test statistic, thresholds must be determined to guarantee $1-\alpha$ coverage:

• Pointwise Threshold: $d^*(\mu) = \sup_{Hx=\mu, Ax \leq b} Q_{x,1-\alpha}$
• Global Threshold: $D^* = \sup_{Ax \leq b} Q_{x,1-\alpha}$

where $Q_{x,1-\alpha}$ is the $(1-\alpha)$ quantile of $Z_x = \lambda(Hx, Kx + \varepsilon)$.

Key Theoretical Results

Convexity Theorem

Theorem 5.4: For any fixed $0 < \alpha < 1$, the quantile function $Q_u^2(x)$ is convex in $x$.

Maximization Theorem

Theorem 5.6 (Linear Constraints): Under linear constraints $Ax^* \leq b$, $\sup_{x \in P} Q_u^2(x) = \max_{i=1:m} Q_u^2(p_i)$ where $\{p_i\}_{i=1}^m$ is the set of extreme points of polyhedron $P$.

Theorem 5.7 (Cone Constraints): Under cone constraints $x^* \in C$, $\sup_{x \in C} Q_u^2(x) = Q_u^2(0)$

Experimental Setup

Numerical Example

Consider a non-negativity constrained problem: $y = Kx + \varepsilon, \quad \varepsilon \sim N(0, I), \quad x^* \geq 0$ where: $K = \begin{pmatrix} 2 & 1 & 1 \\ 0 & 1 & 1 \end{pmatrix}, \quad H = \begin{pmatrix} 1 & -1 & 0 \\ 0 & 1 & -1 \end{pmatrix}$

Comparison Methods

1. SSB_x: Simultaneous strict bounds $x$-description bounding box
2. SSB_μ: Simultaneous strict bounds $\mu$-description
3. QuantileZero_x/μ: Improved version using optimal constants
4. Bonferroni: Bonferroni-corrected product intervals
5. Split Method: Row space/null space separation technique

Evaluation Metrics

• Empirical Coverage Rate: Verified through $N = 10^5$ resamples
• Region Area: Computed using polar coordinate integration

Experimental Results

Main Results

Confidence Regions for Fixed Observations

For $y = (0,0)$ and $y = (20,10)$, the $\mu$-description method produces convex sets strictly contained within $x$-description bounding boxes, significantly reducing region area.

Coverage Rate and Area Distribution

• $x^* = (0,0,0)$: QuantileZero_μ method achieves approximately exact 68% coverage rate with minimal average area
• $x^* = (5,5,5)$: All methods achieve coverage, but $\mu$-description methods maintain significant area advantages

Quantified Improvements

1. Calibration Constant Improvement: For 68% and 95% confidence levels, optimal constants are 1.644 and 5.139 respectively, showing significant improvement over $\chi^2_2$ distribution values of 2.279 and 5.991
2. Area Reduction: $\mu$-description achieves approximately 30-50% average area reduction compared to $x$-description bounding boxes

Related Work

Historical Development

1. Burrus (1964): First proposed optimization-based methods for constrained confidence intervals
2. Rust & O'Leary (1986): Developed practical algorithms
3. Stark (1992): Introduced strict bounds methods
4. Tenorio et al. (2007): Developed TFM reduction techniques

Modern Advances

• Batlle et al. (2023): Refuted the Burrus conjecture through hypothesis testing inversion framework
• Constrained Inference Literature: Connections with $\chi^2$-bar distribution theory
• Conformal Prediction: Distinctions in objectives and assumptions

Conclusions and Discussion

Main Conclusions

1. Theoretical Contribution: Establishes a unified hypothesis testing inversion framework for multi-functional constrained confidence regions
2. Computational Advantages: Provides scalable algorithms for high-dimensional problems
3. Performance Improvements: Significantly reduces confidence region volume compared to classical methods

Limitations

1. $\lambda_c^2$ Statistic: Quantile functions lack convexity; maximization problems remain open
2. Computational Complexity: Extreme point search may be difficult in high dimensions
3. Pointwise Thresholds: Computing the entire function $d^*(\mu)$ is typically challenging

Future Directions

1. Non-Gaussian Extensions: Extend to general log-concave distributions
2. $\lambda_c^2$ Calibration: Develop calibration algorithms for constrained second-term statistics
3. Asymptotic Theory: Study large-sample properties
4. Application Domains: Extend to shape constraints and other statistical problems

In-Depth Evaluation

Strengths

1. Theoretical Rigor: Provides a complete mathematical framework including convexity proofs and optimality results
2. Practical Value: Develops scalable algorithms addressing high-dimensional practical problems
3. Unified Perspective: Unifies historically dispersed methods under the hypothesis testing inversion framework
4. Significant Improvements: Substantially reduces confidence regions while maintaining theoretical guarantees

Weaknesses

1. Theoretical Gaps: Complete theory for $\lambda_c^2$ statistics remains undeveloped
2. Computational Limitations: Computational complexity in certain high-dimensional cases
3. Limited Experiments: Numerical experiments are relatively simple, lacking complex real-world applications

Impact

1. Academic Contribution: Provides new theoretical foundations for uncertainty quantification in constrained inverse problems
2. Practical Applications: Broad application prospects in physical sciences, engineering, and other fields requiring constrained inference
3. Methodological Significance: The hypothesis testing inversion framework may inspire solutions to other statistical problems

Applicable Scenarios

• Simultaneous multi-functional inference in linear inverse problems
• Parameter estimation with physical constraints
• Scientific computing requiring strict frequentist guarantees
• Uncertainty quantification in high-dimensional constrained optimization problems

References

The paper cites 47 relevant references spanning constrained inference, inverse problems, optimization theory, and statistics, providing a solid theoretical foundation for the research.