Phase Transitions of the Additive Uniform Noise Channel with Peak Amplitude and Cost Constraint

Basic Information

• Paper ID: 2510.12427
• Title: Phase Transitions of the Additive Uniform Noise Channel with Peak Amplitude and Cost Constraint
• Authors: Jonas Stapmanns, Luke Eilers, Catarina Dias, Tobias Kühn, Jean-Pascal Pfister
• Classification: cs.IT math.IT
• Publication Time/Conference: IEEE International Symposium on Information Theory (ISIT) 2025 (partial content)
• Paper Link: https://arxiv.org/abs/2510.12427

Abstract

Under what conditions is quantization optimal? This paper addresses this question in the context of an additive uniform noise channel with peak amplitude and cost constraints. We analytically compute the capacity-achieving input distributions as a function of noise level, average cost constraint, and cost function curvature. The study reveals that when the cost function is concave, the capacity-achieving input distribution is discrete; conversely, when the cost function is convex and the cost constraint is active, the support of the capacity-achieving input distribution spans the entire interval. For discrete capacity-achieving input distributions, we derive analytical expressions for channel capacity.

Research Background and Motivation

Core Problem

The core problem investigated is: under what conditions is quantized input information-theoretically optimal? This is a fundamental information-theoretic question involving the efficiency comparison between discrete and continuous input distributions.

Problem Significance

1. Theoretical Importance: Since Shannon introduced the concept of channel capacity, capacity-achieving input distributions have been a central topic in information theory research
2. Practical Applications: In many practical systems, particularly under peak amplitude constraints, capacity-achieving input distributions are often discrete
3. Biological Applications: In biological neural networks, signals are typically discrete (e.g., action potentials), and understanding the optimality conditions for discreteness has important implications

Limitations of Existing Methods

Existing research primarily employs non-constructive methods to analyze discreteness conditions, such as work by Das, Tchamkerten, and Fahs, but these methods are not conducive to detailed analysis of potential phase transition phenomena.

Research Motivation

This paper selects the additive uniform noise channel as the subject of study because it permits completely analytical treatment, enabling detailed investigation of phase transitions in capacity-achieving input distributions from discrete to continuous support sets.

Core Contributions

1. Complete Phase Transition Analysis: First comprehensive description of phase transition conditions for capacity-achieving input distributions from discrete to continuous support sets
2. Analytical Solutions: Provides complete analytical solutions for the additive uniform noise channel under peak amplitude and cost constraints
3. Phase Transition Mechanism Identification: Identifies two mechanisms causing phase transitions:
   • Cost function transitioning from concave to convex
   • Under convex cost functions, cost budget below critical threshold
4. Capacity Formula: Derives exact capacity expressions for the discrete case
5. Constructive Proof: Provides constructive proof methods that explicitly analyze phase transition phenomena

Methodology Details

Problem Formulation

Consider the additive uniform noise channel: $Y = X + N, \quad N \sim \text{Uniform}(-b, b)$

Constraints:

• Peak Amplitude Constraint: $P(X < 0) = P(X > 1) = 0$
• Cost Constraint: $\langle c_\alpha(x) \rangle \leq \bar{c}$

where the cost function $c_\alpha(x) = x^\alpha$ satisfies:

• $0 < \alpha < 1$: strictly concave function
• $\alpha = 1$: linear function
• $\alpha > 1$: strictly convex function

Optimization Framework

Employs Lagrange multiplier method to construct the optimization problem: $\mathcal{L}[p_X, \nu, \lambda] = \mathcal{L}_0[p_X, \nu] - \lambda\left(\int_0^1 p_X(x)c(x)dx - \bar{c}\right)$

where $\mathcal{L}_0$ contains mutual information terms and normalization constraints.

Smith Optimality Conditions

The capacity-achieving input distribution $p_X^*$ must satisfy:

• Inequality Constraint: $i(x; p_X^*) \leq I(p_X^*) + \lambda(c(x) - \bar{c})$ for all $x \in [0,1]$
• Equality Constraint: $i(x; p_X^*) = I(p_X^*) + \lambda(c(x) - \bar{c})$ for all $x \in S$ (support set)

where $i(x; p_X)$ is the marginal information density.

Technical Innovations

1. Case Classification Strategy

Separately handles cases based on whether noise parameter $r = 1/(2b)$ is an integer:

• $r \in \mathbb{N}$: noise outputs do not overlap
• $r \notin \mathbb{N}$: noise outputs overlap, requiring more complex analysis

2. Constructive Proof Method

Employs a "guess-and-verify" constructive approach:

1. Conjecture support set $S$
2. Solve equality constraints to obtain mass distribution
3. Verify inequality constraints
4. Prove uniqueness

3. Piecewise Linearity of Marginal Information Density

Lemma 13 proves that marginal information density is linear between adjacent support points, which is key to verifying inequality constraints.

Experimental Setup

Theoretical Verification

This is primarily theoretical work, with results verified through analytical derivation. Numerical verification uses the Blahut-Arimoto algorithm for comparison.

Parameter Settings

• Noise parameters: $r \in \{2, 2.4, 3.9, 4, 4.4, 6.2\}$
• Cost function exponents: $\alpha \in \{0.5, 0.7, 1, 1.5\}$
• Cost budgets: $\bar{c} \in (0, \bar{c}^*]$

Experimental Results

Main Results

Case I: Cost Constraint Inactive ($\bar{c} \geq \bar{c}^*$)

The capacity-achieving input distribution is discrete with number of mass points: $N_r = \begin{cases} n & \text{if } r \in \mathbb{N} \\ 2n & \text{if } r \notin \mathbb{N} \end{cases}$

where $n = \lfloor r \rfloor + 1$.

Case IIa: Cost Constraint Active with $\alpha \leq 1$, $r \in \mathbb{N}$

Mass distribution given by: $m_j = \frac{1}{z}e^{-\lambda^* c_j}, \quad z = \sum_{j=1}^{N_r} e^{-\lambda^* c_j}$

Case IIb: Cost Constraint Active with $\alpha \leq 1$, $r \notin \mathbb{N}$

There exist $n-1$ thresholds $0 < \theta_{n-2} < \ldots < \theta_0 < \bar{c}^*$, with support set determined piecewise according to cost budget.

Case III: $\alpha > 1$ with Active Cost Constraint

The support set of the capacity-achieving input distribution contains intervals where the cost function is strictly convex. Specifically, if $c(x)$ is strictly convex on $[0,1]$, the support set is the entire interval $[0,1]$.

Capacity Formula

For the discrete case, capacity is given by:

• $r \in \mathbb{N}$: $C = \log(n)$ (unconstrained) or $C = H(m)$ (constrained)
• $r \notin \mathbb{N}$: $C = \rho\log(n+1) + (1-\rho)\log(n)$ (unconstrained) or $C = \rho H(\hat{m}) + (1-\rho)H(\bar{m})$ (constrained)

where $\rho = r - \lfloor r \rfloor$ and $H(\cdot)$ is the entropy function.

Numerical Verification

Figure 7 shows that theoretical results perfectly align with numerical results from the Blahut-Arimoto algorithm, validating the correctness of the theoretical analysis.

Related Work

Classical Research

• Shannon (1948): Established foundational theory of channel capacity
• Smith (1971): Studied capacity-achieving input distributions for Gaussian additive noise channels
• Oettli (1974): Analyzed additive channels with piecewise constant noise

Discreteness Condition Research

• Das (2000), Tchamkerten (2004), Fahs & Abou-Faycal (2018): Investigated general conditions for discreteness of capacity-achieving input distributions
• Dytso et al. (2018-2020): Studied capacity-achieving input distributions under various constraints

Relationship to Related Work

This paper extends Oettli's work by introducing adjustable cost constraints to enable phase transition analysis from continuous to discrete. Compared to Tchamkerten's work, this paper provides necessary and sufficient conditions rather than only sufficient conditions, and considers bounded noise rather than unbounded noise.

Conclusions and Discussion

Main Conclusions

1. Phase Transition Mechanisms: Identifies two phase transition mechanisms: cost function curvature changes and cost budget variations
2. Support Set Structure: When the cost function is concave, the support set is always a subset of the original problem's support set
3. Equivalence: In the discrete case, channel capacity is equivalent to the capacity of a noiseless channel

Limitations

1. Noise Type Restriction: Only considers uniform noise; extension to other noise types requires further research
2. Cost Function Form: Primarily analyzes power function forms of cost functions
3. Dimensional Limitation: Only considers one-dimensional cases

Future Directions

1. Noise Extension: Extend results to more general additive noise, such as $p_N(N) \propto \exp(-|N/N_0|^\gamma)$
2. Constraint Relaxation: Consider soft peak constraints, such as $c(x) = x^\alpha + x^\beta$
3. High-Dimensional Extension: Study vector Gaussian channels with $L_1$ ball constraints
4. Biological Applications: Applications in biological systems such as neuroscience and gene expression

In-Depth Evaluation

Strengths

1. Theoretical Completeness: Provides complete analytical solutions and rigorous mathematical proofs
2. Methodological Innovation: Constructive proof methods make phase transition analysis possible
3. Result Depth: Not only provides phase transition conditions but also exact capacity formulas
4. Clear Presentation: Paper structure is clear and mathematical derivations are rigorous
5. Practical Value: Results provide guidance for understanding practical communication systems and biological systems

Weaknesses

1. Limited Scope: Results are restricted to specific noise models and constraint forms
2. Computational Complexity: Analysis for $r \notin \mathbb{N}$ cases is quite involved
3. Numerical Verification: Primarily relies on theoretical derivation with relatively simple numerical experiments

Impact

1. Theoretical Contribution: Provides new analytical framework for discreteness problems in information theory
2. Methodological Significance: Constructive proof methods may be applicable to other channel models
3. Interdisciplinary Value: Has potential applications in neuroscience, statistical learning, and other fields

Applicable Scenarios

1. Communication System Design: Optimize input distributions in power or amplitude-limited communication systems
2. Neural Coding: Understand optimality of discrete signals in biological neural networks
3. Statistical Inference: Select optimal prior distributions in constrained optimization problems

References

This paper cites classical literature in information theory, including Shannon's pioneering work, Smith's research on Gaussian channels, and recent important studies on discreteness of capacity-achieving input distributions. Particularly noteworthy are comparisons and extensions of work by Oettli, Tchamkerten, and others.

Overall Assessment: This is a high-quality theoretical information theory paper that rigorously addresses a fundamental problem through mathematical analysis. The paper's main value lies in providing complete analytical solutions and in-depth phase transition analysis, offering important insights for understanding optimality conditions of quantization. While results are limited to specific models, the methodology has general significance and may inspire broader research.