A reverse entropy power inequality for i.i.d. log-concave random variables

Basic Information

• Paper ID: 2510.09206
• Title: A reverse entropy power inequality for i.i.d. log-concave random variables
• Authors: Zhen Fu, Jiange Li
• Classification: math.PR cs.IT math.FA math.IT
• Submission Date: October 10, 2025 (arXiv submission)
• Paper Link: https://arxiv.org/abs/2510.09206

Abstract

This paper investigates the reverse entropy power inequality for independent identically distributed (i.i.d.) log-concave random variables. It is proven that for i.i.d. log-concave random variables $X$ and $Y$, the entropy increment $h_\infty(X+Y)-h_\infty(X)$ achieves its maximum when they follow an exponential distribution, where $h_\infty(\cdot)$ denotes the $\infty$-order Rényi entropy. Additionally, the paper establishes analogous results for integer-valued log-concave random variables.

Research Background and Motivation

1. Core Problem: This paper addresses the reverse entropy power inequality (reverse EPI) for log-concave random variables, specifically determining under which distributions the entropy increment $h(X+Y)-h(X)$ achieves its maximum value.
2. Significance:
   • The entropy power inequality (EPI) is a fundamental inequality in information theory and probability theory with important applications in channel capacity research
   • EPI is closely related to the Brunn-Minkowski inequality in convex geometry
   • Research on reverse inequalities helps understand extremal properties of log-concave distributions
3. Limitations of Existing Research:
   • Bobkov and Madiman proved a reverse EPI for log-concave random vectors, but requiring volume-preserving linear maps
   • Results by Cover and Zhang do not assume independence, but the bounds are not tight
   • Sharp reverse EPI for the i.i.d. case remains an open problem
4. Research Motivation: To verify a folklore conjecture (Conjecture 1.1): for i.i.d. log-concave random vectors, the entropy increment achieves its maximum when following an exponential distribution.

Core Contributions

1. Main Theoretical Result: Proves that for independent real-valued log-concave random variables $X$ and $Y$, the $\infty$-order Rényi entropy increment $h_\infty(X+Y)-h_\infty(X)$ achieves its maximum when following an exponential distribution
2. Technical Innovations:
   • Establishes convolution inequalities for unimodal functions (Lemma 3.4)
   • Develops proof techniques based on rearrangement and majorization theory
   • Provides simplified proofs based on optimal transport
3. Extension to Discrete Case: Establishes analogous results for integer-valued log-concave random variables, proving corresponding entropy inequalities
4. Second-Order Rényi Entropy Results: As a byproduct, proves $h_2(X+Y) \leq h_2(X) + d\log 2$, with equality holding for exponential distributions

Methodology Details

Problem Formulation

Given i.i.d. log-concave random variables $X$ and $Y$, the objective is to prove: $h_\infty(X+Y) \leq h_\infty(Z+W)$ where $Z$ and $W$ are independent exponential random variables satisfying $h_\infty(Z) = h_\infty(X)$ and $h_\infty(W) = h_\infty(Y)$.

Core Technical Framework

1. Rearrangement Theory

• Decreasing Rearrangement: For a measurable set $A \subseteq \mathbb{R}$, its decreasing rearrangement is defined as $A^\downarrow = [0,|A|)$
• Function Rearrangement: For a function $f$, define $f^\downarrow(x) = \sup\{\lambda \in \mathbb{R} : x \in \{f > \lambda\}^\downarrow\}$
• Preservation Property: Proves that log-concavity is preserved under rearrangement

2. Majorization Theory

A function $f$ is majorized by $g$, denoted $f \prec g$, if and only if for all $t \geq 0$: $\int_\mathbb{R} (f(x)-t)_+ dx \leq \int_\mathbb{R} (g(x)-t)_+ dx$

Key Lemma: If $f$ is majorized by $g$, and $\phi$ is a convex function, then: $\int_\mathbb{R} \phi(f(x))dx \leq \int_\mathbb{R} \phi(g(x))dx$

3. Hardy-Littlewood Type Inequalities

Lemma 2.3: For non-negative functions $f,g$ supported on $[a,b]$: $\int_a^b f(x)g(x)dx \geq \int_0^{b-a} f^\downarrow(x)g^\downarrow(b-a-x)dx$

Main Proof Strategy

Proof Path for Theorem 3.3:

1. Convolution Inequality for Unimodal Functions (Lemma 3.4): $\|f * g\|_\infty \geq \|f^\downarrow * g^\downarrow\|_\infty$
2. Monotone Density Case (Lemma 3.5): For independent log-concave random variables with monotone densities, proves inequalities for all orders of Rényi entropy
3. Optimal Transport Method: Constructs mapping $\phi = F_W^{-1} \circ F_Y$, utilizing its expansion property to complete the proof

Experimental Setup

Theoretical Verification

This paper is primarily theoretical work, verifying results through rigorous mathematical proofs. Main verifications include:

1. Extremal Case Analysis: Proves that exponential distribution indeed achieves equality conditions
2. Boundary Cases: Analyzes the extremal properties of geometric distribution in the discrete case
3. Asymptotic Behavior: Studies behavior as parameters approach limits

Specific Examples

• Geometric Distribution: For $f(k) = (1-\lambda)\lambda^k$, verifies the tightness of the inequality
• Exponential Distribution: Proves optimality in the continuous case

Experimental Results

Main Theoretical Results

Theorem 1.2 (Main Theorem): For independent real-valued log-concave random variables $X$ and $Y$, and corresponding exponential random variables $Z$ and $W$: $h_\infty(X+Y) \leq h_\infty(Z+W)$

For the i.i.d. case: $h_\infty(X+Y) \leq h_\infty(X) + 1$

Proposition 3.1 (Second-Order Case): $h_2(X+Y) \leq h_2(X) + d\log 2$

Discrete Case Results

Theorem 4.3: For i.i.d. integer-valued log-concave random variables with monotone probability mass functions: $H_\infty(X+Y) < H_\infty(X) + 1$

Proposition 4.1: $H_2(X+Y) < H_2(X) + \log 2$

Tightness Analysis

• Geometric distribution $f(k) = (1-\lambda)\lambda^k$ achieves the lower bound in the discrete case
• As $\lambda \to 1$, the ratio approaches the optimal constant $1/e$

Related Work

1. Classical EPI: Shannon-Stam entropy power inequality and its applications in information theory
2. Reverse Inequalities: Bobkov-Madiman reverse Brunn-Minkowski and reverse EPI
3. Log-Concave Distributions: Research by Ball et al. in the two-dimensional case
4. Rényi Entropy: Melbourne-Tkocz research on the $X-Y$ case
5. Discrete Analogues: Melbourne-Palafox-Castillo research on integer-valued cases

Conclusions and Discussion

Main Conclusions

1. Confirms that for $\infty$-order Rényi entropy, exponential distribution indeed maximizes entropy increment
2. Establishes a unified theoretical framework for continuous and discrete cases
3. Develops applications of rearrangement and majorization theory to entropy inequalities

Limitations

1. Currently only resolves the $\infty$-order and second-order Rényi entropy cases
2. The transition point $p_0$ for general $p$-order Rényi entropy remains undetermined
3. Complete characterization in high dimensions remains an open problem

Future Directions

1. Determine the critical point $p_0$ for Rényi entropy order
2. Investigate the case of high-dimensional log-concave random vectors
3. Explore deeper connections with Young convolution inequalities

In-Depth Evaluation

Strengths

1. Theoretical Rigor: Complete proofs with strong technical content, employing multiple advanced analytical tools
2. Problem Importance: Resolves fundamental problems in probability theory and information theory
3. Methodological Innovation: Comprehensive approach combining rearrangement theory, majorization theory, and optimal transport
4. Result Completeness: Addresses both continuous and discrete cases

Weaknesses

1. Scope of Applicability: Currently applicable only to specific orders of Rényi entropy
2. Computational Complexity: Practical computation of theoretical results may be complex
3. Application Orientation: Lacks concrete application scenarios and numerical experiments

Impact

1. Theoretical Contribution: Provides new theoretical tools for probability theory and information theory
2. Methodological Value: Techniques from rearrangement and majorization theory applicable to other problems
3. Subsequent Research: Provides important foundation for related research

Applicable Scenarios

1. Information Theory Research: Channel capacity and coding theory
2. Probability Theory: Study of extremal properties of log-concave distributions
3. Convex Geometry: Connections with Brunn-Minkowski type inequalities
4. Statistics: Concentration inequalities in high-dimensional statistics

References

The paper cites 18 important references covering classical and frontier work in entropy power inequalities, convex geometry, probability theory, and other fields, providing a solid theoretical foundation for the research.