On estimation of weighted cumulative residual Tsallis entropy

Basic Information

• Paper ID: 2510.12442
• Title: On estimation of weighted cumulative residual Tsallis entropy
• Authors: Siddhartha Chakraborty, Asok K. Nanda (Indian Institute of Science Education and Research Kolkata)
• Classification: math.ST stat.TH (Statistics Theory)
• Publication Date: October 14, 2025
• Paper Link: https://arxiv.org/abs/2510.12442

Abstract

This paper investigates the weighted cumulative residual Tsallis entropy (WCRTE) as a generalization of weighted cumulative residual entropy. The article explores novel properties of the WCRTE measure, proposes several nonparametric estimators for this measure, and discusses their asymptotic properties. The performance of estimators is compared using mean squared error, while nonparametric estimation of the weighted cumulative residual entropy (WCRE) measure is also discussed. Based on estimators of these two measures, two uniformity tests are proposed and their power is compared with several popular testing methods.

Research Background and Motivation

Problem Background

1. Information-theoretic foundations: Shannon entropy serves as a core concept in information theory with important applications across multiple fields, but its differential entropy form has limitations (can be negative, undefined for distributions without densities, etc.)
2. Development of cumulative residual entropy: The cumulative residual entropy (CRE) proposed by Rao et al. (2004) overcomes the defects of differential entropy by using survival functions instead of density functions, exhibiting superior properties
3. Generalization of Tsallis entropy: The generalized entropy proposed by Tsallis (1988) is an important extension of Shannon entropy with parameter α, which reduces to Shannon entropy as α→1
4. Need for weighted information measures: In practical applications, it is necessary to consider not only the probabilistic information of events but also their utility or importance, motivating the introduction of weight functions

Research Motivation

The primary motivations of this paper are:

1. To deeply investigate the theoretical properties of the WCRTE measure
2. To develop effective nonparametric estimation methods
3. To provide practical tools for statistical inference (such as uniformity tests)

Core Contributions

1. Theoretical contributions:
   • Establishes sufficient conditions for the existence of WCRTE (requiring second moment existence when α>1)
   • Provides lower bound estimates for WCRTE
   • Gives equivalent representations of WCRTE
2. Estimation methods:
   • Proposes four nonparametric estimators for WCRTE
   • Develops corresponding estimators for WCRE
   • Proves consistency and asymptotic normality of estimators
3. Statistical applications:
   • Constructs uniformity tests based on WCRTE and WCRE estimators
   • Compares performance of different estimators through simulation
   • Validates the effectiveness of new testing methods

Methodology Details

Core Concept Definition

Weighted Cumulative Residual Tsallis Entropy (WCRTE) is defined as:

ξ^w_α(X) = 1/(α-1) ∫₀^∞ x[F̄(x) - F̄^α(x)]dx, 0 < α ≠ 1

where F̄(x) is the survival function and x is the linear weight function.

Key properties:

• Reduces to weighted cumulative residual entropy (WCRE) as α→1
• Relates to Gini mean difference when α=2
• Possesses scale transformation property: ξ^w_α(θX) = θ²ξ^w_α(X)

Estimator Design

1. Basic Estimator

Estimator based on empirical distribution function:

ξ̂^w_α(X) = 1/(2(α-1)) Σᵢ₌₁^(n-1) (X²₍ᵢ₊₁₎ - X²₍ᵢ₎)[(1-i/n) - (1-i/n)^α]

2. Vasicek-type Estimator

ξ^w_αV = 1/(4m(α-1)) Σᵢ₌₁ⁿ (X²₍ᵢ₊ₘ₎ - X²₍ᵢ₋ₘ₎)[1-i/n - (1-i/n)^α]

3. Ebrahimi-type Estimator

Introduces weight function Cᵢ to improve estimation at extreme points:

ξ^w_αE = 1/(2m(α-1)) Σᵢ₌₁ⁿ (X²₍ᵢ₊ₘ₎ - X²₍ᵢ₋ₘ₎)/Cᵢ [1-i/n - (1-i/n)^α]

4. Improved Estimator

ξ^w_αN = 1/(m(α-1)) Σᵢ₌₁ⁿ (X²₍ᵢ₊ₘ₎ - X²₍ᵢ₋ₘ₎)/C²ᵢ [1-i/n - (1-i/n)^α]

5. Linear Combination Estimator

ξ^w_αL = 1/(2(α-1)) · 1/n Σᵢ₌₁ⁿ X²₍ᵢ₎[1 - α(1-i/n)^(α-1)]

Asymptotic Properties

Consistency: All proposed estimators are consistent under appropriate conditions.

Asymptotic Normality: For the ξ^w_αL estimator:

√n(ξ^w_αL - ξ^w_α(X)) →ᵈ N(0, σ²)

where the expression for σ² is provided along with a consistent estimator.

Experimental Setup

Datasets

Simulation data are generated from the following theoretical distributions:

1. Exponential distribution: Exp(1), Exp(2)
2. Uniform distribution: U(0,1)
3. Weibull distribution: WE(2,1) (i.e., Rayleigh distribution)

Evaluation Metrics

• Bias: Eθ̂ - θ
• Mean Squared Error (MSE): E(θ̂ - θ)²

Experimental Parameters

• Sample sizes: n = 10, 20, 30
• Tsallis parameter: α = 2 (primary choice, as WCRTE existence conditions are less restrictive when α>1)
• Window sizes: m = 1, 2, ..., ⌊n/2⌋-1
• Number of simulations: 10,000

Experimental Results

Main Results

1. Basic Estimator Comparison

For estimators not requiring window parameters (ξ̂^w_α(X) and ξ^w_αL):

• Under Exp(1) and Exp(2) distributions, ξ^w_αL performs better
• Under U(0,1) and WE(2,1) distributions, ξ̂^w_α(X) is slightly superior with minimal differences
• As sample size increases, both bias and MSE decrease significantly

2. Window-dependent Estimator Performance

Simulation results reveal:

• ξ^w_αN performs best: Achieves minimum MSE in most cases
• ξ^w_αV performs worst: However, is least sensitive to window size m
• ξ^w_αE is intermediate: Performance falls between the two extremes

3. Window Size Selection Guidance

Based on simulation results, recommendations for window size selection are provided:

• For ξ^w_αV and ξ^w_αE: select m=n/2-1 when n≤20; select m=n/3 when n=30
• For ξ^w_αN: select m=n/4+1

Uniformity Test Results

Test Statistics

Uniformity tests based on WCRTE and WCRE estimators are constructed and compared with:

• Kolmogorov-Smirnov (KS) test
• Cramer-von Mises (CvM) test
• Anderson-Darling (AD) test
• Vasicek entropy test (ENT)

Power Comparison

Test power under seven alternative distributions shows:

• For Aⱼ-type alternatives (mean shift), the proposed test performs best
• For Bⱼ-type alternatives (variance reduction), the ENT test is superior
• For Cⱼ-type alternatives (variance increase), the proposed test significantly outperforms other methods
• WCRTE test (α=2) generally outperforms WCRE test (α→1)

Related Work

Development of Entropy Measures

1. Shannon entropy (1948): Foundation of information theory
2. Tsallis entropy (1988): Generalization for non-additive statistical mechanics
3. Cumulative residual entropy (Rao et al. 2004): Overcomes limitations of differential entropy
4. Weighted entropy (Belis & Guiasu 1968): Considers event utility
5. WCRTE (Chakraborty & Pradhan 2023): Subject of this paper

Development of Estimation Methods

• Vasicek method (1976): Entropy estimation based on slope estimation
• Ebrahimi improvement (1994): Introduces weight functions to improve extreme point estimation
• This paper proposes new improvements based on these foundations

Conclusions and Discussion

Main Conclusions

1. Theoretical completeness: Establishes a comprehensive theoretical framework for WCRTE, including existence conditions and bound estimates
2. Estimation methods: Proposes multiple effective nonparametric estimators, with ξ^w_αN showing the best overall performance
3. Statistical applications: The developed uniformity test exhibits superior performance under specific alternative hypotheses

Limitations

1. Parameter selection: Window size m selection still requires adjustment based on distribution type and sample size
2. Computational complexity: Some estimators are relatively sensitive to window parameters
3. Theoretical analysis: Complete asymptotic distribution is provided only for one estimator

Future Directions

1. Develop adaptive window selection methods
2. Extend to multivariate settings
3. Investigate applications to other statistical inference problems

In-depth Evaluation

Strengths

1. Solid theoretical contributions: Provides comprehensive theoretical analysis including existence, consistency, and asymptotic normality
2. Strong methodological innovation: Proposes substantial improvements based on classical Vasicek and Ebrahimi methods
3. Comprehensive experimental design: Thoroughly evaluates method performance through simulations with multiple distributions and sample sizes
4. Clear practical value: Uniformity test possesses practical statistical significance
5. Clear and rigorous presentation: Detailed mathematical derivations and comprehensive experimental results

Weaknesses

1. Imbalanced theoretical analysis: Asymptotic distribution provided only for ξ^w_αL; theoretical analysis of other estimators is relatively weak
2. Limited computational guidance: While empirical formulas for window selection are provided, they lack theoretical justification
3. Limited application scope: Only considers uniformity testing; other statistical inference problems are not explored
4. Limited comparison baselines: Lacks comparison with other entropy estimation methods in estimator comparisons

Impact

1. Academic value: Provides new theoretical tools for the intersection of information theory and statistics
2. Practical value: Proposed estimators and testing methods can be directly applied to data analysis
3. Reproducibility: Clear experimental setup with easily reproducible results

Applicable Scenarios

1. Reliability analysis: Utilizes weighted properties to analyze heavy-tail risks
2. Quality control: Uniformity test has important applications in random number generation verification
3. Information measurement: Scenarios requiring information measures that account for observation importance

References

The paper cites 28 relevant references covering important works in information theory, statistics, and reliability theory, providing a solid theoretical foundation. Key references include Shannon's (1948) foundational information theory work, Tsallis's (1988) entropy generalization, and Rao et al.'s (2004) cumulative residual entropy theory.

Overall Assessment: This is a high-quality statistical theory paper that makes substantial contributions to the field of weighted information measures. The theoretical analysis is rigorous, the experimental design is comprehensive, and it possesses good academic value and application prospects.