Title:

Tight Lower Bound on Differential Entropy for Mixed Gaussian Distributions, Journal of Telecommunications and Information Technology, 2024, nr 2

Group publication title:

2024, nr 2, JTIT-artykuły

Creator:

Marconi, Abdelrahman ; Elghandour, Ahmed H. ; Elbayoumy, Ashraf D. ; Abdelaziz, Amr

Subject and Keywords:

differential entropy ; lower bound ; mixture random variable ; multimodal Gaussian

Abstrakt:

In this paper, a tight lower bound for the differential entropy of the Gaussian mixture model is presented. First, the probability model of mixed Gaussian distribution that is created by mixing both discrete and continuous random variables is investigated in order to represent symmetric bimodal Gaussian distribution using the hyperbolic cosine function, on which a tighter upper bound is set. Then, this tight upper bound is used to derive a tight lower bound for the differential entropy of the Gaussian mixture model introduced. The proposed lower bound allows to maintain its tightness over the entire range of the model's parameters and shows more tightness when compared with other bounds that lose their tightness over certain parameter ranges. The presented results are then extended to introduce a more general tight lower bound for asymmetric bimodal Gaussian distribution, in which the two modes have a symmetric mean but differ in terms of their weights.

Publisher:

National Institute of Telecommunications

Date:

2024

Resource Type:

artykuł

Resource Identifier:

ISSN 1509-4553, on-line: ISSN 1899-8852

DOI:

10.26636/jtit.2024.2.1444

ISSN:

1509-4553

eISSN:

1899-8852

Source:

Journal of Telecommunications and Information Technology

Language:

ang

License:

CC BY 4.0

rights owner:

Instytut Łączności - Państwowy Instytut Badawczy