More Web Proxy on the site http://driver.im/

article

Expressions for Rényi and Shannon entropies for bivariate distributions

Authors:

K. ZografosAuthors Info & Claims

Information Sciences—Informatics and Computer Science, Intelligent Systems, Applications: An International Journal, Volume 170, Issue 2-4

Pages 173 - 189

https://doi.org/10.1016/j.ins.2004.02.020

Published: 25 February 2005 Publication History

Abstract

Exact forms of Rényi and Shannon entropies are determined for 27 continuous bivariate distributions, including the Kotz type distribution, truncated normal distribution, distributions with normal and centered normal conditionals, natural exponential distribution, Freund's exponential distribution, Marshall and Olkin's exponential distribution, exponential mixture distribution, Arnold and Strauss's exponential distribution, McKay's gamma distribution, distribution with gamma conditionals, gamma exponential distribution, Dirichlet distribution, inverted beta distribution, distribution with beta conditionals, beta stacy distribution, Cuadras and Augé's distribution, Farlie Gumbel Morgenstern distribution, logistic distribution, Pearson type VII distribution, Pearson type II distribution, distribution with Cauchy conditionals, bilateral Pareto distribution, Muliere and Scarsini's Pareto distribution, distribution with Pareto conditionals and the distribution with Gumbel conditionals. We believe that the results presented here will serve as an important reference for scientists and engineers in many areas.

References

[1]

{1} N.A. Ahmed, D.V. Gokhale, Entropy expressions and their estimators for multivariate distributions, IEEE Transactions on Information Theory IT-35 (1989) 688-692.

[2]

{2} C. Arndt, Information Measures, Springer, Berlin, 2001.

[3]

{3} B.C. Arnold, E. Castillo, J.M. Sarabia, Conditional Specification of Statistical Models, Springer Verlag, New York, 1999.

[4]

{4} B.C. Arnold, D. Strauss, Bivariate distributions with exponential conditionals, Journal of the American Statistical Association 83 (1988) 522-527.

[5]

{5} G. Aulogiaris, K. Zografos, A maximum entropy characterization of symmetric Kotz type and Burr multivariate distributions, in press.

[6]

{6} J. Burbea, C.R. Rao, On the convexity of some divergence measures based on entropy functions, IEEE Transactions on Information Theory IT-28 (1982) 489-495.

[7]

{7} C.M. Cuadras, J. Augé, A continuous general multivariate distribution and its properties, Communication in Statistics--Theory and Methods 10 (1981) 339-353.

[8]

{8} G.A. Darbellay, I. Vajda, Entropy expressions for multivariate continuous distributions, IEEE Transactions on Information Theory IT-46 (2000) 709-712.

Digital Library

[9]

{9} M.H. De Groot, Optimal Statistical Decisions, McGraw-Hill, New York, 1970.

[10]

{10} J. Freund, A bivariate extension of the exponential distribution, Journal of the American Statistical Association 56 (1961) 971-977.

[11]

{11} A. Golan, G. Judge, D. Miller, Maximum Entropy Econometrics: Robust Estimation with Limited Data, Wiley, New York, 1996.

[12]

{12} A. Golan, J.M. Perloff, Comparison of maximum entropy and higher-order entropy estimators, Journal of Econometrics 107 (2002) 195-211.

[13]

{13} I.S. Gradshteyn, I.M. Ryzhik, Table of Integrals, Series and Products, sixth ed., Academic Press, San Diego, 2000.

[14]

{14} B. Harris, Entropy, in: S. Kotz, N.L. Johnson (Eds.), Encyclopedia of Statistical Sciences, vol. 2, Wiley, New York, 1982, pp. 512-516.

[15]

{15} S. Kotz, Multivariate distributions at a cross road, in: G.P. Patil, S. Kotz, J.K. Ord (Eds.), Statistical Distributions in Scientific Work, 1, D. Reidel Publishing Company, Dordrecht, 1975.

[16]

{16} S. Kotz, N. Balakrishnan, N.L. Johnson, Continuous Multivariate Distributions, volume 1: Models and Applications, second ed., Wiley, New York, 2000.

[17]

{17} S. Kotz, S. Nadarajah, Extreme Value Distributions: Theory and Applications, Imperial College Press, London, 2000.

[18]

{18} D.V. Lindley, N.D. Singpurwalla, Multivariate distributions for the life lengths of components of a system sharing a common parameter, Journal of Applied Probability 23 (1986) 418-431.

[19]

{19} A.W. Marshall, I. Olkin, A generalized bivariate exponential distribution, Journal of Applied Probability 4 (1967) 291-302.

[20]

{20} A.T. McKay, Sampling from batches, Journal of the Royal Statistical Society, Supplement 1 (1934) 207-216.

[21]

{21} G.A. Mihram, A.R. Hultquist, A bivariate warning-time/failure-time distribution, Journal of the American Statistical Association 62 (1967) 589-599.

[22]

{22} P. Muliere, M. Scarsini, Characterization of a Marshall-Olkin type class of distributions, Annals of the Institute of Statistical Mathematics 39 (1987) 429-441.

[23]

{23} S. Nadarajah, K. Zografos, Rényi information and an intrinsic distribution measure for univariate distributions, Information Sciences 155 (2003) 119-138.

Digital Library

[24]

{24} A. Rényi, On measures of entropy and information, in: Proceedings of the 4th Berkeley Symposium on Mathematical Statistics and Probability, vol. I, University of California Press, Berkeley, 1961, pp. 547-561.

[25]

{25} C.E. Shannon, A mathematical theory of communication, Bell System Technical Journal 27 (1948) 379-432.

[26]

{26} E.S. Soofi, Principal information theoretic approaches, Journal of the American Statistical Association 95 (2000) 1349-1353.

[27]

{27} I. Vajda, Theory of Statistical Inference and Information, Kluwer Academic Publishers, 1989.

[28]

{28} K. Zografos, On maximum entropy characterization of Pearson's type II and VII multivariate distributions, Journal of Multivariate Analysis 71 (1999) 67-75.

Digital Library

Cited By

Nascimento AFerreira JFrery A(2023)Unsupervised segmentation of PolSAR data with complex Wishart and distributions and Shannon entropyStatistics and Computing10.1007/s11222-023-10298-033:6Online publication date: 3-Oct-2023
https://dl.acm.org/doi/10.1007/s11222-023-10298-0
Soofi EZhao HNazareth D(2010)Information measuresWIREs Computational Statistics10.1002/wics.622:1(75-86)Online publication date: 19-Jan-2010
https://dl.acm.org/doi/10.1002/wics.62
De Gregorio AIacus S(2009)On Rényi information for ergodic diffusion processesInformation Sciences: an International Journal10.1016/j.ins.2008.09.016179:3(279-291)Online publication date: 15-Jan-2009
https://dl.acm.org/doi/10.1016/j.ins.2008.09.016
Show More Cited By

Index Terms

Expressions for Rényi and Shannon entropies for bivariate distributions
1. Mathematics of computing
  1. Information theory
  2. Probability and statistics
    1. Distribution functions
    2. Multivariate statistics

Recommendations

On Mardia's and Song's measures of kurtosis in elliptical distributions

The main objective of this paper is the calculation and the comparative study of two general measures of multivariate kurtosis, namely Mardia's measure @b"2","p and Song's measure S(f). In this context, general formulas for the said measures are derived ...
The Meta-elliptical Distributions with Given Marginals

Based on an analysis of copulas of elliptically contoured distributions, joint densities of continuous variables with given strictly increasing marginal distributions are constructed. A method utilized for this procedure is to embed the spherical ...
Noncentral elliptical configuration density

The noncentral configuration density, derived under an elliptical model, generalizes and corrects the Gaussian configuration and some Pearson results. Partition theory is then used to obtain explicit configuration densities associated with matrix ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Information Sciences: an International Journal

Information Sciences: an International Journal Volume 170, Issue 2-4

25 February 2005

321 pages

ISSN:0020-0255

Issue’s Table of Contents

Publisher

Elsevier Science Inc.

United States

Publication History

Published: 25 February 2005

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 09 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Nascimento AFerreira JFrery A(2023)Unsupervised segmentation of PolSAR data with complex Wishart and distributions and Shannon entropyStatistics and Computing10.1007/s11222-023-10298-033:6Online publication date: 3-Oct-2023
https://dl.acm.org/doi/10.1007/s11222-023-10298-0
Soofi EZhao HNazareth D(2010)Information measuresWIREs Computational Statistics10.1002/wics.622:1(75-86)Online publication date: 19-Jan-2010
https://dl.acm.org/doi/10.1002/wics.62
De Gregorio AIacus S(2009)On Rényi information for ergodic diffusion processesInformation Sciences: an International Journal10.1016/j.ins.2008.09.016179:3(279-291)Online publication date: 15-Jan-2009
https://dl.acm.org/doi/10.1016/j.ins.2008.09.016
Ebrahimi NSoofi ESoyer R(2008)Multivariate maximum entropy identification, transformation, and dependenceJournal of Multivariate Analysis10.1016/j.jmva.2007.08.00499:6(1217-1231)Online publication date: 1-Jul-2008
https://dl.acm.org/doi/10.1016/j.jmva.2007.08.004
Hung WYang M(2008)On the J-divergence of intuitionistic fuzzy sets with its application to pattern recognitionInformation Sciences: an International Journal10.1016/j.ins.2007.11.006178:6(1641-1650)Online publication date: 20-Mar-2008
https://dl.acm.org/doi/10.1016/j.ins.2007.11.006
Dukkipati ABhatnagar SNarasimha Murty M(2007)Gelfand-Yaglom-Perez theorem for generalized relative entropy functionalsInformation Sciences: an International Journal10.1016/j.ins.2007.07.017177:24(5707-5714)Online publication date: 20-Dec-2007
https://dl.acm.org/doi/10.1016/j.ins.2007.07.017

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents