Abstract—Conditional entropy measures information flow from one discrete random to another. Conventional entropy is calculated from conditional or joint probabilities of two variables. When the joint probability is not known, the conditional entropy cannot be calculated. This paper presents an approach to estimate the conditional entropy using indirect conditional information. The result of the estimation is overestimated by less than ten percent of the actual value for normal cases.

Index Terms—Entropy, conditional entropy, information flow, indirect conditional information.

Nol Premasathian is with the Faculty of Information Technology, King Mongkut’s Institute of Technology Ladkrabang, Bangkok, Thailand (e-mail:nol@it.kmitl.ac.th).
Watcharee Tantikittipisut is with the Bangkok, Thailand (e-mail:charlieante@gmail.com).

Cite: Nol Premasathian and Watcharee Tantikittipiisut, "Information Entropy Estimation Using Indirect Conditional Information," International Journal of Computer and Communication Engineering vol. 2, no. 2, pp. 210-212 , 2013.