Estimating Functions of Probability Distributions From A Finite Set of Samples

Part I: Bayes Estimators and the Shannon Entropy. <p> This paper is the first of two on the problem of estimation a function of a probability distribution from a finite set of samples of that distribution. In this paper a Bayerian analysis of this problem is presented, the optimal properties of the Bayes estimators are discussed, and as an example of the formalism, closed form expressions for the Bayes estimators for the moments of the Shannon entropy function are derived. Numerical results are presented that compare the Bayes estimator to the frequency-counts estimator for the Shannon entropy.