Classification of Binary Vectors by Stochastic Complexity,
Stochastic complexity is treated as a tool of classification, i.e., of inferring the number of classes, the class descriptions, and the class memberships for a given data set of binary vectors. The stochastic complexity is evaluated with respect to the family of statistical models defined by finite mixtures of multivariate Bernoulli distributions obtained by the principle of maximum entropy. It is shown that stochastic complexity is asymptotically related to the classification maximum likelihood estimate. The formulae for stochastic complexity have an interpretation as minimum code lengths for certain universal source codes for storing the binary data vectors and their assignments into the classes in a classification. There is also a decomposition of the classification uncertainty in a sum of an intraclass uncertainty, an interclass uncertainty, and a special parsimony term. It is shown that minimizing the stochastic complexity amounts to maximizing the information content of the classification. An algorithm of alternating minimization of stochastic complexity is given. We discuss the relation of the method to the AUTOCLASS system of Bayesian classification. The application of classification by stochastic complexity to an extensive data base of strains ofEnterobacteriaceaeis described.
Year of publication: |
1997
|
---|---|
Authors: | Gyllenberg, Mats ; Koski, Timo ; Verlaan, Martin |
Published in: |
Journal of Multivariate Analysis. - Elsevier, ISSN 0047-259X. - Vol. 63.1997, 1, p. 47-72
|
Publisher: |
Elsevier |
Keywords: | bacterial identification classification maximum likelihood estimate information content maximal predictive classification mixture of multivariate Bernouli distributions principle of maximum entropy universal source codes AUTOCLASS |
Saved in:
Online Resource
Saved in favorites
Similar items by person
-
Corander, Jukka, (2009)
-
Numerical taxonomy and the principle of maximum entropy
Gyllenberg, Mats, (1996)
-
Cramer-Lundberg approximation for nonlinearly perturbed risk processes
Gyllenberg, Mats, (2000)
- More ...