Gauss Mixture Vector Quatization for Compression, Classification, and Modeling
Monday, January 29, 2001 - 11:00am - 12:00pm
Robert Gray (Stanford University)
The worst case attribute of Gaussian vectors for data compression/source coding originally developed by Sakrison and Lapidoth using Shannon rate-distortion theory is developed using the high rate quantization theory of Bennett, Zador, and Gersho and extended to Gauss mixtures, providing an approach to robust data compression for nonGaussian sources such as images. The analysis provides several interesting side results, including a new interpretation of the minimum discrimination information distortion (MDI) measure and its application to clustering models and constructing Gauss mixture models based on training data. High rate quantization theory provides a mathematical connection between the distortion and the performance of a classified vector quantizer for nonGaussian data designed using Gaussian distributions. Although the primary application is compression and classification, several ideas relating maximum entropy estimation of probability densities, the MAXDET problem, and Markov mesh random fields arise in the analysis. The theory provides a hindsight explanation for why CELP speech coders work as well as they do.