with partial support by The Office of Naval Research

Organizers:

Robert Gray
Department of Electrical Engineering
Stanford University
gray@ee.stanford.edu

James Johnston
AT&T Labs Research
jj@research.att.com

Michael T. Orchard
Rice University
(on leave from Department of Electrical Engineering
Princeton University)
orchard@princeton.edu

Shashi Shekhar
Department of Computer Science
University of Minnesota
shekhar@cs.umn.edu

Data modeling and representation is a critical, and possibly the most difficult, step in constructing any kind of datastore, including digital libraries, datawarehouses and databases. Errors in data models may lead to irrelevant or incorrect answers to user queries, as well as poor reliability and performance. The objective of data modeling is to complete a rigorous design with quality checks before the physical implementation of the digital library. A sound approach to data modeling and representation of multimedia content in digital libraries is via the design of relevant Abstract Data Types, e.g. audio, video, images, spatial, temporal, etc., which can be integrated into popular modeling languages using their extensible type systems. The goal of the data models and representations of multimedia data is to support efficient storage (e.g. compression), retrieval (e.g. querying), transmission and rendering (display).

This workshop will survey and compare relevant models, and algorithms for multimedia storage, retrieval, transmission, and rendering. It will develop tools for their evaluation, and study their applications from a variety of viewpoints for a plethora of data types. Methods for quantifying the quality of such models, especially perceived quality of image and audio data, will also be a focus. Participants are encouraged from the fields of signal processing, statistics, computer science, and information theory interested in the theory, algorithms, and applications of mathematical modeling and representing data. A goal of the workshop is to develop a better and broader understanding of the similarities and differences and the relative merits of the best existing approaches to mathematical model fitting and its application to real world problems.

Data models for multimedia data should provide mechanisms to represent the semantics of Quality of Service (QoS) which specifies user requirements for the overall digital library system spanning components of storage, retrieval, transmission, and rendering. Measures of QoS include end-to-end parameters for synchronization (e.g. skew between audio and video streams), human perception (e.g. subjective image or sound quality), and system performance (e.g. delay, bit-rate, video resolution, frame rate). Desirable QoS on the overall digital library system may be translated into the QoS contraints on components responsible for storage, retrieval, transmission and rendering. Data models should support specification of temporal synchronization and time dependence of multi-media to help in selection of algorithms which can meet the QoS contraints using effective resource management, buffering (e.g. prefetching), admission control, and various optimization such as scheduling. Consider the choice between download vs. streaming in context of internet based digital libraries. Downloads promote a a sequential schedule where rendering starts only after transmission and retrieval are complete. With multimedia data (e.g. audio, video ), this result in very long initial wait for end-users. In addition, it may overwhelm the local resources of client computers. Streaming on the other hand uses a pipeline schedule rendering the results as soon as first few packets have arrived reducing initial wait and load on local resources. However, streaming may perform poorly on other QoS measure particularly when transmission channel is overloaded.

Compression of multimedia data is a common issue across storage, transmission and rendering. Mathematical models for compressing multimedia data can be perceptual or statistical. Perceptual coding models take advantage of the characteristics of of the ultimate human receiver, to reduce "irrelevant" information, not detectable by the human observer, as opposed to mathematically "redundant" information. These perceptual measures are, as yet, not reliable for judging the performance of algorithms, but are good enough to provide a very high level of lossy (in the LMS sense) compression while retaining a good to indetectable compressed quality. Statistical modeling or fitting probability distributions to data plays a critical role in a variety of multimedia synthesis, processing and analysis techniques. Traditional Gaussian models provide tractability for analysis , in some cases providing a "worst case'' when second order moments are known. However, the non-Gaussian nature of images and speech has led to mixture or composite models, which may have Gaussian components. Important topics include the construction of models from data using classical statistical tools as well as more recent approaches using the EM algorithm, decision trees, neural nets, flexible or penalized discrimination, Gauss Markov discrimination, and minimum discrimination information methods (based on relative entropy) along with the use of these models in specific signal processing applications, including empirical Bayes methods for regression/estimation and classification/detection/segmentation, compression, analysis, and enhancement.

Topics:

1a) Data Models (Statistical) : Density estimation and inference

1b) Data Models (Perceptual) : Modeling human perception

2) Storage: Compression, Clustering, Indexing, Multimedia Database Structures

3) Transmission/Communication: Compression, Quality of Service