HOME    »    PROGRAMS/ACTIVITIES    »    Annual Thematic Program
IMA Workshop 5
Digital Libraries: Data Modeling and Representation
January 29 - February 2, 2001

with partial support by The Office of Naval Research


Robert Gray
Department of Electrical Engineering
Stanford University

James Johnston
AT&T Labs Research

Michael T. Orchard
Rice University
(on leave from Department of Electrical Engineering
Princeton University)

Shashi Shekhar
Department of Computer Science
University of

Data modeling and representation is a critical, and possibly the most difficult, step in constructing any kind of datastore, including digital libraries, datawarehouses and databases. Errors in data models may lead to irrelevant or incorrect answers to user queries, as well as poor reliability and performance. The objective of data modeling is to complete a rigorous design with quality checks before the physical implementation of the digital library. A sound approach to data modeling and representation of multimedia content in digital libraries is via the design of relevant Abstract Data Types, e.g. audio, video, images, spatial, temporal, etc., which can be integrated into popular modeling languages using their extensible type systems. The goal of the data models and representations of multimedia data is to support efficient storage (e.g. compression), retrieval (e.g. querying), transmission and rendering (display).

This workshop will survey and compare relevant models, and algorithms for multimedia storage, retrieval, transmission, and rendering. It will develop tools for their evaluation, and study their applications from a variety of viewpoints for a plethora of data types. Methods for quantifying the quality of such models, especially perceived quality of image and audio data, will also be a focus. Participants are encouraged from the fields of signal processing, statistics, computer science, and information theory interested in the theory, algorithms, and applications of mathematical modeling and representing data. A goal of the workshop is to develop a better and broader understanding of the similarities and differences and the relative merits of the best existing approaches to mathematical model fitting and its application to real world problems.

Data models for multimedia data should provide mechanisms to represent the semantics of Quality of Service (QoS) which specifies user requirements for the overall digital library system spanning components of storage, retrieval, transmission, and rendering. Measures of QoS include end-to-end parameters for synchronization (e.g. skew between audio and video streams), human perception (e.g. subjective image or sound quality), and system performance (e.g. delay, bit-rate, video resolution, frame rate). Desirable QoS on the overall digital library system may be translated into the QoS contraints on components responsible for storage, retrieval, transmission and rendering. Data models should support specification of temporal synchronization and time dependence of multi-media to help in selection of algorithms which can meet the QoS contraints using effective resource management, buffering (e.g. prefetching), admission control, and various optimization such as scheduling. Consider the choice between download vs. streaming in context of internet based digital libraries. Downloads promote a a sequential schedule where rendering starts only after transmission and retrieval are complete. With multimedia data (e.g. audio, video ), this result in very long initial wait for end-users. In addition, it may overwhelm the local resources of client computers. Streaming on the other hand uses a pipeline schedule rendering the results as soon as first few packets have arrived reducing initial wait and load on local resources. However, streaming may perform poorly on other QoS measure particularly when transmission channel is overloaded.

Compression of multimedia data is a common issue across storage, transmission and rendering. Mathematical models for compressing multimedia data can be perceptual or statistical. Perceptual coding models take advantage of the characteristics of of the ultimate human receiver, to reduce "irrelevant" information, not detectable by the human observer, as opposed to mathematically "redundant" information. These perceptual measures are, as yet, not reliable for judging the performance of algorithms, but are good enough to provide a very high level of lossy (in the LMS sense) compression while retaining a good to indetectable compressed quality. Statistical modeling or fitting probability distributions to data plays a critical role in a variety of multimedia synthesis, processing and analysis techniques. Traditional Gaussian models provide tractability for analysis , in some cases providing a "worst case'' when second order moments are known. However, the non-Gaussian nature of images and speech has led to mixture or composite models, which may have Gaussian components. Important topics include the construction of models from data using classical statistical tools as well as more recent approaches using the EM algorithm, decision trees, neural nets, flexible or penalized discrimination, Gauss Markov discrimination, and minimum discrimination information methods (based on relative entropy) along with the use of these models in specific signal processing applications, including empirical Bayes methods for regression/estimation and classification/detection/segmentation, compression, analysis, and enhancement.


1a) Data Models (Statistical) : Density estimation and inference

1b) Data Models (Perceptual) : Modeling human perception

2) Storage: Compression, Clustering, Indexing, Multimedia Database Structures

3) Transmission/Communication: Compression, Quality of Service

4) Rendering: Audio synthesis and image graphics


Monday Tuesday
All talks are in Lind Hall 400 unless otherwise noted.
8:30 am Coffee and Registration


9:10 am Willard Miller, Fred Dulles, and Michael T. Orchard Introduction
9:30 am Jont Allen
AT&T Labs-Research

The Intensity JND Comes from Poisson Neural Noise: Implications for Image Coding

Complete Paper   pdf   postscript.gz

10:30 am Break  
11:00 am-12:00 pm Robert Gray
Stanford University
Gauss Mixture Vector Quatization for Compression, Classification, and Modeling
2:00 pm Michelle Effros
California Institute of Technology
Network Source Codes in Digital Libraries
3:00 pm Break  
3:30 pm Nuno Vasconcelos
Massachusetts Institute of Technology

A Decision-theoretic View of Image Retrieval

Slides   html    pdf

4:30 pm IMA Tea
A variety of appetizers and beverages will be served.
IMA East, 400 Lind Hall
All talks are in Lind Hall 400 unless otherwise noted.
9:15 am Coffee  
9:30 am Arif Ghafoor
Purdue University
Multimedia Database Management Systems: A Perspective
10:30 am Break  
11:00 am- 12:00 pm Clement Yu
University of Illinois at Chicago
Retrieval of Images of People from the Web
1:30 pm Hanan Samet
University of Maryland - College Park
Spatial Databases and Geographic Information Systems

Spatial Index Demos

2:30 pm Break  
3:30-4:30 pm Shashi Shekhar
University of Minnesota

An Overview of Spatial Databases for Digital Library

Related paper (ps, 212KB)

All talks are in Lind Hall 400 unless otherwise noted.
9:15 am Coffee  
9:30 am Shi-Kuo Chang
University of Pittsburgh
Sentient Map - A Novel Interface for Digital Libraries


10:30 am Break  
11:00 am-12:00 pm Sheila S. Hemami
Cornell University

Perception of Extremely Low-Rate Images and Video: Psychophysical Evaluations and Analyses

Talk   pdf

2:00 pm Alfred Hero
University of Michigan

Divergence Matching Criteria for Registration, Indexing and Retrieval

All talks are in Lind Hall 400 unless otherwise noted.
9:15 am Coffee  
9:30 am Don H. Johnson
Rice University

A Theory of Information Processing

Talk  pdf

10:30 am Break  
11:00 am-12:00 pm James D. Johnston
AT&T Labs Research

Perceptual Coding - Interactions Between Models and Coders


2:00 pm Amy R. Reibman
AT&T Bell Labs

Source Coding Alternatives for Video Transport Over Networks

Talk   pdf    html

3:00 pm Break  
3:30-4:30 pm Nicholas Coult
Augsburg College
Compression and Region-of-interest Extraction for Large Incomplete Data Sets
All talks are in Lind Hall 400 unless otherwise noted.
8:15 am Coffee  
8:30 am Michael T. Orchard
Rice University (on leave from Princeton University)

On Modelling Location Uncertainty in Images: A Coding Perspective


9:30 am Break  
9:45 am Kannan Ramchandran
University of California-Berkeley
Distributed Coding and Robust Transmision of Multimedia Data
10:45 am Break  
11:00 am Sharad Mehrotra
University of California at Irvine
Clustering and Indexing of Multimedia Objects in the MARS System
12:45 pm Julius Smith
Stanford University
Musical Signal Models for Audio Rendering
Monday Tuesday


(in addition to postdocs and long term participants)

as of 1/29/2001
Name Department Affiliation
Jont Allen   AT&T Labs-Research
Sankar Basu   IBM T.J. Watson Research Center
John Baxter Mathematics University of Minnesota
Shi-Kuo Chang Computer Science University of Pittsburgh
Christine Cheng Mathematical Sciences Johns Hopkins University
Nicholas Coult Mathematics Augsburg College
Fred Dulles   Institute for Mathematics & its Applications
Michelle Effros Electrical Engineering, 136-93 California Institute of Technology
Paul Garrett Mathematics University of Minnesota
Arif Ghafoor Elec. & Comp. Eng. Purdue University
Robert Gray Electrical Engineering Stanford University
Bin Han   Institute for Mathematics and its Applications
Sheila Hemami Electrical Engineering Cornell University
Alfred Hero EECS University of Michigan
Don Johnson Chair, ECE Rice University
James Johnston   AT&T Labs Research
Sharad Mehrotra Computer Science University of California at Irvine
Phil Mendelsohn   University of Minnesota
Willard Miller   Institute for Mathematics & its Applications
Michael Orchard Electrical Engineering Rice University
Kannan Ramchandran Elec. Eng. & Comp. Sci. University of California - Berkeley
Amy Reibmen   AT&T Labs Research
Hanan Samet Comp. Sci University of Maryland - College Park
Shashi Shekhar Computer Science University of Minnesota
Julius Smith CCRMA Stanford University
Stephen Strother VA Medical Center, PET Imaging Service University of Minnesota
Allen Tannenbaum Electrical & Computer Engineering Georgia Institute of Technology
Nuno Vasconcelos Compaq Computer Corporation Cambridge Research Laboratory
Clement Yu Electrical Engineering & Computer Science University of Illinois at Chicago
Zhi-Li Zhang Comp. Sci/Eng. University of Minnesota

2000-2001 Program: Mathematics in Multimedia

Back to top of page

Connect With Us: