HOME    »    PROGRAMS/ACTIVITIES    »    Annual Thematic Program
IMA Thematic Year on MATHEMATICS IN MULTIMEDIA
September 2000 - June 2001

   Long Term Visitors   Complete List of Participants 

Logo for the IMA September 2000 - June 2001 Program on Mathematics in Multimedia

Questions? Contact us at staff@ima.umn.edu.

IMA Thematic Year on MATHEMATICS IN MULTIMEDIA September 2000 - June 2001

IMA Thematic Year on MATHEMATICS IN MULTIMEDIA September 2000 - June 2001

The year is divided into three components:

     
Fall Quarter, September-December, 2000: Vision, Speech and Language
Winter Quarter, January-March, 2001: Digital Libraries
Spring Quarter, April-June, 2001: Geometric Design and Computer Graphics

Organizing Committee:
Name Present Institution
Michael Barnsley
mbarnsley@aol.com 		Georgia Institute of Technology (Mathematics and CDSNS)
Rosemary Chang
rchang@coastcom.com		 Coastcom
Tony Derose
derose@pixar.com 		Pixar
Stuart Geman
Stuart_Geman@Brown.edu 		Brown University (Applied Mathematics)
Peter Olver
olver@math.umn.edu 		University of Minnesota (Mathematics)
Roni Rosenfeld
roni@CMU.EDU 		Carnegie-Mellon University (Computer Science)
Larry Schumaker
s@mars.cas.vanderbilt.edu 		Vanderbilt University (Mathematics)
Ahmed Tewfik
tewfik@ee.umn.edu 		University of Minnesota (Electrical Engineering)

 

    The past few years have seen an explosion in the use of digital media. Industry is making significant investments to deliver digital audio, image and video information to consumers and customers. A new infrastructure of digital audio, image and video recorders and players, on-line services and electronic commerce is rapidly being deployed. At the same time, major corporations are converting their audio, image and video archives to an electronic form. Individual consumers may also soon be able to convert their image and video archives to digital form.

    This burgeoning of digital information coupled with advances in computing, interface and communication technologies has paved the way for multimedia. The most distinctive characteristic of a multimedia product is that it carries and delivers digital information in mixed modes on a single platform. What makes multimedia different from traditional information products is a much richer variety of underlying works. A multimedia product may contain graphics, film, video, music, photographs, paintings, animation, text, data, maps, games, and multimedia software.

    Current and future challenges in multimedia technologies include (i) better understanding of the interaction between different media, (ii) two-way man-machine interaction in speech recognition and computer vision, (iii) improved quality of computer generated media, (iv) developing communication protocols that protect data privacy, and (v) restructuring existing data bases to respond to real-time performance demands.

    Mathematics, with its reliance on exposing and exploiting the hidden patterns and structures in physical phenomena, will play a key role in uniting and synthesizing the different modalities inherent in the ongoing multimedia revolution. The mathematical disciplines that are required cover a broad spectrum, ranging from pure algebra and group theory, through geometry and topology, and, naturally, analysis - both analytical and numerical - with probabilistic and stochastic methods playing a particularly important role. The interface between mathematics and multimedia applications forms a two-way street - not only do existing mathematical theories acquire new and unexpected applications, but the multimedia applications themselves point to new problems requiring solutions, which in turn will stimulate new developments in mathematics itself. Thus, we can view multimedia in the role of a twenty-first century reincarnation of the old mathematical paradigm that inseparably synergizes research in pure mathematics and its applications.

    One of the hallmarks of the subject is the incredible variety of mathematical tools and theories that arise. Although perhaps daunting to the novice, this diversity enhances the range of possible interactions and cross fertilizations among the different mathematical disciplines, as each is called into play in quantifying, analyzing and developing practical algorithms. A key goal of the multimedia year at the IMA is to foster the interaction between researchers using similar tools in different multimedia modalities, and, on the other hand, providing researchers utilizing different mathematical techniques to study the same modality to compare results and combine promising methods. The year will play an important role in exposing the mathematical community to a new range of challenging and timely mathematical problems and applications. Particular attention will be paid to the training of postdoctoral researchers to be familiar with a wide variety of mathematical tools and techniques that will hopefully lay the foundations for a genuinely mathematical discipline that will become known as "multimedia."

    The underlying mathematical theories can be broadly divided into several important, overlapping categories. First, since vision, language, music, video and other sensory processes are much less deterministic than ordinary physical phenomena, a large class of methods rely on probabilistic and stochastic paradigms. For example, Markov random fields form the basis of much current research in speech and feature recognition, in visual tracking, in segmentation, and in signal processing and storage. Statistical estimation and Bayesian methods feature in pattern recognition, in language modeling, in optical character recognition, as well as signal image reconstruction. Monte Carlo methods have been successfully used in the numerical analysis of some processes, as well as in computer design and graphics.

    Wavelets and other transform methods appear in a wide variety of contexts, including image and signal compression and enhancement, computer graphics, texture analysis, and the recovery of degraded audio signals. A particularly striking example is the FBI's adoption of a wavelet-based system for compression of fingerprint images. (On the other hand, image enhancement techniques based on nonlinear differential equations have also been dramatically successfully in forensics and video enhancement.) Wavelet methods are particularly useful in the as yet poorly understood theory of image textures, but successfully incorporating them into the variational and partial differential equation approaches to segmentation has yet to be completed.

    Since digital imagery is a fundamentally discrete data system, a particular surprise is the relevance of both nonlinear partial differential equations and variational methods in low level image and video processing. The mathematical techniques include the various recent approaches to curve shortening and mean curvature flows (originally studied by differential geometers) and the variational approaches of the type used in minimal surfaces. These methods are used in denoising images and in edge detecion of features, and have seen success in visual tracking and recognition of objects, medical imaging, video enhancement, and collision avoidance. Implementation of the relevant nonlinear diffusion equations has led to the development of fast, sophisticated numerical algorithms that have broad applicability. The passage from discrete pixel-based image to continuous partial differential equation and back to discrete finite difference numerical implementation sounds paradoxical, until one realizes that the fundamental processes in continuum mechanics and fluid dynamics work in precisely the same paradigm - the discrete molecular system is modeled by a continuous system, e.g. the Navier-Stokes equations, which are themselves integrated numerically via a discrete approximation.

    The nonlinear partial differential equation approach to image processing relies on classical differential geometry. Modern fractal geometry, which arose in the study of natural phenomena, also has immediate applications in computer graphics, animation and the construction of artificial scenes. Fractal-based algorithms has had dramatic results in practical image compression, while very recent wavelet-fractal hybrids indicate the potentialities for cross disciplinary approaches to multimedia processes.

    Algebraic theories arise in cryptography and information theoretic approaches to signal processing. Secure computing, copyright protection, and language grammars also rely on fundamental algebraic methods for implementation.

    In computer graphics and geometric design, the digital representation of fully three-dimensional objects has been the subject of intense research activity in the past decade, leading to powerful new algorithms. Algebraic geometry, splines, wavelets, optics and solid geometry are but a few of the mathematical tools brought to bear on this vital area. Applications include computer graphics, surface and solid rendering, animation, flight simulation, virtual reality, and scientific and medical imagery. But real-time display of fully three-dimensional images remains problematic, necessitating further research in the fundamental mathematical issues. Inevitable technological advances in graphics performance will continuously change the requirements for real-time rendering and the complexity of models to be visualized.

    Group theory is recognized to play an increasingly important role in media analysis; for instance, both human and camera visual systems naturally incorporate certain symmetries, including translations, rotations, scalings, and then progressing on to the less familiar groups of affine and projective transformations. Galilean invariance plays an important role in movie restoration and recognition of moving objects. In general, the reliable recognition of objects must form an integral part of any functional multimedia system. These include recognition of visual objects, target recognition, satellite images, speech and language recognition, optical character recognition, and language. Most of these are still in their infancy, with the underlying mathematical theory still in need of development. Applying symmetry groups to recognition problems in a natural fashion requires the adaptation of classical geometric theories of differential invariants and algebraic invariants, subjects that formed the core of pure mathematical research in the last century. Recent work on symmetry-incorporating numerical approximations hold significant promise for computation of invariants and object recognition in physical images.

    Use of multimedia in manufacturing holds great potential for industrial applications. Visual control of the robots used in industrial processes, such as semiconductor manufacturing and etching, requires efficient, real-time processing of images, incorporating the denoising, enhancement, segmentation, and object recognition techniques from a computer vision system into a broader control-theoretic loop. Some striking experiments, including an image-based vehicular navigational system, show significant promise. Fast and efficient numerical implementation of the analytical algorithms provides the key to real-time applications and automatic control. Both finite difference and finite element methods have been successfully employed. For example, the use of level set methods for front tracking and interface evolution in phase transitions requires fast marching algorithms and efficient numerical implementations. However, to date the marriage of computer vision and control theory has yet to be properly consummated, requiring a new synthesis of the underlying mathematical theories.

    In summary, while much progress can be seen on individual aspects, their synthesis into a mathematical theory of multimedia is as yet unexplored. This year at the I.M.A. will provide a unique and unprecedented opportunity to bring together researchers in a wide variety of mathematical disciplines and applications. Forging these disparate subjects into a new and vital subject will have long range effects, both within mathematics, and in the practical applications to multimedia science.

 

Go to:
 Fall Quarter, September-December, 2000:
Vision, Speech and Language

Winter Quarter, January-March, 2001:
Digital Libraries

Spring Quarter, April-June, 2001:
Geometric Design and Computer Graphics

Fall Quarter (September - December, 2000)

Vision, Speech and Language

  • Workshop 6B: Digital Libraries - Classification, Retrieval and Visualization, February 26-March 2, 2001

  • Special Workshop: Frontiers of Mathematics in Geosciences , March 5-7, 2001


    Back to top of page

    Spring Quarter (April - June, 2001)

    • Geometric Design and Computer Graphics

  • Joint IDR-IMA Workshop: Ideal Data Representation, April 9-13, 2001


    • Introduction to Workshops 7 and 8 on Mathematical Methods in Geometric Design and Computer Graphics:

    Most multimedia content today consists primarily of text, 2D images, video, and audio. Multimedia of the near future will additionally rely heavily on 3D graphics. The creation and use of 3D graphics in turn draws upon the two closely related disciplines of geometric design (the creation of the geometric objects), and computer graphics (the animation and display of the objects). These disciplines have been extraordinarily vibrant and diverse, with current applications including flight simulation, medical imaging, scientific visualization, computer-aided design, and entertainment. Results obtained in research laboratories over the past twenty years have given rise to a multi-billion dollar industry. The pending 3D graphics revolution is due in large part to impressive improvements in processor speeds and memory capacities. More impressive still are numerous algorithmic improvements that have opened up new application areas and have provided remarkable asymptotic reductions in both running time and memory requirements. Many of the algorithmic advances are based on mathematical methods such as wavelets, finite element analysis, Monte Carlo methods, and algebraic geometry. There is every reason to suspect that this trend can continue, so it is essential to stimulate collaboration between the mathematics and graphics communities. The goal of this pair of workshops is to do just that.

  • Tutorial: Geometric Design, April 19-20, 2001

 

SUMMER 2001 PROGRAMS

2001 Summer Program: Geometric Methods in Inverse Problems and PDE Control , July 16-27, 2001

IMA Special Workshop:Mathematical Opportunities in Large-Scale Network Dynamics, August 6-7, 2001

"Hot Topics" Workshop: Wireless Networks, August 8-10 2001

Back to top of page

Go