Web: http://www.ima.umn.edu | Email: ima-staff@ima.umn.edu | Telephone: (612) 624-6066 | Fax: (612) 626-7370
Additional newsletters available at http://www.ima.umn.edu/newsletters

IMA Newsletter #352

February 2006

2005-2006 Program

Imaging

See http://www.ima.umn.edu/2005-2006 for a full description of the 2005-2006 program on Imaging.

IMA Events

IMA Annual Program Year Workshop

The Mathematics and Art of Film Editing and Restoration

February 6-10, 2006

Organizers: Guillermo R. Sapiro (University of Minnesota), Lance J. Williams (Applied Minds), Andrew Zisserman (Oxford University)

We enjoy movies on a daily basis, but are rarely aware of the image processing involved in creating and editing a film. Examples of frequent operations are format conversion, compression, scratch and blotch removal, introducing special effects, motion estimation, camera calibration, segmentation and super-resolution. These techniques are so important that, today, virtually every movie undergoes some digital image processing. Still, some of the fundamental problems remain unsolved, often involving very time-consuming manual labor. The quality standards are also very different—often far stricter—than those in most common applications of image processing. This brings significant mathematical and engineering challenges. The goal of this workshop is to have a uniquely interactive setting in which engineers, mathematicians, and artists implicated in film editing, restoration and “post-processing” are assembled in order to inform and instruct each other as well as discuss possible solutions and directions of research. The workshop will start with a number of tutorials by leaders in the area (from industry and academia) and will be followed by interleaved talks in the interdisciplinary aspects of movie editing and restoration.

Math Matters: IMA Public Lecture

Artful Mathematics

Daniel Rockmore

February 8, 2006

All too often we see mathematics and the arts as two sides of the science/humanities coin. In this talk we'll see a place in which the two come naturally together in exciting new research. In today's world in which almost all aspects of life are brought to the common medium of the computer, it is now possible to quantify and extract the style of an artist via computation. Examples are gleaned from the literary, visual, and dance arts, and include applications to the problem of authentication. Taken together, this work reveals just how stylish math can be.
Schedule

Friday, February 3

1:25p-2:25pThe mathematics of web information retrievalRuchira Datta (Google)Vincent Hall 20 IPS

Monday, February 6

8:15a-9:00aRegistration and coffeeEE/CS 3-176 W2.6-10.06
9:00a-9:10aWelcome to the IMADouglas N. Arnold (University of Minnesota)EE/CS 3-180 W2.6-10.06
9:10a-9:20aOpening RemarksGuillermo R. Sapiro (University of Minnesota)
Lance J. Williams (Applied Minds)
Andrew Zisserman (Oxford University)
EE/CS 3-180 W2.6-10.06
9:20a-10:20aThe grammar of storytelling on film survives technologyJay Cassidy (Mathematical Technologies, Inc.)EE/CS 3-180 W2.6-10.06
10:20a-10:45acoffeeEE/CS 3-176 W2.6-10.06
10:45a-11:45aPushing pixels: From restoration to post-production Anil Kokaram (Trinity College)EE/CS 3-180 W2.6-10.06
11:45a-1:30plunch W2.6-10.06
1:30p-2:30pThe evolution of film editing technique and its implications to the parsing and summarization of motion picturesJohn W. Mateer (University of York)EE/CS 3-180 W2.6-10.06
2:30p-3:00pSecond ChancesEE/CS 3-180 W2.6-10.06
3:10p-3:15pgroup photo W2.6-10.06
3:15p-4:30pPoster Session/ReceptionLind Hall 400 W2.6-10.06
Variational approach to interpolate and correct adhesion bias in low-baseline stereo correlationAndres Almansa (Facultad de Ingeniería)
Cross-generalization: learning novel classes from a single example by feature replacementEvgeniy Bart (University of Minnesota)
Digital restoration of antique documentsSebastiano Battiato (University of Catania)
Novel image processing algorithms for European Union Digital Cinema projectsMarcelo Jose Bertalmio (Universitat Pompeu Fabra)
Scratch detectionBernard Besserer (Universite de La Rochelle)
Getting rid of scratches and blotchesHarald Grossauer (University of Innsbruck)
Restoration and zoom of irregularly sampled, blurred and noisy images by accurate total variation minimization with local constraintsGloria Haro Ortega (University of Minnesota)
PrestoSpace projectLaurent Joyeux (Joanneum Research)
A variational approach to blending based on warping for non-overlapped imagesChang-Ock Lee (KAIST)
New approaches to line scratch restorationLucia Maddalena (National Research Council (CNR))
Video content analysisAlvaro Pardo (Instituto de Ingenieria Electrica)
A human color perception model for film unsupervised digital restorationAlessandro Rizzi (University of Milan)

Tuesday, February 7

8:45a-9:00acoffeeEE/CS 3-176 W2.6-10.06
9:00a-10:00a3D models from image sequencesAndrew Fitzgibbon (Oxford University)EE/CS 3-180 W2.6-10.06
10:00a-10:30acoffeeEE/CS 3-176 W2.6-10.06
10:30a-11:30aComputer vision success stories in visual media production : Camera tracking and motion captureDavid Capel (2d3 Limited)EE/CS 3-180 W2.6-10.06
11:30a-1:30plunch W2.6-10.06
1:30p-2:30pVideo-based renderingRichard Szeliski (Microsoft Research)EE/CS 3-180 W2.6-10.06
2:30p-3:00pSecond ChancesEE/CS 3-180 W2.6-10.06

Wednesday, February 8

8:45a-9:00acoffeeEE/CS 3-176 W2.6-10.06
9:00a-10:00aImage and video inpaintingGuillermo R. Sapiro (University of Minnesota)EE/CS 3-180 W2.6-10.06
10:00a-10:30acoffeeEE/CS 3-176 W2.6-10.06
10:30a-11:30a"But you said it was 99% accurate! Why is there a defect in the first frame?”
Gentle techniques for using mathematics in motion picture processes to marry film and digital
Daniel Rosen (Warner Bros.)EE/CS 3-180 W2.6-10.06
11:30a-1:30plunch W2.6-10.06
1:30p-2:30pTechniques and rationale for motion picture restorationKevin Manbeck (MTI Technologies)EE/CS 3-180 W2.6-10.06
2:30p-3:00pSecond ChancesEE/CS 3-180 W2.6-10.06
3:15p-4:30pPoster SessionLind Hall 400 W2.6-10.06
7:00p-8:00pMath Matters: IMA Public Lecture   Artful mathematicsDaniel Rockmore (Dartmouth College)Willey Hall 125

Thursday, February 9

8:45a-9:00acoffeeEE/CS 3-176 W2.6-10.06
9:00a-10:00aRetrieving video of people and placesAndrew Zisserman (Oxford University)EE/CS 3-180 W2.6-10.06
10:00a-10:30acoffeeEE/CS 3-176 W2.6-10.06
10:30a-11:30a Tracking planes in images: Applications in post-productionPhilip McLauchlan (Imagineer Systems Ltd)EE/CS 3-180 W2.6-10.06
11:30a-1:30plunch W2.6-10.06
12:00p-1:00pMathematics: Maker and muse for modern artDaniel Rockmore (Dartmouth College)Weisman Art Museum
1:30p-2:30pComputer vision in movie production: problems and expectationsEugene Vendrovsky (Rhythm & Hues Studios, Inc.)EE/CS 3-180 W2.6-10.06
2:30p-3:00pSecond ChancesEE/CS 3-180 W2.6-10.06
3:00p-4:00pChallenging shots: Demo your problem and discuss it with usPierre Jasmin (RE:Vision Effects, Inc.)EE/CS 3-180 W2.6-10.06
6:00p-7:30pworkshop dinnerTBA W2.6-10.06

Friday, February 10

8:45a-9:00acoffeeEE/CS 3-176 W2.6-10.06
9:00a-10:00aMovie restoration methodsToni Buades (Universitat of Illes Balears)EE/CS 3-180 W2.6-10.06
10:00a-10:30acoffeeEE/CS 3-176 W2.6-10.06
10:30a-11:30aDenoising archival films using a field-of-experts model of film grain and natural image statisticsMichael Black (Brown University)EE/CS 3-180 W2.6-10.06
11:30a-12:00pSecond ChancesEE/CS 3-180 W2.6-10.06

Monday, February 13

11:15a-12:15pTBASung Ha Kang (University of Kentucky)Lind Hall 409

Tuesday, February 14

11:15a-12:15pTBAArnd Scheel (University of Minnesota)Lind Hall 409 PS

Wednesday, February 15

2:30p-3:30pAnalysis of Space-Discrete Image Filters by Dynamical SystemsMartin Welk (University of the Saarland)Lind Hall 409 PS

Friday, February 24

1:25p-2:25pTBALeo Grady (Siemens)Vincent Hall 20 IPS

Tuesday, February 28

11:15a-12:15pTBAGloria Haro Ortega (University of Minnesota)Lind Hall 409 PS

Event Legend:

IPSIndustrial Problems Seminar
PSIMA Postdoc Seminar
W2.6-10.06The Mathematics and Art of Film Editing and Restoration
Abstracts
Andres Almansa (Facultad de Ingeniería) Variational approach to interpolate and correct adhesion bias in low-baseline stereo correlation
Abstract: Stereo correlation techniques used to automatically compute DEMs (Digital Elevation Models) by photogrammetry typically use stereo pairs with a relatively high baseline/height ratio, in order to diminish the relative importance of the adhesion phenomenon, which is a distortion of the model that appears near strong discontinuities or borders of the image. This phenomenon is directly related to the correlation process, and the magnitudes of the artifacts cannot be neglected when trying to obtain sub-pixel accuracies. Correctly modelling and correcting this bias will allow the use of stereo pairs with much lower b/h ratios, which has the great advantage of avoiding many problems due to the occluded parts in the image.

The work by Delon and Rougé characterizes this phenomenon, giving a link between measured and true disparities, and allowing detection of uncorrelatable regions (or regions providing no useful information for correlation). Since this leads to a very ill posed system of equations, many simplifying assumptions have been adopted in order to easily solve it, leading to the so called barycentric correction of the adhesion phenomenon. Even though the result is highly improved with respect to the raw correlation disparities, one still observes a slightly blurred disparity map, which is specially annoying in urban areas. In this work we propose more precise and natural assumptions to solve this system, namely to regularize the solution by a modified minimal surface term. Such an approach is naturally expected to allow less blurred edges while still filling in empty areas (without meaningful correlation information) in a reasonable manner.

Our future research will explore the extension of these techniques to motion estimation in image sequences, and its application (in conjunction with our irregular sampling work) to multi-frame superresolution.

Evgeniy Bart (University of Minnesota) Cross-generalization: learning novel classes from a single example by feature replacement
Abstract: We develop an object classification method that can learn a novel class from a single training example. In this method, experience with already learned classes is used to facilitate the learning of novel classes. Our classification scheme employs features that discriminate between class and non-class images. For a novel class, new features are derived by selecting features that proved useful for already learned classification tasks, and adapting these features to the new classification task. This adaptation is performed by replacing the features from already learned classes with similar features taken from the novel class. A single example of a novel class is sufficient to perform feature adaptation and achieve useful classification performance. Experiments demonstrate that the proposed algorithm can learn a novel class from a single training example, using 10 additional familiar classes. The performance is significantly improved compared to using no feature adaptation. The robustness of the proposed feature adaptation concept is demonstrated by similar performance gains across 107 widely varying object categories.
Sebastiano Battiato (University of Catania) Digital restoration of antique documents
Abstract: Antique documents such as photographic prints and books can be affected by several kinds of artefacts: foxing/yellowing, water blotches, fragmented glass plate, screening, etc. Each specific "problem" can be attacked by using advanced algorithms able to recover the original appearance. In this work a brief review of our solutions for virtual restoration is reported. Also some visual example will be depicted just to verify the effectiveness of the proposed approaches.
Marcelo Jose Bertalmio (Universitat Pompeu Fabra) Novel image processing algorithms for European Union Digital Cinema projects
Abstract: Starting in 2002, Universitat Pompeu Fabra (Barcelona, Spain) has been a partner in several Digital Cinema projects for the European Union, involving major European companies. Within this framework, our Image Processing Group has developed several novel algorithms for digital cinema postproduction and exhibition. These works include:
-A Day for Night algorithm that accurately models human visual perception regarding color and contrast modification, but also loss of accuity through a novel anisotropic diffusion Partial Differential Equation.
-A Depth of Field algorithm that performs real-time, accurate depth of field simulation by running an anisotropic diffusion equation on a programmable graphics card.
-A robust tracking algorithm that improves the Geodesic Active Regions formulation.
-A fast and robust segmentation algorithm based on the Tree of Shapes formulation.
-An Interlaced to Progressive Conversion algorithm that achieves real-time, state of the art results on a regular PC by implementing a variational energy minimization approach on a graphics card.
Bernard Besserer (Universite de La Rochelle) Scratch detection
Abstract: A suitable detection and tracking approach is proposed for line scratch removal in a digital film restoration process. Unlike impulsive distortions such as dirt spots, which appear randomly in an image, line scratch artifacts persist across several frames. Hence, motion compensated methods will fail for persistent line scratches. Single-frame based methods will also fail if scratches are unsteady or fragmented. The proposed method uses as input a composite image built up from projections of each image of the original sequence. First, a simple 1D-extrema detector provides line scratch candidates for both bright and dark scratches. Next, a MHT (Multiple Hypothesis Tracker) stage uses these candidates to create and keep multiple hypothesis. As the tracking goes further through the sequence, each hypothesis gains or looses evidence. To avoid a combinatorial explosion, the hypothesis tree is sequentially pruned. As hypothesis are set up at each iteration, even if no information is available, a tracked path might cross gaps (missed detection or speckled scratches). The tracking stage then feeds the correction process with valid scratch trajectories.
Michael Black (Brown University) Denoising archival films using a field-of-experts model of film grain and natural image statistics
Abstract: Bayesian denoising of archival film requires a likelihood model that captures the image noise and a spatial prior that captures the statistics of natural scenes. For the former we learn a statistical model of film noise that varies as a function of image brightness. For the latter we use the recently proposed Field-of-Experts framework to learn a generic image prior that capture the statistics of natural scenes. The approach extends traditional Markov Random Field (MRF) models by learning potential functions over extended pixel neighborhoods. Field potentials are modeled using a Products-of-Experts framework that exploits non-linear functions of many linear filter responses. In contrast to previous MRF approaches all parameters, including the linear filters themselves, are learned from training data. The prior model alone can be used to inpaint missing image structures and the data noise model can be used to simulate realistic film grain. Additionally we demonstrate how the learned likelihood and prior models can be used to denoise archival film footage. Joint work with Stefan Roth and Teodor Moldovan.
Toni Buades (Universitat of Illes Balears) Movie restoration methods
Abstract: The state of the art movie restoration methods compensate the motion by an optical flow estimate and then filter out the compensated movie. Now, the motion estimation problem is fundamentally ill-posed. This fact is known as the aperture problem: trajectories are ambiguous since they could coincide with any promenade in the space-time isophote surface. In this talk, we show that the aperture problem can be taken advantage of. This observation leads to use for movies the recently introduced NL-means algorithm. This static 3D algorithm involves the whole movie isophote and not just a trajectory.
David Capel (2d3 Limited) Computer vision success stories in visual media production : Camera tracking and motion capture
Abstract: The twin technologies of camera tracking and motion capture are key components in the modern movie production pipeline, without which such effects-laden productions as "Revenge of the Sith" and "The Lord of the Rings" simply would not be possible. Recent advances have been driven by the successful application of algorithms developed in the computer vision research community to these real-world problems. The resulting highly automated, robust software solutions have greatly reduced the time and level of specialist skill required of the operator, hence reducing the overall costs of camera tracking and motion capture. Consequently, these technologies are now commonly being used in much lower budget productions such as television advertising, music promos and video games. I will talk about the underlying problems involved in camera tracking and motion capture, and will illustrate the modern approaches to them using 2d3's "boujou" camera tracker and ViconPeak's "IQ" motion capture software.
Jay Cassidy (Mathematical Technologies, Inc.) The grammar of storytelling on film survives technology
Abstract: An art form barely 100 years old has seen the evolution of film grammar as new technologies emerge. Just as paintings changed when oil paint was overtaken by acrylic paint, motion pictures have seen three systemic technological changes: silent film to sound, black and white film to color, photochemical to digital. Each of these changes affected how filmmakers told stories. This talk - with many film clip samples - will attempt to give an overview of post-production technology with special emphasis on how the movement from photochemical to digital has affected the film editing process. In addition, the speaker, a film editor currently working in Hollywood, will describe his fifteen year adventure developing digital tools for film restoration.
Ruchira Datta (Google) The mathematics of web information retrieval
Abstract: Classical information retrieval was built on the assumption that patient users are trying to extract information from a clean, trustworthy, manageable corpus and no adversaries are trying to get in the way. We can't rely on any of these assumptions in the era of the web. In this talk we'll see how leveraging mathematics and large-scale systems leads us to good web information retrieval. Finally we'll describe an interesting open problem with applications to computing PageRank.
Andrew Fitzgibbon (Oxford University) 3D models from image sequences
Abstract: I shall talk about building 3D models from image sequences, and in particular about rendering new views of existing sequences in order to create stereoscopic 3D from monocular footage. I shall show how existing strategies for image-based rendering can be augmented using image-based priors to create realistic 3D views. In addition I will talk about the difficult problem of creating 3D when there is no camera motion.
Harald Grossauer (University of Innsbruck) Getting rid of scratches and blotches
Abstract: Currently lots of old celluloid movies are digitized to save them from decay. Most of the footage has already suffered from aging or from abrasion and should be processed to improve its quality. In our work we focus on removing scratches and blotches resulting from mechanical damage to the celluloid layer. Bearing in mind that even movies of moderate length consist of several thousands of frames we face two main problems. First, manually highlighting the corrupted pixels is not feasible, they have to be detected automatically. Second, processing time should be kept low. We employ a method based on the optical flow for detection and removal of scratches. Where this method fails we apply a hybrid still image inpainting technique, utilizing PDE inpainting and texture synthesis methods. Due to the use of efficient numerical algorithms an optimized implementation can achieve a processing time of a few ten seconds per frame. Further the algorithm is highly parallelizable except for a single step.
Gloria Haro Ortega (University of Minnesota) Restoration and zoom of irregularly sampled, blurred and noisy images by accurate total variation minimization with local constraints
Abstract: We propose an algorithm to solve a problem in image restoration which considers several different aspects of it, namely: irregular sampling, denoising, deconvolution, and zooming. Our algorithm is based on an extension of a previous image denoising algorithm proposed by A. Chambolle using total variation, combined with irregular to regular sampling algorithms proposed by H.G. Feichtinger, K. Groechenig, M. Rauth and T. Strohmer. Finally we present some experimental results and we compare them with those obtained with the algorithm proposed by K. Groechenig et al.

Joint work with Andres Almansa, Vicent Caselles and Bernard Rouge.

Pierre Jasmin (RE:Vision Effects, Inc.) Challenging shots: Demo your problem and discuss it with us.
Abstract: This is a moderated session. If anyone has media they can share in advance with other participants, so they can try things ahead of time, the purpose would be to structure a bit fast problem demos on really specific things, providing a context as well to bridge discussions between commercial tools and research. So, in some way, present in 5 minutes or less to a room of experts a real, specific problem.

Additional information and samples available at http://www.revisioneffects.com/IMA/ChallengingImaDemos.htm.

Laurent Joyeux (Joanneum Research) PrestoSpace project
Abstract: The European project PrestoSpace was started on February 2004. The project aims to provide a complete solution to preserve audiovisual material found in archives (e.g. BBC, RAI or INA). We are presenting a general overview of this project and focus on the restoration task by presenting all partners research topic. Finally, we will present more deeply Joanneum Research activities inside this project.
Anil Kokaram (Trinity College) Pushing pixels: From restoration to postproduction
Abstract: Joint with Bill Collis (The Foundry).

The need for the design of new image manipulation tools for both consumer and professional postproduction has substantially widened the breath of research in image/video/vision processing. While machine vision tools have been used successfully by industry for many years, it is only with the success of Digital Television and Digital Media Streaming that more sophisticated moving image processing has shown mainstream success in the worldwide community. This talk tries to chart a course showing how tools in restoration that have been considered for over a decade now, have migrated into the post-production community where they have metamorphosed into other applications. It highlights some emerging trends and tries to explore why researchers and post-production industrialists have become friends.

Chang-Ock Lee (KAIST) A variational approach to blending based on warping for non-overlapped images
Abstract: We present a new model for image blending based on warping. The model is represented by partial differential equations (PDEs) and gives a sequence of images, which has the properties of both blending of image intensities and warping of image shapes. We modified the energy functional in the work by Liao et. al. (2002) in order to adapt the idea of the shape warping to the image blending. The PDEs from the proposed energy functional cover not only overlapped images but also non-overlapped ones.
Lucia Maddalena (National Research Council (CNR)) New approaches to line scratch restoration
Abstract: We consider the problem of detecting and removing line scratches from digital image sequences. In particular, we present an approach based on data fusion techniques for combining relatively well settled distinct techniques. Moreover, focusing on blue scratches, we describe a detection method and a removal method that strongly rely on the specific features of such scratches. Evaluation of the proposed methods and numerical experiments on real images are reported.
Kevin Manbeck (MTI Technologies) Techniques and rationale for motion picture restoration
Abstract: Motion picture restoration spans the gamut from archival preservation of historically and culturally significant works to pragmatic treatment of low budget titles to extensive polishing of today's blockbusters. Each restoration project has its own idiosyncrasies, including original storage technology, type of damage, and final delivery requirements. Each project needs to strike its own balance between speed and accuracy of processing. Restoration must be approached in a way that addresses the peculiarities and considerations unique to the material. This talk presents some of the prototypical challenges encountered in motion picture restoration. The evolution and consequences of several prevalent storage technologies are discussed. Various sources of image degradation and damage---both common and unusual---are demonstrated. State-of-the-art restorations techniques are presented and critiqued. The requirements and properties of modern delivery mechanisms are explained.
John W. Mateer (University of York) The evolution of film editing technique and its implications to the parsing and summarization of motion pictures
Abstract: Techniques employed in film editing have evolved rapidly with increasingly sophisticated and complex methods being used to enhance storytelling. This talk will examine the relationship between scene and shot, picture and sound with a discussion of how an understanding of editing technique can be leveraged to enhance the automated analysis of film content.

About the Speaker: John Mateer joined the University of York in 2001 specifically to design and develop the media production and analysis components of an innovative new teaching and research initiative in Media Technology. His expertise lies in the integration and application of new media technologies in different traditional media production contexts. Prior to this appointment, he worked for over 15 years as a producer, director and production consultant. He is a graduate of NYU's Tisch School of the Arts and AFI's Center for Advanced Film and Television Studies, and is an active member of the Directors Guild of Great Britain.

Philip McLauchlan (Imagineer Systems Ltd) Tracking planes in images: Applications in post-production
Abstract:Imagineer Systems Ltd was founded in 2000 with the aim of building innovative products based around computer vision technology. Our first products mokey, has helped to automate various important tasks in film and video post-production, including wire and rig removal, stabilisation, lens distortion correction and matte creation. Two new products, monet and motor, are specialised to compositing and rotoscoping applications. Our core technology is a fast and accurate tracker for affine and projective 2D motion.

In my seminar I shall relate some of the history of the company and summarise the algorithms and software we have developed, in particular our "Gandalf" computer vision library (see gandalf-library.sf.net). There will be extensive demonstrations of mokey and monet. If time permits, I will present a mathematical conundrum in the area of the normalisation of projective quantitities.

Alvaro Pardo (Instituto de Ingenieria Electrica) Video content analysis
Abstract: In this work a software framework for processing video data integrating existing open source libraries and a set of applications for video content analysis is presented. These are partial results of an ongoing project. Due to the huge amount of data contained in video sequences, a set of constraints were considered while designing the system to keep the computational cost and memory requirements as low as possible, which led to a simple and effective system architecture. The system also includes a MPEG7-like description module that allows extracting and storing a video content description. Based on this system a set of applications for object extraction, shot detection, and content analysis were developed. These applications were used to test the developed software and to develop new solutions. Novel results for object extraction, shot detection and content description are presented.
Alessandro Rizzi (University of Milan) A human color perception model for film unsupervised digital restoration
Abstract: Film based media becomes unstable over time, unless they are stored at low temperatures and the humidity is controlled. Some defects, such as bleaching, are difficult to solve using photochemical restoration methods; in such cases, digital restoration can be an alternative solution. The basic idea of the proposed work is to mimic the robust capabilities of the human vision system (HVS) to set up a tool to filter damaged frames in a partially automated way. In fact, film colour cast, caused by ageing, can be considered as generic chromatic noise, thus a colour constancy method can be suitable for restoring it. Moreover a colour constancy method inspired by the HVS behaviour does not need any a-priori information about the colour cast and its magnitude. Another advantage of HVS inspired algorithms is their local effect since film chemical deterioration is usually non-uniform. Several test have been performed with an algorithm called ACE (Automatic Colour Equalization). The technique, presented here, is not just an application of ACE on movie images, but also an enhancement of ACE principles to meet the requirements of digital film restoration practice. The basic ACE computation extracts autonomously the visual content of the frame, correcting colour cast if present and expanding its dynamic range. This behaviour is not always a good restoring solution: there are cases in which the cast has to be maintained (e.g. underwater shots) or the dynamic range has not to be expanded (e.g. sunset or night shots). To this aim, new functions have been added to preserve the natural histogram shape, adding new efficacy in the restoration process. Examples are presented to discuss characteristics, advantages and limits of the use of perceptual models in digital movies colour restoration.
Daniel Rockmore (Dartmouth College) Math Matters: IMA Public Lecture     Artful mathematics
Abstract: All too often we see mathematics and the arts as two sides of the science/humanities coin. In this talk we'll see a place in which the two come naturally together in exciting new research. In today's world in which almost all aspects of life are brought to the common medium of the computer, it is now possible to quantify and extract the style of an artist via computation. Examples are gleaned from the literary, visual, and dance arts, and include applications to the problem of authentication. Taken together this work reveals just how stylish math can be. See http://www.ima.umn.edu/2005-2006/PUB2.8.06/
Daniel Rockmore (Dartmouth College) Mathematics: Maker and muse for modern art
Abstract:For all the discussion of right brain/left brain conflict, mathematics and the arts actually have a very healthy relationship, one which can perhaps be traced to their common goal of finding a way to give expression to the grand truths of experience. Mathematician Dan Rockmore will take us on a tour of modern art history and point out some of the surprising ways in which mathematical ideas have been and continue to be an enabler as well as inspiration for some of the big ideas in the visual arts.
Daniel Rosen (Warner Bros.) "But you said it was 99% accurate! Why is there a defect in the first frame?”
Gentle techniques for using mathematics in motion picture processes to marry film and digital
Abstract: The world of motion picture production and restoration is populated by artists and business people; not mathematicians. Yet, as more and more digital processes, many of which implement non-deterministic algorithms, are introduced into the industry, the probability of error at any stage has increased while the probability of identification and correction has decreased. In addition, many people who came into the industry when silver halide film and photochemistry were the only tools, lack the understanding of digital tools of people newer to the industry and vice versa. I will discuss in detail the mathematical and social solutions developed to solve problems in principal photography and post production related to digital capture defect correction, film image quality control and work flow. I will also detail how a thorough understanding of traditional photochemical film processes allows for the creation of processes with an optimal combination of mathematics, art, analog and digital while, at the same time, educating cinematographers, directors and executives in how to recognize and overcome the complexities of these processes.
Guillermo R. Sapiro (University of Minnesota) Image and video inpainting
Abstract: In this talk we will review basic techniques for image inpainting and present new ones for video inpainting under constrained camera motion.
Richard Szeliski (Microsoft Research) Video-based rendering
Abstract: Image-based rendering has been one of the hottest areas in computer graphics in recent years. Instead of using CAD and painting tools to construct graphics models by hand, IBR uses real-world imagery to rapidly create extremely photorealistic shape and appearance models. However, IBR results to date have mostly been restricted to static objects and scenes.

Video-based rendering brings the same kind of realism to computer animation, using video instead of still images as the source material. Examples of VBR include facial animation from sample video, repetitive video textures that can be used to animate still scenes and photos, 3D environment walkthroughs built from panoramic video, and 3D video constructed from multiple synchronized cameras. In this talk, I survey a number of such systems developed by our group and by others, and suggest how this kind of approach has the potential to fundamentally transform the production (and consumption) of interactive visual media.

About the Speaker : Richard Szeliski leads the Interactive Visual Media Group at Microsoft Research, which does research in digital and computational photography, video scene analysis, 3-D computer vision, and image-based rendering. He received a Ph.D. degree in Computer Science from Carnegie Mellon University, Pittsburgh, in 1988. He joined Microsoft Research in 1995. Prior to Microsoft, he worked at Bell-Northern Research, Schlumberger Palo Alto Research, the Artificial Intelligence Center of SRI International, and the Cambridge Research Lab of Digital Equipment Corporation. Dr. Szeliski has published over 100 research papers in computer vision, computer graphics, medical imaging, and neural nets, as well as the book Bayesian Modeling of Uncertainty in Low-Level Vision. He was a Program Committee Chair for ICCV'2001 and the 1999 Vision Algorithms Workshop, served as an Associate Editor of the IEEE Transactions on Pattern Analysis and Machine Intelligence and on the Editorial Board of the International Journal of Computer Vision, and is a Founding Editor of Foundations and Trends in Computer Graphics and Vision.

Eugene Vendrovsky (Rhythm & Hues Studios, Inc.) Computer vision in movie production: problems and expectations
Abstract: I shall talk about some practical problems of special effects / animation production that could be broadly defined as Inverse Problems: data is recovered from observed images rather than generated on user specification. Most of Computer Vision tasks in movie production, opposite to most Computer Graphics tasks, fall in Inverse Problems category. The problems I will speak of are: rigid body surveyless, articulated body (as hierarchy of rigid objects), face and flexible surface tracking (markers based, markerless, featureless) and photomodelling including what is practically important for our industry in algorithms development and our expectations. Automatic 3d terrain generation from monocular camera image sequence will be presented as an example of discussed problems and solutions.
Martin Welk (University of the Saarland) Analysis of Space-Discrete Image Filters by Dynamical Systems
Abstract: In digital image processing, filters based on partial differential equations (PDEs) are known as a versatile tool. Among the most interesting filters of this kind are some that rely on PDEs with singularities which make them difficult to analyse and implement. Dynamical systems of ordinary differential equations can be used to model spatially discretised versions of PDE-based filters. Singularities are often easier to control, and useful theoretical properties for the discretised filters can be derived. In some cases, it is even possible to obtain explicit analytic solutions that lead to novel numerical methods for the underlying PDEs. The examples discussed in the talk reveal also interesting relations to image filters defined via wavelets and via local image statistics.
Visitors in Residence
Andres Almansa Facultad de Ingeniería 2/5/2006 - 2/12/2006
Jung-Ha An University of Minnesota 9/1/2005 - 8/31/2007
Pablo Arias Universidad de la Republica 2/5/2006 - 2/11/2006
Douglas N. Arnold University of Minnesota 7/15/2001 - 8/31/2006
Donald G. Aronson University of Minnesota 9/1/2002 - 8/31/2006
Evgeniy Bart University of Minnesota 9/1/2005 - 8/31/2007
Sebastiano Battiato University of Catania 2/4/2006 - 2/10/2006
Marcelo Jose Bertalmio Universitat Pompeu Fabra 2/3/2006 - 2/17/2006
Bernard Besserer Universite de La Rochelle 2/4/2006 - 2/8/2006
Michael Black Brown University 2/5/2006 - 2/10/2006
Francisco Blanco-Silva Purdue University 9/1/2005 - 6/30/2006
Tom Brigham RVI 2/5/2006 - 2/10/2006
Toni Buades Universitat of Illes Balears 2/4/2006 - 2/11/2006
David Capel 2d3 Limited 2/4/2006 - 2/10/2006
Jay Cassidy Mathematical Technologies, Inc. 2/5/2006 - 2/8/2006
Qianyong Chen University of Minnesota 9/1/2004 - 8/31/2006
Steven Benjamin Damelin Georgia Southern University 8/9/2005 - 6/30/2006
James Damon University of North Carolina 2/27/2006 - 6/30/2006
Ruchira Datta Google 2/2/2006 - 2/3/2006
Julie Delon Ecole Normale Superieure de Cachan 2/5/2006 - 2/12/2006
Brian DiDonna University of Minnesota 9/1/2004 - 8/31/2006
Andrew Fitzgibbon Oxford University 2/5/2006 - 2/10/2006
Harald Grossauer University of Innsbruck 2/5/2006 - 2/12/2006
Changfeng Gui University of Connecticut 9/12/2005 - 6/30/2006
Jooyoung Hahn KAIST 8/26/2005 - 7/31/2006
Margaret Hanson University of Minnesota 1/18/2006 - 1/18/2007
Gloria Haro Ortega University of Minnesota 9/1/2005 - 8/31/2007
Sheryl Hemkin Kenyon College 2/6/2006 - 2/10/2006
Michael Hintermueller University of Graz 2/5/2006 - 2/16/2006
Xiang Huang University of Connecticut 9/1/2005 - 6/30/2006
Allan Jaenicke Imagineer Systems Ltd 2/5/2006 - 2/10/2006
Pierre Jasmin RE:Vision Effects, Inc. 2/5/2006 - 2/10/2006
Sookyung Joo University of Minnesota 9/1/2004 - 8/31/2006
Laurent Joyeux Joanneum Research 2/4/2006 - 2/11/2006
Sung Ha Kang University of Kentucky 1/1/2006 - 5/31/2006
Chiu Yen Kao University of Minnesota 9/1/2004 - 8/31/2006
Gisela Klette University of Auckland 2/4/2006 - 5/31/2006
Reinhard Klette University of Auckland 2/4/2006 - 5/31/2006
Anil Kokaram Trinity College 2/5/2006 - 2/10/2006
Shin Kurokawa Maximum Output Designs, Inc. 2/5/2006 - 2/10/2006
Matthias Kurzke University of Minnesota 9/1/2004 - 8/31/2006
Song-Hwa Kwon University of Minnesota 8/30/2005 - 8/31/2007
Louis Laborelli Institut de l'Audiovisuel 2/5/2006 - 2/10/2006
Chang-Ock Lee KAIST 8/1/2005 - 7/31/2006
Nam-Yong Lee Inje University 1/20/2006 - 2/11/2006
Stacey E. Levine Duquesne University 12/30/2005 - 6/30/2006
Debra Lewis University of Minnesota 7/15/2004 - 8/31/2006
Hstau Liao University of Minnesota 9/2/2005 - 8/31/2007
Peter Litwinowicz RE:Vision Effects, Inc. 2/5/2006 - 2/10/2006
Bradley J. Lucier Purdue University 8/15/2005 - 6/30/2006
Lucia Maddalena National Research Council (CNR) 2/5/2006 - 2/12/2006
Alison Malcolm University of Minnesota 9/1/2005 - 8/31/2006
Kevin Manbeck MTI Technologies 2/5/2006 - 2/10/2006
John W. Mateer University of York 2/4/2006 - 2/12/2006
Donald E. McClure Brown University 2/5/2006 - 2/10/2006
Philip McLauchlan Imagineer Systems Ltd 2/5/2006 - 2/10/2006
Mila Nikolova Ecole Normale Superieure de Cachan 2/5/2006 - 2/12/2006
Peter J. Olver University of Minnesota 9/1/2005 - 6/30/2006
Alvaro Pardo Instituto de Ingenieria Electrica 2/5/2006 - 2/19/2006
Peter Philip University of Minnesota 8/22/2004 - 8/31/2006
Gerard Benjamin Pierre Pocket Rocket FX 2/5/2006 - 2/10/2006
Gregory J. Randall Universidad de la Republica 8/18/2005 - 7/31/2006
Walter Richardson, Jr. University of Texas - San Antonio 9/1/2005 - 6/30/2006
Alessandro Rizzi University of Milan 2/4/2006 - 2/11/2006
Daniel Rockmore Dartmouth College 2/7/2006 - 2/9/2006
Daniel Rosen Warner Bros. 2/5/2006 - 2/10/2006
Fadil Santosa University of Minnesota 9/1/2005 - 6/30/2006
Guillermo R. Sapiro University of Minnesota 9/1/2005 - 6/30/2006
Arnd Scheel University of Minnesota 7/15/2004 - 8/31/2006
Jin Keun Seo Yonsei University 1/5/2006 - 6/5/2006
Tatiana Soleski University of Minnesota 9/1/2005 - 8/31/2007
Pablo Sprechmann Universidad de la Republica 2/6/2006 - 2/10/2006
Vladimir Sverak University of Minnesota 9/1/2005 - 6/30/2006
Richard Szeliski Microsoft Research 2/5/2006 - 2/8/2006
Carl Toews University of Minnesota 9/1/2005 - 8/31/2007
Eugene Vendrovsky Rhythm & Hues Studios, Inc. 2/5/2006 - 2/10/2006
Jingyue Wang Purdue University 9/1/2005 - 6/30/2006
Xiaoqiang Wang University of Minnesota 9/1/2005 - 8/31/2007
Martin Welk University of the Saarland 12/5/2005 - 3/10/2006
Michael Werman The Hebrew University 2/6/2006 - 2/26/2006
Lance J. Williams Applied Minds 2/5/2006 - 2/10/2006
Ofer Zeitouni University of Minnesota 9/1/2005 - 6/30/2006
Andrew Zisserman Oxford University 2/4/2006 - 2/10/2006
Legend: Postdoc or Industrial Postdoc Long-term Visitor

Participating Institutions: Air Force Research Laboratory, Carnegie Mellon University, Consiglio Nazionale delle Ricerche, Georgia Institute of Technology, Indiana University, Iowa State University, Kent State University, Lawrence Livermore National Laboratories, Los Alamos National Laboratory, Michigan State University, Mississippi State University, Northern Illinois University, Ohio State University, Pennsylvania State University, Purdue University, Rice University, Rutgers University, Sandia National Laboratories, Seoul National University, Texas A & M University, University of Chicago, University of Cincinnati, University of Delaware, University of Houston, University of Illinois - Urbana-Champaign, University of Iowa, University of Kentucky, University of Maryland, University of Michigan, University of Minnesota, University of Notre Dame, University of Pittsburgh, University of Texas - Austin, University of Wisconsin - Madison, University of Wyoming, Wayne State University
Participating Corporations: 3M, Boeing, Corning, ExxonMobil, Ford, General Electric, General Motors, Honeywell, IBM, Johnson & Johnson, Lockheed Martin, Medtronic, Motorola, Schlumberger, Siemens, Telcordia