This is joint work with Michael Wakin, Hyeokho Choi, and David Donoho.

FRF are of interest because:
(a) they are commonly used as texture features in automated texture classification systems, and
(b) they are typically proposed as the "back pocket model" of the first stage of the human visual system.

I'll show how naïve classifiers built using these simple features can fail, and how the model can be used to produce a classifier that is robust to illumination variation. What this will show is that single still images are not often not sufficient for the purposes of surface classification - either for human or automated systems. I'll conclude by describing some of our recent research that is investigating our perceptions of surface texture.

In this talk, we explain how it is possible to determine a catalog of possible local models for the generic interplay between geometric features and shadow curves. This catalogue can be expanded to included the expected changes in such models under movement in viewing direction. Such a catalog is constructed through the use of singularity theory, which is a mathematical theory that allows construction of such classifications based on stability and possible perturbations. We explain the general features of the classification and indicate how it is obtained.

This is the result of joint work carried out with Peter Giblin and Gareth Haslinger.

In this talk I will discuss recent work that attempts to bring together psychophysical, computational and physiological approaches to understanding contour processing in natural images. A unifying foundation for this effort is a continuing project to measure and model the statistics of natural image contours. These ecological results lead to new computer vision algorithms for natural contour grouping, normative models for contour processing that may be evaluated psychophysically, and to new models for neural selectivity to natural image contours that may be tested against physiological data.

Invitation to submit examples: I invite audience members to submit examples of motion-blurred photographs to me a few days ahead of time. I'll show the images you submit, and the result of our algorithm applied to them. If you have a favorite blind deconvolution or restoration algorithm, please apply it to your image and send it and I'll show that, too.

Joint work with: Rob Fergus, Barun Singh, both from MIT CSAIL, and Aaron Hertzman and Sam Roweis, both from the University of Toronto.

The first problem is that of color interpolation (also called demosaicking). The pixels of most silicon sensors capture a single color measurement, usually a red, green, or blue color value. Because a fully processed color image requires all three color values at each pixel, two additional color values must be provided at each pixel. An algorithm is presented that addresses this problem for sensors using the Bayer color filter pattern.

The second problem is that of color aliasing. While the problem of aliasing is always present in a discrete imaging system, it is compounded for color sensors because the different color channels can alias in different ways. The resulting interference patterns have distinctive color components which constitute an obvious and annoying imaging artifact. An algorithm is presented that addresses this problem for sensors using the Bayer color filter pattern.

The talk informs at a general level about new architectures of panoramic cameras (as designed and produced at DLR, the German Air and Space Institute at Berlin), their use for stereo imaging based on studies at CITR, and the combination of those high-resolution images (about 350 Megapixel each) with range data generated by a laser range finder. Results are illustrated for different objects such as the castle "Neuschwanstein" in Bavaria/Germany.

Researchers studying surface color perception have typically used stimuli that consist of a small number of matte patches (real or simulated) embedded in a plane perpendicular to the line of sight (a Mondrian, Land & McCann, 1971). Reliable estimation of surface properties analogous to color is a difficult if not impossible computational problem in such limited scenes (Maloney, 1999). In more realistic, three-dimensional scenes the problem is not intractable, in part because considerable information about the spatial and spectral distribution of the illumination is usually available. We describe a series of experiments that (1) explore how the human visual system discounts the spatial and spectral distribution of the illumination (SSDI) in judging matte surface color and (2) what cues the visual system uses in estimating the SSDI of in a scene. We find that the human visual system uses information from cast shadows and specular reflections in estimating the SSDI and, when more than one cue type is present, combines these cues effectively. The SSDI can be very complex in scenes with many different light sources. We examine (3) the limits of human visual representation of the SSDI, reporting an experiment intended to tests these limits. Our results indicate that the human visual representation of the SSD of the illumination in a scene is well- matched to the task of perception of matte surface color perception.

Land, E. H. & McCann, J. J. (1971), Lightness and retinex theory. Journal of the Optical Society of America, 61,1-11. Maloney, L. T. (1999), Physics-based approaches to modeling surface color perception. In Gegenfurtner, K. R., & Sharpe, L. T. [Eds] (1999), Color Vision: From Genes to Perception. Cambridge, UK: Cambridge University Press, pp. 387-422.

A framework for studying texture in general, and for texture mixing in particular, is presented in this work. The work follows concepts from universal type classes and universal simulation. Based on the well-known Lempel and Ziv (LZ) universal compression scheme, the universal type class of a one dimensional sequence is defined as the set of possible sequences of the same length which produce the same dictionary (or parsing tree) with the classical LZ incremental parsing algorithm. Universal simulation is realized by sampling uniformly from the universal type class, which can be efficiently implemented. Starting with a source texture image, we use universal simulation to synthesize new textures that have, asymptotically, the same statistics of any order as the source texture, yet have as much uncertainty as possible, in the sense that they are sampled from the broadest pool of possible sequences that comply with the statistical constraint. When considering two or more textures, a parsing tree is constructed for each one, and samples from the trees are randomly interleaved according to pre-defined proportions, thus obtaining a mixed texture. As with single texture synthesis, the k-th order statistics of this mixture, for any k, asymptotically approach the weighted mixture of the k-th order statistics of each individual texture used in the mixing. We present the underlying principles of universal types, universal simulation, and their extensions and application to mixing two or more textures with pre-defined proportions.

The increasing photorealism for computer graphics has made computer graphics a convincing form of image forgery. Therefore, classifying photographic images and photorealistic computer graphics has become an important problem for image forgery detection. We propose a new geometry based image model, motivated by the physical image generation process, to tackle the above-mentioned problem. The proposed model reveals certain physical differences between the two image categories, such as the gamma correction in photographic images and the sharp structures in computer graphics. For the problem of image forgery detection, we propose two levels of image authenticity definition, i.e., imaging-process authenticity and scene authenticity, and analyze our technique against these definitions. Such definition is important for making the concept of image authenticity computable. Apart from offering physical insights, our technique with a classification accuracy of 83.5% outperforms those in the prior work, i.e., wavelet features at 80.3% and cartoon features at 71.0%. We also consider a recapturing attack scenario and propose a counter-attack measure.