HOME    »    PROGRAMS/ACTIVITIES    »    Annual Thematic Program
Abstracts and Talk Materials
Integration of Sensing and Processing
December 5-9, 2005


Multi-sensor adaptive ISP

In this talk we examine the adaptive integration of sensing and signal processing in several settings. The underlying algorithms are based on information-theoretic active learning, as well as partially observable Markov decision processes (POMDPs). Modern techniques from machine learning are also considered. We concentrate on applying the underlying mathematics to specific practical sensing challenges, such as detection of buried landmines and unexploded ordnance, adaptive hyper-spectral sensing, and underwater acoustic sensing of concealed targets.


Second Chances

Thursday, December 8


Second Chances

Monday, December 5


Second Chances

Friday, December 9


Second Chances

Tuesday, December 6


Second Chances

Wednesday, December 7


Jung-Ha An (University of Minnesota Twin Cities)

Modified Mumford-Shah model based simultaneous segmentation and registration

A new variational region based model for a simultaneous image segmentation and registration using modified Mumford-Shah technique is suggested. The purpose of the model is segment and register given images simultaneously utilizing modified Mumford-Shah technique and region intensity values. The segmentation is obtained by minimizing modified Mumford-Shah model. A global rigid registration is assisted by the segmentation information and region intensity values. The numerical experiments of the proposed model are tested against synthetic and simulated human brain MRI images. The experimental results show the effectiveness of the model in detecting the boundaries of the given objects and registering given images simultaneously.


D. Gregory Arnold (Air Force Research Laboratory)

Closing the loop for ISP using performance prediction

ATR Theory, especially the performance prediction aspects, are fundamental to integrated sensing and processing. Offline and online prediction and feedback are essential processing tools for assessing which sensing actions will likely provide the most information. I'll discuss the overlap between Active Vision, ATR Theory, and ISP and highlight relevant research issues along the way.


Richard E. Blahut (University of Illinois at Urbana-Champaign)

The Richardson-Lucy algorithm in image processing

video recording only


David J. Brady (Duke University) http://www.ee.duke.edu/faculty/profile.php?id=204

Compressive optical spectroscopy

Optical spectroscopy offers an ideal area for testing and development of compressive sensing systems. Measurement costs can be high, generalized sampling strategies are easily implemented and prior information and feature specific tasks are common. This talk describes experimental and conceptual studies of compressive spectroscopy in the DISP group over the past several years.


Emmanuel J. Candes (California Institute of Technology)

Compressive sampling

Conventional wisdom and common practice in acquisition and reconstruction of images or signals from frequency data follows the basic principle of the Nyquist density sampling theory. This principle states that to reconstruct an image/signal, the number of Fourier samples we need to acquire must match the desired resolution of the image/ signal, e.g. the number of pixels in the image.

This talk introduces a newly emerged sampling theory which shows that this conventional wisdom is inaccurate. We show that perhaps surprisingly, images or signals of scientific interest can be recovered accurately and sometimes even exactly from a limited number of nonadaptive random measurements. In effect, the talk introduces a theory suggesting ``the possibility of compressed data acquisition protocols which perform as if it were possible to directly acquire just the important information about the image of interest.'' In other words, by collecting a comparably small number of measurements rather than pixel values, one could in principle reconstruct an image with essentially the same resolution as that one would obtain by measuring all the pixels, a phenomenon with far reaching implications.

The reconstruction algorithms are very concrete, stable (in the sense that they degrade smoothly as the noise level increases) and practical; in fact, they only involve solving convenient convex optimization programs.


Ronald Raphael Coifman (Yale University)

Integration of intrinsic geometries of data into the sensing and processing streams

video recording only


Marco F. Duarte (Rice University) http://www.ece.rice.edu/~duarte

Distributed compressive sampling: A framework for integrated sensing and processing for signal ensembles

Compressive sampling is an emerging field based on the revelation that a small collection of linear projections of a sparse signal contains enough information for reconstruction. We introduce a new theory for distributed compressive sampling (DCS) that enables new distributed coding algorithms for multi-signal ensembles that exploit both intra- and inter-signal correlation structures, which are prevalent in sensor networks. The DCS theory rests on a new concept that we term the joint sparsity of a signal ensemble. We study in detail three simple models for jointly sparse signals, propose algorithms for joint recovery of multiple signals from incoherent projections, and establish upper and lower bounds on the measurement rates required for encoding such signals.

This is joint work with Shriram Sarvotham, Dror Baron, Michael Wakin and Richard Baraniuk.


James R. Fienup (University of Rochester)

Imaging without an imaging system

One can trade off complexity in imaging system hardware and strict system tolerances for complexity in post-detection data processing. We describe an extreme example in this trade-off space: a coherent imaging approach that eliminates the imaging system hardware entirely (except for the detector array), relying on a phase retrieval algorithm to form the image in a computer. It has applications ranging from long-range imaging, such as ballistic missile defense, to microscopy.


Michael E. Gehm (Duke University)

Tomographic hyperspectral imaging without a missing-cone

A hyperspectral imager provides a 3-D data cube in which the spatial information (2-D) of an image is complemented by spectral information (1-D) about each spatial location. In a tomographic hyperspectral imager, the full 3-D data cube is reconstructed from a series of 2-D projections. In current systems, the projections are taken over a range of angles with respect to (and about) the wavelength-axis. Because a full range of angles cannot be measured, the system is undersampled in the Fourier domain (the so-called "missing cone"). We propose a new technique that is based upon our static, Hadamard-coded spectrometer. Using aperture coding allows us to measure projections through the data cube that are perpendicular to the wavelength-axis. With these projections, we can sample the a full angular range and avoid a "missing cone". As a result, the quality of the tomographic reconstruction should be significantly improved. We have constructed some preliminary instruments that provide proof-of-concept, and have begun working on more robust implementations.


Alfred O. Hero III (University of Michigan) http://www.eecs.umich.edu/~hero/hero.html

Self-localization in wireless sensor networks via manifold learning

Given a set of noisy pair-wise measurements in a wireless network of N sensors the self-localization problem is to estimate all N coordinates. We present a manifold learning approach to this problem using measured connectivity (within range or out-of-range) or received signal strength (RSS) btwn pairs of sensors. The advantage of such approaches is that they use dimensionality reduction to find robust estimates of the sensor locations without requireing a specific propagation model for the medium. Two methods are presented and compared: Distributed weighted multi-dimensional scaling (dwMDS) for range-based localization (J.A. Costa, N. Patwari, A.O. Hero, Distributed Weighted Multidimensional Scaling for Node Localization in Sensor Networks, ACM Journal on Sensor Networks, 2006) and Laplacian Eigenmaps (LE) for connectivity-based localization (N. Patwari and A.O. Hero Adaptive Neighborhoods for Manifold Learning-based Sensor Localization, IEEE SPAWC 05, June 2005).


Alfred O. Hero III (University of Michigan) http://www.eecs.umich.edu/~hero/hero.html

Dimensionality reduction for integrated sensing and processing

Dimensionality reduction methods have played a central role in exploration of parsimonious structural models, complexity regularization in inverse problems, and data compression. Examples are PCA, Laplacian eigenmaps, ISOMAP, and matching pursuits which attempt to fit a subspace to the data. For integrated sensing and processing (ISP) systems dimensionality reduction must go beyond simply fitting the data geometry. One must account for how dimension reduction will affect the performance of the processing task, e.g., image reconstruction or classification, and sensor scheduling, e.g., path planning or waveform design. We will present some thoughts and recent progress on dimensionality reduction for ISP.


Philip J. Holmes (Princeton University) http://mae.princeton.edu/people/e21/holmes/profile.html

IMA Public Lecture: Does Math Matter to Brain Matter?

The human brain contains about 100 billion neurons, each making about 1000 synaptic connections with other neurons. This huge dynamical system communicates with itself and its environment via electrical impulses called spikes. How is incoming information turned into spikes, and how do spikes create decisions and behaviors? I will show how mathematics helps us model and analyze such questions, involving events from single neural spikes to decisions that change our lives.


Seongjai Kim (Mississippi State University)

The Method of Nonflat Time Evolution (MONTE) in PDE-based image restoration

Joint work with Youngjoon Cha.

This article is concerned with effective numerical techniques for partial differential equation (PDE)-based image restoration. Numerical realizations of most PDE-based denoising models show a common drawback: loss of fine structures. In order to overcome the drawback, the article introduces a new time-stepping procedure, called the method of nonflat time evolution (MONTE), in which the timestep size is determined based on local image characteristics such as the curvature or the diffusion magnitude. The MONTE provides the PDE-based restoration models with an effective mechanism for the equalization of the net diffusion over a wide range of image frequency components. It can be easily applied to diverse evolutionary PDE-based restoration models and their spatial and temporal discretizations. It has been numerically verified that the MONTE results in a significant reduction in nonphysical dissipation and preserves fine structures such as edges and textures satisfactorily, while it removes the noise with an improved efficiency. Various numerical results are shown to confirm the claim.


Stephane Lafon (Google)

Data fusion and multi-cue data matching using diffusion maps

Data fusion and multi-cue data matching are fundamental tasks arising in a variety of systems that process large amounts of data. These tasks often rely on dimensionality reduction techniques that traditionally follow a data acquisition/reprocessing phase. In this talk, I will describe a powerful framework based on diffusions that can be used in order to learn the intrinsic geometry of data sets. These techniques allow to simultaneously handle data acquisition issues and data processing tasks. In particular, I will explain how we can use this set of tools in order to address three major challenges related to data fusion:

1) How to deal with data coming from sensors/sources sampled at different rates, and possibly at different times. We provide algorithms to obtain density-invariant descriptors (parametrization) of data sets.

2) How to integrate and combine information streams coming from different sensors into one representation of the data. The diffusion coordinates allow to learn the geometry of the data captured by each sensor independently, and then to combine the various representations into a unified description of the data.

3) How to do matching of data sets based on their intrinsic geometry. As an illustration, I will present numerical results on the integration of audio and video streams for lip-reading and speech recognition. Other examples will be more focused on imaging (multiscale data-driven image segmentation, image data sets alignment).

This is joint work with R.R. Coifman, A. Glaser, Y. Keller and S.W. Zucker (Yale university).


Hstau Liao (University of Minnesota Twin Cities)

Direct reconstruction-segmentation, as motivated by electron microscopy

Quite often in electron microscopy it is desired to segment the reconstructed volumes of biological macromolecules. Knowledge of the 3D structure of the molecules can be crucial for the understanding of their biological functions. We propose approaches that directly produce a label (segmented) image from the tomograms (projections).

Knowing that there are only a finitely many possible labels and by postulating a Gibbs prior on the underlying distribution of label images, we show that it is possible to recover the unknown image from only a few noisy projections.

Joint work with Gabor T. Herman.


Russell Luke (University of Delaware)

A new generation of iterative transform algorithms for phase contrast tomography

In recent years, improvements in electromagnetic sources, detectors,optical components, and computational imaging have made it possible to achieve three-dimensional atomic-scale resolution using tomographic phase-contrast imaging techniques. These greater capabilities have placed a premium on improving the efficiency and stability of phase retrieval algorithms for recovering the missing phase information in diffraction observations. In some cases, so called direct methods suffice, but for large macromolecules and nonperiodic structures one must rely on numerical techniques for reconstructing the missing phase. This is the principal motivation of our work. We report on recent progress in algorithms for iterative phase retrieval. The theory of convex optimisation is used to develop and to gain insight into counterparts for the nonconvex problem of phase retrieval. We propose a relaxation ofaveraged alternating reflectors and determine the fundamental mathematical properties of the related operator in the convex case. Numerical studies support our theoretical observations and demonstrate the effectiveness of the newer generation of algorithms compared to the current state of the art.


Robert Muise (Lockheed Martin)

Target detection using integrated hyper spectral sensing and processing

Joint work with Abhijit Mahalanobis.

The concept of Integrated Sensing and Processing (ISP) suggests that a sensor should collect data in a manner that is consistent with the end objective. Thus ISP seeks to minimize the collection of redundant data, reduce processing time and improve overall performance. We present a case study of ISP using a multi-spectral camera which allows the spatial resolution of the data to be varied in addition to selecting spectral bands that enable the detection of targets in background clutter. This paper is a preliminary description of work in progress, and illustrates the basic concepts by means of several examples.


Robert Nowak (University of Wisconsin)

Active learning vs. compressed sensing

Adaptive sampling, also called ``Active Learning'', uses information gleaned from previous measurements (e.g., feedback) to guide and focus the sampling process. Theoretical and experimental results have shown that adaptive sampling can dramatically outperform conventional non-adaptive sampling schemes. I will review some of the most encouraging theoretical results to date, and focus on new results regarding the capabilities of adaptive sampling methods for learning piecewise smooth functions. I will also contrast adaptive sampling with a new approach known as compressive sampling. Compressive sampling, or ``Compressed Sensing'', has generated a tremendous amount of excitement in the signal processing community and is seen as a strong competitor of adaptive procedures. Compressive sampling involves taking a relatively small number of non-traditional samples in the form of non-adaptive randomized projections that are capable of capturing the most salient information in a signal. I will compare adaptive and compressive sampling in noisy conditions, and show that in certain interesting cases both schemes are near-optimal. This result is remarkable since it is the first evidence of cases in which compressive sampling, which is non-adaptive, cannot be significantly outperformed by adaptive procedures, even in presence of noise.

This is joint work with Rui Castro and Jarvis Haupt.


Sarah K. Patch (University of Wisconsin) http://www.phys.uwm.edu/people/faculty/patchs.html

Thermoacoustic Tomography

Abstract: pdf


Rafael Piestun (University of Colorado)

Shaping light waves in three dimensions for integrated computational imaging

Processing information at the sensor level can not only help reduce the amount of data acquired but also enhance the overall performance of the system. In this talk I will first present methods for shaping the three-dimensional response of an optical system. Then I will discuss how to integrate tailored optical responses with digital postprocessing algorithms to improve specific imaging tasks.


Carey E. Priebe (Johns Hopkins University)

On the role of the conditionality principle in dimensionality reduction

The idea of classifier construction via `Iterative Denoising' trees--- that is, by successively partitioning (at the internal nodes of the tree) a class-labeled training data set into ever-more homogeneous subsets without consideration of class labels, and only subsequently (at the leaves of the tree) using the available class-label information, while at each node (internal or leaf) choosing a dimensionality reduction appropriate to and specific to (the data falling in) that partition cell — may seem counter-intuitive but is in fact in (rough) accordance with Fisher's conditionality principle and can in fact provide performance superior to that of competing approaches. We describe the theory and application of these `Iterative Denoising' trees and illustrate their performance, and relate the ideas to `Integrated Sensing and Processing' and theorized thalamocortical brain circuit computation.


Gustavo Kunde Rohde (Naval Research Laboratory) www.math.umd.edu/~rohdeg

Sub-Pixel Image Registration and Quantitative Parameter Extraction

Registration methods are routinely used to automatically align images before quantitative parameters can be estimated from them. With many authors claiming sub-pixel accuracy in their registration procedures we show that the operations necessary for producing a series of images aligned to sub-pixel accuracy can significantly alter the statistical properties of the images. This, in turn, will have a significant effect on any quantitative parameters extracted from registered images. We look at the stochastic properties of B-spline interpolating basis functions of arbitrary degree and suggest means through which they can be used in quantitative parameter extraction from registered images. This study demonstrates by example the importance of integrating the stochastic properties of image sensors into image processing operations.


Guillermo R. Sapiro (University of Minnesota Twin Cities) http://www.ece.umn.edu/users/guille/

Constrained sensor localization and my wish list on integrated video processing

In this talk I will first show how to use simple and classical results from distance geometry to address the problem of sensor localization under physical constraints. Then I will move into presenting some recent results in video processing that I wish could be done at the sensor level. For example, I will show techniques that reduce the video data to only regions of interest. This will tremendously reduce transmission cost if done at the sensors level.

The work is in collaboration with members of my group (M. Mahmoudi and K. Patwardhan) and also in part with Honeywell (V. Morellas).


Nitesh Shah (Raytheon Company)

Dimensionality reduction and divergence estimation for polarization-resolution trade in SAR images

In the Integrated Sensing and Processing paradigm, agile sensors operating in a setting of limited power, bandwidth, etc. can receive feedback regarding sensor settings for subsequent data collection (sensor scheduling). In the system design phase, trades are often conducted balancing utility of different sensor settings or sensor configurations. Information-theoretic tools are useful for assisting in evaluating information gain in both of these settings. In particular, given fully polarimetric, measured synthetic aperture radar (SAR) images of two targets, we apply two dimensionality reduction techniques and a non-density-based divergence estimation approach to evaluate the relative target divergences over differences in effective spatial resolution and in the number of available polarization states. Divergence at the pre-classifier stage serves as a surrogate for target separability in an Automatic Target Recognition (ATR) setting, avoiding the extra layer of complexity induced by the choice of classifier and classifier parameter settings.


Kamil Ugurbil (University of Minnesota Twin Cities)

Imaging brain activity and chemistry using high magnetic fields

video recording only


Miguel Velez-Reyes (University of Puerto Rico) http://www.ece.uprm.edu/~mvelez

Hyperspectral Image Processing

We will present some of our work in hyperspectral image processing. Two projects will be presented. In the first, we will focus on multiscale representation and the use of PDE methods for image representation. In the second, we will present the use of Positive Matrix Factorization for hyperspectral unmixing.


Brani Vidakovic (Georgia Institute of Technology)

Wavelets in biomedical data analysis: Scaling and functional design in applications

Measured bioresponses are often characterized by an intrinsic high frequency and strong persistent correlations inhibiting statistical modeling by the traditional techniques. The talk overviews two novel wavelet-based techniques for modeling such challenging data.

Wavelet domains provide natural modeling environments for data that scale, as well as for data consisting of continuous n-dimensional functions. We briefly discuss technicalities and describe in detail two applications. First application deals with wavelet analysis of functional ANOVA (FANOVA) where the observations are curves coming from clinical research. The second application discusses classification methods based on wavelet-based measures of irregular scaling (multifractal spectra) applied on high frequency pupillary responses for patients with various eye pathologies.


Rebecca Willett (Duke University)

Smaller Infrared Cameras via Superresolution Image Reconstruction

Infrared camera systems can be make dramatically smaller by simultaneously collecting several low-resolution images with multiple narrow aperture lenses rather than collecting a single high- resolution image with one wide aperture lens. In this poster, we will describe this new infrared sensing system and a multiscale approach to processing the output of these novel sensors. The camera uses a 3x3 lenslet array having an effective focal length of 1.9 mm and we achieve image resolution comparable to a conventional single lens system having a focal length of 21mm, although the image dynamic range and linearity are reduced. The wavelet- based regularization utilized during image reconstruction reduces the appearance of artifacts while preserving key features such as edges and singularities. The processing method is very fast, making the integrated sensing and processing viable for both time- sensitive applications and massive collections of sensor outputs.

Infrared camera systems can be make dramatically smaller by simultaneously collecting several low-resolution images with multiple narrow aperture lenses rather than collecting a single high- resolution image with one wide aperture lens. In this poster, we will describe this new infrared sensing system and a multiscale approach to processing the output of these novel sensors. The camera uses a 3x3 lenslet array having an effective focal length of 1.9 mm and we achieve image resolution comparable to a conventional single lens system having a focal length of 21mm, although the image dynamic range and linearity are reduced. The wavelet- based regularization utilized during image reconstruction reduces the appearance of artifacts while preserving key features such as edges and singularities. The processing method is very fast, making the integrated sensing and processing viable for both time- sensitive applications and massive collections of sensor outputs.

This is joint work with David Brady, Mohan Shankar and Andrew Portnoy.


Steven W. Zucker (Yale University)

The geometry of color

The representation of color for perception differs from the representation for displays. We consider the map from (r,g,b)-space to (Intensity, hue, saturation)-space, and show how this non-linear space is useful for sensing devices.

Connect With Us:
Go