In this talk we examine the adaptive integration of sensing and signal
processing in several settings. The underlying algorithms are based on
information-theoretic active learning, as well as partially observable
Markov decision processes (POMDPs). Modern techniques from machine learning
are also considered. We concentrate on applying the underlying mathematics
to specific practical sensing challenges, such as detection of buried
landmines and unexploded ordnance, adaptive hyper-spectral sensing, and
underwater acoustic sensing of concealed targets.
Modified Mumford-Shah model based simultaneous
segmentation and registration
A new variational region based model for a
simultaneous image segmentation and registration using
modified Mumford-Shah technique is suggested. The
purpose of the model is segment and register given
images simultaneously utilizing modified Mumford-Shah
technique and region intensity values. The
segmentation is obtained by minimizing modified
Mumford-Shah model. A global rigid registration is
assisted by the segmentation information and region
intensity values. The numerical experiments of the
proposed model are tested against synthetic and
simulated human brain MRI images. The experimental
results show the effectiveness of the model in
detecting the boundaries of the given objects and
registering given images simultaneously.
D. Gregory Arnold (Air Force Research Laboratory)
Closing the loop for ISP using performance prediction
ATR Theory, especially the performance prediction aspects, are fundamental
to integrated sensing and processing. Offline and online prediction and
feedback are essential processing tools for assessing which sensing actions
will likely provide the most information. I'll discuss the overlap between
Active Vision, ATR Theory, and ISP and highlight relevant research issues
along the way.
Richard E. Blahut (University of Illinois at Urbana-Champaign)
Optical spectroscopy offers an ideal area for testing and development of compressive sensing systems. Measurement costs can be high, generalized sampling strategies are easily implemented and prior information and feature specific tasks are common. This talk describes experimental and conceptual studies of compressive spectroscopy in the DISP group over the past several years.
Emmanuel J. Candes (California Institute of Technology)
Compressive sampling
Conventional wisdom and common practice in acquisition and
reconstruction of images or signals from frequency data follows the
basic
principle of the Nyquist density sampling theory. This principle
states that to reconstruct an image/signal, the number of Fourier
samples we
need to acquire must match the desired resolution of the image/
signal, e.g.
the number of pixels in the image.
This talk introduces a newly emerged sampling theory which shows that
this conventional wisdom is inaccurate. We show that perhaps
surprisingly, images or signals of scientific interest can be
recovered accurately and sometimes even exactly from a limited number
of nonadaptive random measurements. In effect, the talk introduces a
theory suggesting ``the possibility of compressed data acquisition
protocols which perform as if it were possible to directly acquire
just the important information about the image of interest.'' In other
words, by collecting a comparably small number of measurements rather
than pixel values, one could in principle reconstruct an image with
essentially the same resolution as that one would obtain by measuring
all the pixels, a phenomenon with far reaching implications.The reconstruction algorithms are very concrete, stable (in the sense
that they degrade smoothly as the noise level increases) and
practical; in fact, they only involve solving convenient convex
optimization programs.
Ronald Raphael Coifman (Yale University)
Integration of intrinsic geometries of data into the sensing and processing streams
Compressive sampling is an emerging field based on the revelation that a
small collection of linear projections of a sparse signal contains enough
information for reconstruction. We introduce a new theory for distributed
compressive sampling (DCS) that enables new distributed coding algorithms
for multi-signal ensembles that exploit both intra- and inter-signal
correlation structures, which are prevalent in sensor networks. The DCS
theory rests on a new concept that we term the joint sparsity of a
signal
ensemble. We study in detail three simple models for jointly sparse
signals,
propose algorithms for joint recovery of multiple signals from
incoherent
projections, and establish upper and lower bounds on the measurement
rates required for encoding such signals.
This is joint work with
Shriram Sarvotham, Dror Baron, Michael Wakin and Richard Baraniuk.
One can trade off complexity in imaging system hardware and strict
system tolerances for complexity in post-detection data processing.
We describe an extreme example in this trade-off space: a coherent
imaging approach that eliminates the imaging system hardware entirely
(except for the detector array), relying on a phase retrieval
algorithm to form the image in a computer. It has applications
ranging from long-range imaging, such as ballistic missile defense,
to microscopy.
Michael E. Gehm (Duke University)
Tomographic hyperspectral imaging without a missing-cone
A hyperspectral imager provides a 3-D data cube in which the spatial
information (2-D) of an image is complemented by spectral information
(1-D) about each spatial location. In a tomographic hyperspectral
imager, the full 3-D data cube is reconstructed from a series of 2-D
projections. In current systems, the projections are taken over a range
of angles with respect to (and about) the wavelength-axis. Because a
full range of angles cannot be measured, the system is undersampled in
the Fourier domain (the so-called "missing cone"). We propose a new
technique that is based upon our static, Hadamard-coded spectrometer.
Using aperture coding allows us to measure projections through the data
cube that are perpendicular to the wavelength-axis. With these
projections, we can sample the a full angular range and avoid a "missing
cone". As a result, the quality of the tomographic reconstruction should
be significantly improved. We have constructed some preliminary
instruments that provide proof-of-concept, and have begun working on
more robust implementations.
Self-localization in wireless sensor networks via manifold learning
Given a set of noisy pair-wise measurements in a wireless network of N sensors the self-localization problem is to estimate all N
coordinates. We present a manifold learning approach to this problem using measured connectivity (within range or out-of-range) or
received signal strength (RSS) btwn pairs of sensors. The advantage of such approaches is that they use dimensionality reduction
to find robust estimates of the sensor locations without requireing a specific propagation model for the medium. Two methods are
presented and compared: Distributed weighted multi-dimensional scaling (dwMDS) for range-based localization (J.A. Costa, N.
Patwari, A.O. Hero, Distributed Weighted Multidimensional Scaling for Node Localization in Sensor Networks, ACM Journal on Sensor
Networks, 2006) and Laplacian Eigenmaps (LE) for connectivity-based localization (N. Patwari and A.O. Hero Adaptive Neighborhoods
for Manifold Learning-based Sensor Localization, IEEE SPAWC 05, June 2005).
Dimensionality reduction methods have played a central role in exploration of parsimonious structural models, complexity regularization
in inverse problems, and data compression. Examples are PCA, Laplacian eigenmaps, ISOMAP, and matching pursuits which attempt to fit a
subspace to the data. For integrated sensing and processing (ISP) systems dimensionality reduction must go beyond simply fitting the data
geometry. One must account for how dimension reduction will affect the performance of the processing task, e.g., image reconstruction or
classification, and sensor scheduling,
e.g., path planning or waveform design. We will present some thoughts and recent
progress on dimensionality reduction for ISP.
The human brain contains about 100 billion neurons, each making about 1000 synaptic connections with other neurons. This huge dynamical system communicates with itself and its environment via electrical impulses called spikes. How is incoming information turned into spikes, and how do spikes create decisions and behaviors? I will show how mathematics helps us model and analyze such questions, involving events from single neural spikes to decisions that change our lives.
Seongjai Kim (Mississippi State University)
The Method of Nonflat Time Evolution (MONTE)
in PDE-based image restoration
This article is concerned with effective numerical techniques for
partial differential equation (PDE)-based image restoration.
Numerical realizations of most PDE-based denoising models show a common
drawback: loss of fine structures.
In order to overcome the drawback, the article introduces a new
time-stepping procedure, called the method of nonflat time evolution
(MONTE), in which the timestep size is determined based on local image
characteristics such as the curvature or the diffusion magnitude.
The MONTE provides the PDE-based restoration models with an effective
mechanism for the equalization of the net diffusion over a wide
range of image frequency components.
It can be easily applied to diverse evolutionary PDE-based restoration
models and their spatial and temporal discretizations.
It has been numerically verified that the MONTE results in a significant
reduction in nonphysical dissipation and preserves fine structures such
as edges and textures satisfactorily, while it removes the noise with
an improved efficiency.
Various numerical results are shown to confirm the claim.
Stephane Lafon (Google)
Data fusion and multi-cue data matching using diffusion maps
Data fusion and multi-cue data matching are fundamental tasks arising
in a variety of systems that process large amounts of data. These
tasks often rely on dimensionality reduction techniques that
traditionally follow a data acquisition/reprocessing phase.
In this talk, I will describe a powerful framework based on diffusions
that can be used in order to learn the intrinsic geometry of data
sets. These techniques allow to simultaneously handle data acquisition
issues and data processing tasks. In particular, I will explain how we
can use this set of tools in order to address three major challenges
related to data fusion:
1) How to deal with data coming from sensors/sources sampled at
different rates, and possibly at different times. We provide
algorithms to obtain density-invariant descriptors (parametrization)
of data sets.2) How to integrate and combine information streams coming from
different sensors into one representation of the data. The diffusion
coordinates allow to learn the geometry of the data captured by each
sensor independently, and then to combine the various representations
into a unified description of the data.3) How to do matching of data sets based on their intrinsic geometry.
As an illustration, I will present numerical results on the
integration of audio and video streams for lip-reading and speech
recognition. Other examples will be more focused on imaging
(multiscale data-driven image segmentation, image data sets
alignment).This is joint work with R.R. Coifman, A. Glaser, Y. Keller and S.W.
Zucker (Yale university).
Hstau Liao (University of Minnesota Twin Cities)
Direct reconstruction-segmentation, as motivated by electron microscopy
Quite often in electron microscopy it is desired to segment the
reconstructed volumes of biological macromolecules. Knowledge of the 3D
structure of the molecules can be crucial for the understanding of their
biological functions. We propose approaches that directly produce a label
(segmented) image from the tomograms (projections).
Knowing that there are only a finitely many possible labels and by
postulating a Gibbs prior on the underlying distribution of label images, we
show that it is possible to recover the unknown image from only a few noisy
projections.
Joint work with Gabor T. Herman.
Russell Luke (University of Delaware)
A new generation of iterative transform algorithms for phase contrast tomography
In recent years, improvements in electromagnetic sources,
detectors,optical components, and computational imaging have made it
possible to achieve three-dimensional atomic-scale resolution using
tomographic phase-contrast imaging techniques. These greater capabilities
have placed a premium on improving the efficiency and stability of phase
retrieval algorithms for recovering the missing phase information in
diffraction observations. In some cases, so called direct methods suffice,
but for large macromolecules and nonperiodic structures one must
rely on numerical techniques for reconstructing the missing phase.
This is the principal motivation of our work.
We report on recent progress in algorithms for iterative
phase retrieval. The theory of convex optimisation
is used to develop and to gain insight into counterparts for the
nonconvex problem of phase retrieval. We propose a relaxation ofaveraged alternating reflectors and determine the fundamental mathematical
properties of the related operator in the convex case. Numerical studies
support our theoretical observations and demonstrate the
effectiveness of the newer generation of algorithms
compared to the current state of the art.
Robert Muise (Lockheed Martin)
Target detection using integrated hyper spectral sensing and processing
The concept of Integrated Sensing and Processing (ISP) suggests that a
sensor should collect data in a manner that is consistent with the end
objective. Thus ISP seeks to minimize the collection of redundant data,
reduce processing time and improve overall performance. We present a
case study of ISP using a multi-spectral camera which allows the spatial
resolution of the data to be varied in addition to selecting spectral
bands that enable the detection of targets in background clutter. This
paper is a preliminary description of work in progress, and illustrates
the basic concepts by means of several examples.
Adaptive sampling, also called ``Active Learning'', uses information
gleaned from previous measurements (e.g., feedback) to guide and focus
the sampling process. Theoretical and experimental results have shown
that adaptive sampling can dramatically outperform conventional
non-adaptive sampling schemes. I will review some of the most
encouraging theoretical results to date, and focus on new results
regarding the capabilities of adaptive sampling methods for learning
piecewise smooth functions. I will also contrast adaptive sampling
with a new approach known as compressive sampling. Compressive
sampling, or ``Compressed Sensing'', has generated a tremendous amount
of excitement in the signal processing community and is seen as a
strong competitor of adaptive procedures. Compressive sampling
involves taking a relatively small number of non-traditional samples
in the form of non-adaptive randomized projections that are capable of
capturing the most salient information in a signal. I will compare
adaptive and compressive sampling in noisy conditions, and show that
in certain interesting cases both schemes are near-optimal. This
result is remarkable since it is the first evidence of cases in which
compressive sampling, which is non-adaptive, cannot be significantly
outperformed by adaptive procedures, even in presence of noise.
This is joint work with Rui Castro and Jarvis Haupt.
Processing information at the sensor level can not only help reduce the
amount of data acquired but also enhance the overall performance of the
system. In this talk I will first present methods for shaping the
three-dimensional response of an optical system. Then I will discuss how
to integrate tailored optical responses with digital postprocessing
algorithms to improve specific imaging tasks.
Carey E. Priebe (Johns Hopkins University)
On the role of the conditionality principle in dimensionality reduction
The idea of classifier construction via `Iterative Denoising' trees---
that is, by successively partitioning (at the internal nodes of the tree)
a class-labeled training data set into ever-more homogeneous subsets
without consideration of class labels, and only subsequently (at the
leaves of the tree) using the available class-label information, while at
each node (internal or leaf) choosing a dimensionality reduction
appropriate to and specific to (the data falling in) that partition cell — may seem counter-intuitive but is in fact in (rough) accordance with
Fisher's conditionality principle and can in fact provide performance
superior to that of competing approaches.
We describe the theory and application of these `Iterative Denoising'
trees and illustrate their performance, and relate the ideas to
`Integrated Sensing and Processing' and theorized thalamocortical brain
circuit computation.
Sub-Pixel Image Registration and Quantitative Parameter Extraction
Registration methods are routinely used to automatically align images
before quantitative parameters can be estimated from them. With many
authors claiming sub-pixel accuracy in their registration procedures
we show that the operations necessary for producing a series of
images aligned to sub-pixel accuracy can significantly alter the
statistical properties of the images. This, in turn, will have a
significant effect on any quantitative parameters extracted from
registered images. We look at the stochastic properties of B-spline
interpolating basis functions of arbitrary degree and suggest means
through which they can be used in quantitative parameter extraction
from registered images. This study demonstrates by example the
importance of integrating the stochastic properties of image sensors
into image processing operations.
In this talk I will first show how to use simple
and classical results from distance geometry to
address the problem of sensor localization under
physical constraints.
Then I will move into presenting some recent results
in video processing that I wish could be done at the sensor
level. For example, I will show techniques that
reduce the video data to only regions of interest.
This will tremendously reduce transmission cost if done
at the sensors level.
The work is in collaboration with members of my group
(M. Mahmoudi and K. Patwardhan) and also in part
with Honeywell (V. Morellas).
Nitesh Shah (Raytheon Company)
Dimensionality reduction and divergence estimation for
polarization-resolution trade in SAR images
In the Integrated Sensing and Processing paradigm, agile sensors operating
in a setting of limited power, bandwidth, etc. can receive feedback
regarding sensor settings for subsequent data collection (sensor
scheduling). In the system design phase, trades are often conducted
balancing utility of different sensor settings or sensor configurations.
Information-theoretic tools are useful for assisting in evaluating
information gain in both of these settings. In particular, given fully
polarimetric, measured synthetic aperture radar (SAR) images of two
targets, we apply two dimensionality reduction techniques and a
non-density-based divergence estimation approach to evaluate the relative
target divergences over differences in effective spatial resolution and in
the number of available polarization states. Divergence at the
pre-classifier stage serves as a surrogate for target separability in an
Automatic Target Recognition (ATR) setting, avoiding the extra layer of
complexity induced by the choice of classifier and classifier parameter
settings.
Kamil Ugurbil (University of Minnesota Twin Cities)
Imaging brain activity and chemistry using high magnetic fields
We will present some of our work in hyperspectral image processing. Two
projects will be presented. In the first, we will focus on multiscale
representation and the use of PDE methods for image representation. In
the second, we will present the use of Positive Matrix Factorization for
hyperspectral unmixing.
Brani Vidakovic (Georgia Institute of Technology)
Wavelets in biomedical data analysis:
Scaling and functional design in applications
Measured bioresponses are often characterized by
an intrinsic high frequency and strong persistent correlations
inhibiting statistical modeling by the traditional techniques.
The talk overviews two novel wavelet-based techniques
for modeling such challenging data.
Wavelet domains provide natural modeling
environments for data that scale, as well as for data
consisting of continuous n-dimensional functions.
We briefly discuss technicalities and describe in detail
two applications.
First application deals with wavelet analysis of
functional ANOVA (FANOVA) where the observations
are curves coming from clinical research.
The second application discusses classification methods
based on wavelet-based measures of irregular scaling
(multifractal spectra) applied on high frequency pupillary
responses for patients with various eye pathologies.
Rebecca Willett (Duke University)
Smaller Infrared Cameras via Superresolution Image Reconstruction
Infrared camera systems can be make dramatically smaller by
simultaneously collecting several low-resolution images with
multiple narrow aperture lenses rather than collecting a
single
high- resolution image with one wide aperture lens. In this
poster, we will describe this new infrared sensing system
and a
multiscale approach to processing the output of these novel
sensors. The camera uses a 3x3 lenslet array having an
effective
focal length of 1.9 mm and we achieve image resolution
comparable
to a conventional single lens system having a focal length
of
21mm, although the image dynamic range and linearity are
reduced.
The wavelet- based regularization utilized during image
reconstruction reduces the appearance of artifacts while
preserving key features such as edges and singularities. The
processing method is very fast, making the integrated
sensing and
processing viable for both time- sensitive applications and
massive collections of sensor outputs.
Infrared camera systems can be make dramatically smaller by
simultaneously collecting several low-resolution images with
multiple narrow aperture lenses rather than collecting a
single
high- resolution image with one wide aperture lens. In this
poster, we will describe this new infrared sensing system
and a
multiscale approach to processing the output of these novel
sensors. The camera uses a 3x3 lenslet array having an
effective
focal length of 1.9 mm and we achieve image resolution
comparable
to a conventional single lens system having a focal length
of
21mm, although the image dynamic range and linearity are
reduced.
The wavelet- based regularization utilized during image
reconstruction reduces the appearance of artifacts while
preserving key features such as edges and singularities. The
processing method is very fast, making the integrated
sensing and
processing viable for both time- sensitive applications and
massive collections of sensor outputs.
This is joint work with David Brady, Mohan Shankar and
Andrew
Portnoy.
The representation of color for perception differs from the
representation
for displays. We consider the map from (r,g,b)-space to
(Intensity, hue, saturation)-space,
and show how this non-linear space is useful for sensing
devices.
-Minimax wavelet shrinkage a robust incorporation of information about energy of a signal in denoising applications (pdf)