Poster Session and Reception
Tuesday, October 15, 2019 - 4:15pm - 6:00pm
- Generative Models for Low-rank Video Representation and Reconstruction from Compressive Measurements
Salman Asif (University of California, Riverside)
Generative models have recently received considerable attention in the field of compressive sensing. If an image belongs to the range of a pretrained generative network, we can recover it from its compressive measurements by estimating the underlying compact latent code. In practice, all the pretrained generators have certain range beyond which they fail to generate reliably. Recent researches show that convolutional generative structures are biased to generate natural images. Based on this hypothesis, we propose joint optimization of latent codes and the weights of the generative network in compressive sensing. The main advantage of this method is that we no longer need a pretrained generator as we are optimizing weights of the network. Furthermore, we are getting compact representation of each image from latent code optimization. We empirically demonstrate that our proposed method provides better or comparable accuracy and low complexity compared to the existing methods on different video compressive sensing problems.
- Blind Unitary Transform Learning for
Inverse Problems in Light-field Imaging
Cameron Blocker (University of Michigan)
We propose using blind unitary transform learning (UTL) as a regularizer for underdetermined inverse problems in light-field imaging. UTL attempts to learn a set of filters that maximize the sparsity of the encoded representation. This work investigates which dimensions of a light-field are most sparsifiable by UTL and lead to the best reconstruction performance. We apply the UTL regularizer to light-field inpainting and focal stack reconstruction problems and find that it improves performance over traditional hand-crafted regularizers.
- 3D Variable-density SPARKLING Trajectories for High-resolution T2*-weighted Magnetic Resonance Imaging
Philippe Ciuciu (Commissariat à l'Énergie Atomique Saclay (CEA))
We have recently proposed a new optimization algorithm called SPARKLING (Spreading Projection Algorithm for Rapid K-space sampLING) to design efficient Compressive Sampling patterns for Magnetic Resonance Imaging. This method has a few advantages over conventional non-Cartesian trajectories such as radial lines or spirals: i) it allows to sample the k-space along any arbitrary density while the other two are restricted to radial densities and ii) it achieves a higher image quality for a given readout time. Here, we introduce an extension of the SPARKLING method for 3D imaging by considering both stacks-of-SPARKLING and fully 3D SPARKLING trajectories. Our method allowed to achieve an isotropic resolution of 600µm in just 45 seconds, compared to a scan duration of 14min31s using 4-fold accelerated parallel imaging, for T2*-weighted ex vivo brain imaging at 7 Tesla over a field-of-view of 200 x 200 x 140 mm3.
- Compressive classification from single pixel measurements via deep learning
Claudia Correa (Universidad Industrial de Santander)
Single-pixel camera (SPC) is a low-cost compressive imaging architecture that obtains random projections of the scenes us- ing binary coded apertures. After the acquisitions, image reconstructions are usually obtained by nonlinear and relatively expensive optimization-based algorithms. Recent works have focused on designing the binary coded apertures to improve the speed of the reconstruction algorithms and the sampling complexity of the compressed sensing systems. However, it has been shown that image recovery is not necessary for image classification from compressive measurements, where only specific features of the images are required. This work proposes a deep learning approach for image classification directly from SPC measurements. In this approach, a neural network is trained to simultaneously learn the linear binary sensing matrix and the non-linear classification parameters, considering the constraints imposed by the SPC. Specifically, the first layer learns the sensing matrix, and subsequent lay- ers perform the classification directly on the compressed measurements. Simulation results from two image datasets validate the proposed method, which provides the best classification accuracy along with a binary sensing matrix.
- An Open-source X-ray and CT Simulation Tool for Applications in Cancer Imaging
Bruno De Man (GE Global Research)
Medical imaging plays a crucial role in cancer detection, diagnosis, staging, treatment planning and therapy response monitoring. We started a project to develop and disseminate a realistic X-ray-based Cancer Imaging Simulation Toolkit (XCIST). This project builds on prior software development by Duke University, University of Massachusetts Lowell and GE Research. Drawing on each team's specific capabilities, we will create an integrated software toolkit hosted in an open-source environment. The toolkit will include GE Research's X-ray and CT imaging models and dose estimation models, Duke University's realistic digital representations of patients and tumors, and the University of Massachusetts Lowell's state-of-the-art image reconstruction implementations.
- Wavelet-Domain Low-Rank/Group-Sparse Destriping for Hyperspectral Imagery
Jim Fowler (Mississippi State University)
Pushbroom acquisition of hyperspectral imagery is prone to striping artifacts in the along-track direction. A hyperspectral destriping algorithm is proposed such that subbands of a 3D wavelet transform most effected by pushbroom stripes---namely, those with spatially vertical orientation---are the exclusive focus of destriping. The proposed method features an iterative image decomposition composed of a low-rank model for the stripes coupled with a group-sparse prior on the wavelet coefficients of the subbands in question. While low-rank stripe models have been widely used in the past, they typically have been deployed in conjunction with a total-variation prior on the image which is prone to over-smoothing and residual stripe artifacts. On the other hand, the proposed group-sparse prior not only captures the well-known sparse nature of wavelet coefficients but also capitalizes on their vertical clustering in the subbands in question. Additionally, while many prior destriping methods are wavelet-based, they employ 2D transforms band by band. In contrast, the proposed 3D wavelet transform provides greater concentration of stripe information into fewer wavelet coefficients, leading to more effective destriping. Experimental results on both synthetically striped imagery as well as real striped imagery from an actual hyperspectral sensor demonstrate superior image quality for the proposed method as compared to other state-of-the-art methods.
- Optimization of a moving colored coded aperture in compressive spectral imaging
Laura Galvis (Universidad Industrial de Santander)
Coded aperture compressive spectral imagers allow sensing a three-dimensional (3D) data cube by using two-dimensional (2D) projections of the coded and spectrally dispersed source. The traditional block-unblock coded apertures have been recently replaced by patterned optical filter arrays, allowing to modulate the spatial and spectral information. The real implementation of these patterned or “colored” coded apertures in terms of cost and complexity, directly depends on the number of filters to be used as well as the number of snapshots to be captured. This paper introduces a coded aperture optimization having in consideration these restrictions, the final design obtained is a moving colored coded aperture, which improves the reconstruction quality of the data cube and is physically implementable. Simulations show the accuracy and performance achieved with the proposed approach yielding up to 3 dB gain in PSNR over the current literature designs.
- Integrating Data and Image Domain Deep Learning for Limited Angle Tomography using Consensus Equilibrium
Muhammad Usman Ghani (Boston University)
Computed Tomography (CT) is a non-invasive imaging modality with applications ranging from healthcare to security. It reconstructs cross-sectional images of an object using a collection of projection data collected at different angles. Conventional methods, such as FBP, require that the projection data be uniformly acquired over the complete angular range. In some applications, it is not possible to acquire such data. Security is one such domain where non-rotational scanning configurations are being developed which violate the complete data assumption. Conventional methods produce images from such data that are filled with artifacts. The recent success of deep learning (DL) methods has inspired researchers to post-process these artifact laden images using deep neural networks (DNNs). This approach has seen limited success on real CT problems. Another approach has been to pre-process the incomplete data using DNNs aiming to avoid the creation of artifacts altogether. Due to imperfections in the learning process, this approach can still leave perceptible residual artifacts. In this work, we aim to combine the power of deep learning in both the data and image domains through a two-step process based on the consensus equilibrium (CE) framework. Specifically, we use conditional generative adversarial networks (cGANs) in both the data and the image domain for enhanced performance and efficient computation and combine them through a consensus process. We demonstrate the effectiveness of our approach on a real security CT dataset for a challenging 90 degree limited-angle problem. The same framework can be applied to other limited data problems arising in applications such as electron microscopy, non-destructive evaluation, and medical imaging.
- Learning Deep Patchwise Regularizers For Iterative Inverse Problem Solvers
Davis Gilton (University of Wisconsin, Madison)
Many modern approaches to image reconstruction are based on learning a regularizer that implicitly encodes a prior over the space of images. For large-scale images common in imaging domains like remote sensing, medical imaging, astronomy, and others, learning the entire image prior requires an often-impractical amount of training data. This work describes a deep image patch-based regularization approach that can be incorporated into a variety of modern algorithms. Learning a regularizer amounts to learning the a prior for image patches, greatly reducing the dimension of the space to be learned and hence the sample complexity. Demonstrations in a remote sensing application illustrates that learning patch-based regularizers produces high-quality reconstructions and even permits learning from a single ground-truth image.
- Weighted Belief Propagation
Anna Grim (Brown University)
We introduce weighted belief propagation algorithms that con- verge to unique fixed points on graphs with arbitrary topology. The standard min-sum and max-product algorithms converge when the graph is acyclic, but may not converge or converge to different fixed points depending on the initialization when the graph contains a cycle. This paper describes an alternative to the standard belief propagation algorithms that obtain good results and are guaranteed to converge regardless of the topology of the graph.
- Manifold Model for High-Resolution OSSI fMRI Reconstruction and Quantification
Shouchang Guo (University of Michigan)
Oscillating Steady-State Imaging (OSSI) is a new fMRI acquisition method that can provide 2x higher SNR compared to the standard approach, but does so at the expense of imaging time. To improve the spatial and temporal resolutions, instead of using linear subspace models such as low-rank, we propose a manifold model that builds MRI physics for OSSI signal generation into the reconstruction. The manifold model not only accurately represents OSSI nonlinearities and enables a 12-fold acceleration with 1.3 mm isotropic spatial resolution, but also provides quantitative T2’ estimations for fMRI.
- Exploiting Light Field Spectra for Passive NLoS Imaging
Connor Hashemi (University of Minnesota, Twin Cities)
Passive non-line-of-sight (NLoS) imaging aims at reconstructing information of occluded objects by analyzing their indirect reflections off surfaces. It has many applications across military, search and rescue, medical, autonomous vehicles, and general remote sensing domains. Previously we presented a method called Differential Field-of-View (DFoV) which leverages the “corner camera” approach by utilizing the light field and the occluder to extract information of the hidden objects. However, DFoV still has issues with large background signal rejection and does not fully exploit different spectral information such as the spectral signature of different objects. In our new work, we modified the existing DFoV framework to incorporate data fusion of multi-spectra light fields and show experimental results that our new method gives more accurate reconstructions over the previous methods.
- A Convex Program for Binary Tomography
Ajinkya Kadu (Utrecht State University)
Binary tomography is concerned with the recovery of binary images from a few of their projections (i.e., sums of the pixel values along various directions). To reconstruct an image from noisy projection data, one can pose it as a constrained least-squares problem. As the constraints are non-convex, many approaches for solving it rely on either relaxing the constraints or heuristics. We propose a novel convex formulation, based on the Lagrange dual of the constrained least-squares problem. The resulting problem is a generalized LASSO problem which can be solved efficiently. It is a relaxation in the sense that it can only be guaranteed to give a feasible solution; not necessarily the optimal one. In exhaustive experiments on small images we find, however, that if the problem has a unique solution, our dual approach finds it. In the case of multiple solutions, our approach finds the commonalities between the solutions. This is joint work with Dr. Tristan van Leeuwen (Utrecht University).
- Data-driven Design for Computational Imaging
Michael Kellman (University of California, Berkeley)
Computational imaging systems marry the design of hardware and computation to create a new generation of modalities that image beyond what is currently possible. A computational imaging system’s performance is fundamentally governed by how well the sought information is encoded in (experimental design) and decoded from (computational reconstruction) the measurements. In settings where both the encoding and decoding steps are non-linear---a prominent example is phase retrieval---analytical methods that assess the system’s reconstruction performance become difficult to establish and might not necessarily result in improved system designs.
In this work, we present our method to learn aspects of the experimental design to optimize the performance of a computational imaging system. We consider unrolling the iterations of a traditional model-based image reconstruction algorithm (e.g. compressed sensing or phase retrieval) to form network whose layers are composed of gradient-descent and proximal steps, making it possible to optimize the entire imaging pipeline. As an application, we showcase how a standard microscope can be transformed to image transparent samples without staining beyond its inherent resolution-limit using phase retrieval. In particular, we demonstrate that one can drastically decrease the number of measurements needed to obtain super-resolved images by learning the design of the microscope.
- High-dimensional MRI approaches for better microstructure imaging
Daeun Kim (University of Southern California)
Multiexponential modeling of relaxation or diffusion MR signal decays is a popular approach for estimating and spatially mapping different microstructural tissue compartments. While this approach can be quite powerful, it is often limited by the fact that one-dimensional multiexponential modeling is an ill-posed inverse problem with substantial ambiguities. This work presents an overview of a recent multidimensional correlation spectroscopic imaging approach to this problem. This approach helps to alleviate ill-posedness by leveraging multidimensional contrast encoding (e.g., 2D diffusion-relaxation encoding or 2D relaxation-relaxation encoding) combined with a regularized spatial-spectral estimation technique.
- Physics Based Machine Learning for Radiographic Reconstructions
Marc Klasky (Los Alamos National Laboratory)
A physics based radiographic reconstruction algorithm is presented which incorporates both a physics based de-scattering algorithm as well as a Convolutional Neural Network. Transport simulations utilizing MCNP6 have been performed to generate synthetic radiographic data from which scattering kernels have been inferred and unknown density fields have been reconstructed. Physics arguments are presented to justify the approach and offer possible applications for which the algorithms may be successfully applied. A neural network based on the U-net was developed to learn the Inverse Mapping of transmission to density based on MCNP direct transport simulations. Results of both de-scattering and combined de-scattering/neural network performance are presented.
- Combining Supervised and semi-Blind Residual Dictionary (Super-BReD) Learning
Anish Lahiri (University of Michigan)
Regularization in inverse problems, especially in imaging, often relies upon sparse representation of signals using a linear combination of dictionary atoms. For this purpose, data-driven learning of dictionaries has shown promise in many applications compared to fixed dictionaries. In a 'blind' setting, these dictionaries are learned from the corrupt test data itself, and assumes no external knowledge of clean data. Whereas, in 'fully supervised' dictionary learning, they are learned from uncorrupted training data, and then used on corrupt test data. In this work, we attempt to combine the supervised and blind dictionary learning frameworks to learn two separate dictionaries in a residual fashion to jointly represent test signals. We expect supervised learning to compensate for lack of information when performing blind dictionary learning with highly corrupted data. Our algorithm, Super-BReD Learning, shows promising preliminary results on denoising tasks with highly corrupted image data.
- Shape Prior Metal Artifact Reduction Algorithm for Industrial 3D Cone-beam CT
Chang-Ock Lee (Korea Advanced Institute of Science and Technology (KAIST))
Metal artifact caused by beam hardening especially due to metallic objects is one of the most major factors that degrade the image quality of computed tomography (CT) and it leads misinterpretation in CT image analysis. We present a methodology to reduce the metal artifact in three dimensional industrial cone-beam CT. In order to manage the three dimensional volume data efficiently we develop a registration technique. Through numerical experiments, we verify that the proposed algorithm performs successfully.
- Convex Optimization for Low-Rank Recovery by Tensor Product Norms
Kiryung Lee (The Ohio State University)
Low-rank models have been widely used as an effective prior to solve various inverse problems in signal processing and statistics. It has been shown that convex optimization with the nuclear norm finds the desired low-rank solution. In fact, as one interprets a matrix as a linear operator between two Banach spaces, various tensor product norms generalize the role of the nuclear norm. Inspired by a recent work on the max-norm-based matrix completion, we provide a unified view on a class of tensor product norms and their interlacing relations on low-rank operators. We also derive entropy estimates on tensor products of a family of Banach space pairs and demonstrate their implication to performance guarantees for selected inverse problems. This is a jointwork with Rakshith Sharma Srinivasa, Marius Junge, and Justin Romberg.
- Multi-frame Super-resolution for time-of-flight imaging
Fengqiang Li (Northwestern University)
Recently, time-of-flight (ToF) sensors have emerged as a promising three-dimensional sensing technology that can be manufactured in- expensively in a compact size. However, current state-of-the-art ToF sensors suffer from low spatial resolution due to physical limitations in the fabrication process. In this paper, we analyze the ToF sensor’s output as a complex value coupling the depth and intensity infor- mation in a phasor representation. Based on this analysis, we intro- duce a novel multi-frame superresolution technique that can improve both spatial resolution in intensity and depth images simultaneously. We believe our proposed method can benefit numerous applications where high resolution depth sensing is desirable, such as precision automated navigation and collision avoidance.
- Autoregression and Structured Low-Rank Modeling of Sinograms
Rodrigo Lobos (University of Southern California)
The Radon transform converts an image into a sinogram, and is often used as a model of data acquisition for many tomographic imaging modalities. Although it is well-known that sinograms possess some redundancy because of their bowtie-shaped spectral support, we observe in this work that they can have substantial additional redundancies that can be learned directly from incomplete data. In particular, we demonstrate that sinograms approximately satisfy multiple data-dependent shift-invariant local autoregression relationships. This autoregressive structure implies that samples from the sinogram can be accurately interpolated as a shift-invariant linear combination of neighboring sinogram samples, and that a Toeplitz or Hankel matrix formed from sinogram data should be approximately low-rank. This multi-fold redundancy can be used to impute missing sinogram values or for noise reduction, as we demonstrate with real X-ray CT data.
- Analyzer-free tesor ptychography of dichroic materials
Stefano Marchesini (Lawrence Berkeley National Laboratory)
Linear x-ray dichroism is an important tool to characterize the transmission matrix and determine the nano-crystal or orbital orientation in a material such as enamel. In order to gain high resolution mapping of the transmission properties of such materials, we introduce the linear-dichroism scattering model in ptychographic imaging, and then develop an efficient two-stage reconstruction algorithm. The proposed algorithm can recover the dichroic transmission matrix without a polarizer by using ptychography measurements with as few as three different polarization angles, with the help of an empty region to remove phase ambiguities.
- Learning Regularization Filters for Image Reconstruction
Michael McCann (Michigan State University)
We present early results on learning regularization filters to optimize image reconstruction performance on training data. Image reconstruction is the process of recovering a image from its noisy measurements. Many image reconstruction algorithms consist of minimizing a cost functional that consists of a data term, which promotes reconstructions that explain the measurements, and a regularization term, which promotes plausible reconstructions; regularization terms often involve applying a linear filter to the reconstruction. The idea here is to learn these filters from training data via a bilevel optimization—picking regularization filters so that variational reconstruction of training measurements gives reconstructions that are close to the ground truth. The results of a simple denoising experiment show that learned filters can outperform fixed ones and that they can easily generalize to unseen images and noise levels.
- Block Axial Checkerboarding (BAC): A Distributed Algorithm for Helical X-ray CT
Naveen Murthy (University of Michigan)
Model-Based Iterative Reconstruction (MBIR) methods for X-ray CT provide improved image quality compared to conventional techniques like filtered backprojection (FBP), but their computational burden is undesirably high. Distributed algorithms have the potential to significantly reduce reconstruction time, but the communication overhead of existing methods has been a considerable bottleneck. This work proposes a distributed algorithm called Block-Axial Checkerboarding (BAC) that utilizes the special structure found in helical CT geometry to reduce inter-node communication. Preliminary results using a simulated 3D helical CT scan suggest that the proposed algorithm has the potential to reduce reconstruction time in multi-node systems, depending on the balance between compute speed and communication bandwidth.
- Machine Learning in Computational Medical Imaging: A Few Research Examples
Mariappan Nadar (Siemens Healthineers)
In this poster, we outline a few research examples in the application of machine learning in computational medical imaging being investigated in the Digital Technology and Innovation team at Siemens Healthineers
- Dual Energy CT for Improved Stopping Power Estimation in Proton Therapy
Joseph O'Sullivan (Washington University)
Proton radiotherapy has good conformal dose distribution. Currently, an additional 2-3.5% safety margin must be added to the proton range due to uncertainties in the estimation of proton-beam range from using single energy CT to recover stopping power ratio using a calibration procedure. To reduce the underlying uncertainty in proton beam range estimation, our lab has developed a joint statistical image reconstruction method built on a basis vector model which was shown to perform comparatively better than other dual energy CT methods under ideal conditions. For implementing this approach, our group has developed a robust statistical characterization of the received signal and explored methods to determine energy-fluence spectra, wedge profile, and background events. We provide an overview of the progress in characterizing the statistical and physical behavior of the received transmission signal and the most recent results in estimating the proton beam range using our method.
- Neumann Networks for Linear Inverse Problems in Imaging
Greg Ongie (University of Chicago)
We present an end-to-end, data-driven method of solving inverse problems inspired by the Neumann series, which we call a Neumann network. Rather than unroll an iterative optimization algorithm, we truncate a Neumann series which directly solves the lin- ear inverse problem with a data-driven nonlinear regularizer. The Neumann network architecture outperforms traditional inverse problem solution methods, model-free deep learning approaches, and state-of-the-art unrolled iterative methods on standard datasets. Finally, when the images belong to a union of subspaces and under appropriate assumptions on the forward model, we prove there exists a Neumann network configuration that well-approximates the optimal oracle estimator for the inverse problem and demonstrate empirically that the trained Neumann network has the form predicted by theory.
- Tomography Reconstruction using Randomized Newton Method with Regularization by Denoising
Alessandro Perelli (Technical University of Denmark)
Second order methods for solving regularized optimization problems with generalized linear models have been widely studied but despite the superior convergence rate compared to first order methods one weakness relies on the computational cumbersome for calculating the Hessian matrix. Additionally, in imaging applications where the input prior is difficult to model, powerful regularization techniques are based on data-driven models or denoisers.
In this work, we develop an efficient and accurate randomized second order method for model based Computed Tomography (CT) reconstruction. The algorithm combines the idea of dimensionality reduction of the Hessian for the log-likelihood loss function by sketching and an explicit regularizer term which can be implemented by a generic denoiser through the score matching formulation. We consider the cost function composed by a pixel-wise separable convex surrogate and a data-driven prior regularizer. We propose to reduce the computational complexity using a partial randomized Hessian sketching, using the ridge leverage scores, only for the convex likelihood function and the regularization is designed using the denoising score matching framework which retains the complex prior structure. Finally, we show how to compute the gradient and the Hessian of the likelihood and regularizer together with simulated results.
- Exact Recovery for Multichannel Sparse Blind Deconvolution via Gradient Descent
Qing Qu (New York University)
We study the multi-channel sparse blind deconvolution (MCS-BD) problem, whose task is to simultaneously recover a kernel and multiple sparse inputs from their circulant convolution. We formulate the task as a nonconvex optimization problem over the sphere. Under mild statistical assumptions of the data, we prove that the vanilla Riemannian gradient descent (RGD) method, with random initializations, provably recovers both the kernel and the signals up to a signed shift ambiguity. In comparison with state-of-the-art results, our work shows significant improvements in terms of sample complexity and computational efficiency. Our theoretical results are corroborated by numerical experiments, which demonstrate superior performance of the proposed approach over the previous methods on both synthetic and real datasets.
- Short-and-Sparse Deconvolution -- A Geometric Approach
Qing Qu (New York University)
Short-and-sparse deconvolution (SaSD) is the problem of extracting localized, recurring motifs in signals with spatial or temporal structure. Variants of this problem arise in applications such as image deblurring, microscopy, neural spike sorting, and more. The problem is challenging in both theory and practice, as natural optimization formulations are nonconvex. Moreover, practical deconvolution problems involve smooth motifs (kernels) whose spectra decay rapidly, resulting in poor conditioning and numerical challenges. This paper is motivated by recent theoretical advances, which characterize the optimization landscape of a particular nonconvex formulation of SaSD. This is used to derive a provable algorithm which exactly solves certain non-practical instances of the SaSD problem. We leverage the key ideas from this theory (sphere constraints, data-driven initialization) to develop a practical algorithm, which performs well on data arising from a range of application areas. We highlight key additional challenges posed by the ill-conditioning of real SaSD problems, and suggest heuristics (acceleration, continuation, reweighting) to mitigate them. Experiments demonstrate both the performance and generality of the proposed method.
- A Supervised-Unsupervised (SUPER) Learning Framework for Image Reconstruction
Saiprasad Ravishankar (Michigan State University)
Many problems in imaging involve reconstructing or recovering images from limited or corrupted data. For example, low-dose X-ray computed tomography (LDCT) is a well-known approach for reducing radiation exposure in clinical practice. However, reducing X-ray dose leads to undesirable artifacts in conventional image reconstructions such as using the filtered backprojection method. Iterative reconstruction methods relying on sophisticated image priors have been popular for these problems. Supervised learning-based reconstruction methods have also shown recent success in image reconstruction tasks, but often rely on large training sets. Other recent works relying on unsupervised learning-based regularizers such as using learned sparsifying transforms or dictionaries, do not often require large or paired training sets, and the learned models often have good generalization properties, since they capture general features and properties of image sets. We present a Supervised-UnsuPERvised (SUPER) reconstruction framework for image reconstruction that combines the benefits of supervised learning methods and unsupervised or iterative reconstruction methods. For example, the recent transform learning-based iterative LDCT image reconstruction method PWLS-ULTRA involves highly image-adaptive clustering. The proposed SUPER reconstruction model consists of multiple SUPER layers, each of which consists of a supervised learned network and an unsupervised or iterative method. The model allows combining different types of priors and machine learned models in a common reconstruction framework, and is learned in a greedy manner. SUPER learning is shown to outperform both the constituent supervised learning-based network and iterative algorithm for LDCT reconstruction, while using only few iterations in the iterative blocks.
- Two Novel Methods for Alignment of Lateral Jitter in Parallel-beam Nanotomography
Florian Schiffers (Northwestern University)
Recent advances in X-ray diffraction imaging allows to produce images of X-ray absorption and phase contrast with potentially sub-20 nm resolution. In theory, the resolving capability is limited only by the short wavelengths of the radiation used. However, in practice, the resolution of tomographic 3D reconstruction is limited by the accuracy of mechanical motions. For nanotomography systems there are inherent limitations in achieving the required accuracies. Automatic and robust alignment of tomographic images is essential to advance the state-of-the-art in nanoscopic 3D imaging. Our poster presents two methods to solve the alignment problem. The first one is a fast, non-iterative alignment based on geometric constraints from the Radon transform, the second one is a Bayesian approach to jointly solve the reconstruction/alignment problem within an iterative framework.
- On Non-Convex Regularization for Convex Signal Processing
Ivan Selesnick (New York University)
Some effective and systematic approaches for nonlinear signal processing are based on sparse and low-rank signal models. Often, the L1 norm (or nuclear norm) is used, but this tends to underestimate the true values. We present non-convex alternatives to the L1 norm (and nuclear norm). Unlike other non-convex regularizers, the proposed regularizer is designed to maintain the convexity of the objective function to be minimized. Thus, we can retain beneficial properties of both convex and non-convex regularization. The new regularizer can be understood in terms of a generalized Moreau envelope. We present new results applying these ideas to total variation signal denoising.
- Temporally Dense Optical Flow Prediction for Event Sensors
Prasan Shedligeri (Northwestern University)
Event sensors acquire high dynamic range temporal gradient information at a high temporal resolution in the form of events. State of the art event-sensor based optical flow algorithms require the accumulation of a large number of events over a long temporal window. They also assume a linear optical flow throughout this entire temporal window. This effectively reduces the temporal resolution of the event sensors and also decreases optical flow prediction accuracy. We propose to predict optical flow at x10 temporal resolution and naturally relax the linear optical flow assumption. We propose a semi-supervised learning algorithm to predict optical flow by warping successive intensity frames. Specifically, we use a recurrent network that takes in as input a single event frame and simultaneously predicts the intensity frame and the optical flow. To fully exploit the advantages of the event sensors, we tackle the challenging problem of simultaneously predicting optical flow and high dynamic range intensity frames from event frames alone. The intensity frame prediction is supervised by the temporally sparse raw intensity frames from the intensity sensor. The optical flow prediction is supervised by warping successive predicted intensity frames. The quantitative evaluation shows that the proposed method naturally leads to higher accuracy of the optical flow prediction. Overall, we demonstrate an algorithm to predict temporally dense and accurate optical flow from event sensors.
- High-contrast Imaging of Exoplanets: Coronagraph, Adaptive Optics and Integral Field Spectroscopy
He Sun (California Institute of Technology)
Exoplanet detection and characterization is one of the most exciting topics in astrophysics today. Since the first exoplanet (51 Pegasi b, which won 2019 Nobel Prize in Physics) was discovered in 1995, over 4000 exoplanets have been detected. However, most of these discoveries were achieved using indirect detection methods, such as wobbling and transit. Direct imaging of the exoplanets is extremely hard, because typically exoplanets are a billion times fainter than their parent stars, thus requiring an imaging system with extremely high contrast capability. This poster will briefly introduce one promising exoplanet imaging pipeline designed for NASA’s next generation space telescopes, which integrates both ingenious optical design and advanced image processing algorithms. Three sub-systems will be highlighted, the coronagraph instrument (an optical filter that blocks starlight but transmits the planet signals), the adaptive optics (a control system that estimates and corrects the wavefront aberrations), and the integral field spectrograph (an efficient broadband imager that utilizes dispersion optics and image reconstruction algorithms.)
- Sparse Representation Learning: A comparative study
Akanksha Sunalkar (University of Michigan)
Natural signals have an essentially sparse representation (few significant non-zero coefficients) in analytical transform and/or dictionary domain. This property of natural images has this is exploited in inverse imaging problems. The poster presents a comparative study with a focus on the performance of Analysis Dictionary model and the Sparsifying Transform Model. Since, sparse representation has had great success in various image processing problems, this poster looks into one of the inverse problem: Image In-painting, to compare its performance. The aim of this poster is to bring to notice the often disregarded nuances between the two models.
- Deep Feature Factorization for Content-based Image Retrieval and Localization
Sabine Susstrunk (École Polytechnique Fédérale de Lausanne (EPFL))
State of the art content-based image retrieval algorithms owe their
excellent performance to the rich semantics encoded in the deep
activations of a convolutional neural network, and differ mostly in how
activations are combined into a compact global image descriptor. We
propose to use deep feature factorization to achieve this goal. By
factorizing CNN activations, we decompose an input image into semantic
regions, represented by both spatial saliency heatmaps and local
descriptors for those regions. We then combine these local descriptors
to form a single global descriptor for retrieval. Our experiments show
this approach achieves performance on par with the state of the art,
while being simple to implement and widely applicable. In addition, DFF
heatmaps automatically localize the region of interest within the set of
retrieved images, which, as we show quantitatively, yields excellent
- Computational Imaging at Oak Ridge National Lab
Singanallur Venkatakrishnan (Oak Ridge National Laboratory)Amir Ziabari (Oak Ridge National Laboratory)
Computational imaging (CI) systems play a critical role in making scientific discoveries in diverse fields including biology, material sciences and additive manufacturing at Oak Ridge National Lab (ORNL). The first-wave of CI systems typically relied on fast algorithms to invert the measurements based on analytic inversion techniques. However, the performance of these algorithms can be poor when dealing with non-linearities in the measurement, the presence of high-levels of noise, and the limited number of measurements that commonly occur when we seek to dramatically accelerate the imaging. In this poster, we will highlight algorithms for improving the performance of different CI systems at ORNL - enabling faster, more accurate and novel tomographic imaging capabilities. Specifically, we will present examples of traditional model-based image reconstruction techniques as well as deep-learning approaches for a variety of applictions including hyper-spectral neutron tomography, single particle cryo-EM and cone-beam X-ray micro-CT systems.
- The iterative convolution-thresholding method (ICTM) for image segmentation
Dong Wang (The University of Utah)
We propose a novel iterative convolution-thresholding method (ICTM) that is applicable to a range of variational models for image segmentation. A variational model usually minimizes an energy functional consisting of a fidelity term and a regularization term. In the ICTM, the interface between two different segment domains is implicitly represented by their characteristic functions. The fidelity term is then written as a linear functional of the characteristic functions and the regularized term is approximated by a functional of characteristic functions in terms of heat kernel convolution. This allows us to design an iterative convolution-thresholding method to minimize the approximate energy. The method is simple, efficient and enjoys the energy-decaying property. Numerical experiments show that the method is easy to implement, robust and applicable to various image segmentation models.
- Event-driven video frame synthesis
Zihao Wang (Northwestern University)
emporal Video Frame Synthesis (TVFS) aims at synthesizing novel frames at timestamps different from existing frames, which has wide applications in video codec, editing and analysis. In this paper, we propose a high frame-rate TVFS framework which takes hybrid input data from a low-speed frame-based sensor and a high-speed event-based sensor. Compared to frame-based sensors, event-based sensors report brightness changes at very high speed, which may well provide useful spatio-temoral information for high frame-rate TVFS. In our framework, we first introduce a differentiable forward model to approximate the physical sensing process, fusing the two different modes of data as well as unifying a variety of TVFS tasks, i.e., interpolation, prediction and motion deblur. We leverage autodifferentiation which propagates the gradients of a loss defined on the measured data back to the latent high frame-rate video. We show results with better performance compared to state-of-the-art. Second, we develop a deep learning-based strategy to enhance the results from the first step, which we refer as a residual denoising process. Our trained denoiser is beyond Gaussian denoising and shows properties such as contrast enhancement and motion awareness. We show that our framework is capable of handling challenging scenes including both fast motion and strong occlusions.
- Multi-layer Residual Sparsifying Transform Learning Model for Low-dose CT Image Reconstruction
Xikai Yang (Shanghai Jiaotong University)
Signal models based on sparsity, low-rank and other properties have been exploited for image reconstruction from limited and corrupted data in medical imaging and other computational imaging applications. In particular, sparsifying transform models have shown promise in various applications, and offer numerous advantages such as efficiencies in sparse coding and learning. This work investigates Multi-layer Residual Sparsifying Transform learning model (MRST), wherein the transform domain or filtering residuals of the image are further sparsified layer by layer. Preliminary numerical experiments on XCAT Phantom and Mayo clinic data demonstrate the usefulness of a two-layer model over the previous related schemes for CT image reconstruction from low-dose measurements.
- Linearly Convergent Method for Non-Smooth Non-Convex Optimization on the Grassmannian with Applications to Robust Subspace and Dictionary Learning
Zhihui Zhu (Johns Hopkins University)
Minimizing a non-smooth function over the Grassmannian appears in many applications in machine learning. In this paper we show that if the objective satisfies a certain Riemannian regularity condition (RRC) with respect to some point in the Grassmannian, then a projected Riemannian subgradient method with appropriate initialization and geometrically diminishing step size converges at a linear rate to that point. We show that for both the robust subspace learning method Dual Principal Component Pursuit (DPCP) and the Orthogonal Dictionary Learning (ODL) problem, the RRC is satisfied with respect to appropriate points of interest, namely the subspace orthogonal to the sought subspace for DPCP and the orthonormal dictionary atoms for ODL. Consequently, we obtain in a unified framework significant improvements for the convergence theory of both methods. This is joint work with Tianyu Ding, Manolis Tsakiris, Daniel Robinson, and Rene Vidal.