Image and Video Restoration for Spatially Variant Blur
Stanley H. Chan and Truong Q. Nguyen
Introduction

Spatially variant blur is a type of distortion where different pixels in an image are blurred differently. Spatially variant blur are difficult in several aspects. First, in terms of computational speed, spatially variant problems are difficult to be solved efficiently, as the matrix-vector multiplication of a spatially variant blur in general cannot be handled using Fourier Transforms. Second, in terms modeling, the image formation model of a spatially variant blur (out-of-focus blur) is depth dependent. Thus, the classical model that expresses a spatially variant matrix as a sum of invariant matrices is not accurate. The goal of this project is to:

Publication
A. Constructing Spatially Variant Convolution Matrices

Convolution matrix is a block-circulant matrix characterized by the underlying point spread functions (PSF). In classical image restoration problems where the point spread function is spatially invariant, the convolution matrix can be constructed using the toeplitz structure. However, if the blur is spatially variant (as in most real image restoration problems), convolution matrix becomes difficult to be constructed.

In this project, we propose an efficient method to construct a spatially variant convolution matrix. We exploit the submatrix structure of the convolution matrix and systematically assigning values to the nonzero locations. For small to medium sized images, the convolution matrix gives superior speed than some state-of-art convolution operators.

shift hq 1

Figure 1: Examples of spatially variant blur. In this figure, each image is blurred by a spatially variant point spread function characterized by the underlying motion vector field. This simulation of the motion blur is based on the concept of spatio-temporal equivalence. See [Chan 2010] for detailed discussion.

B. Single Image Spatially Variant Out-of-focus Blur Removal

This paper considers an out-of-focus blur problem in which the front ground object is in focus whereas the background scene is out of focus. To recover the details of the background scene, a spatial variant blind deconvolution must be solved. However, spatial variant deconvolution is computationally intensive because Fourier-based methods cannot be used to handle spatial variant convolution operators. The proposed method exploits the invariant structure of the problem by first predicting the background. Then a blind deconvolution algorithm is applied to estimate the blur kernel and an coarse estimate of the image as a side product. Finally, the background is recovered using total variation minimization, which is then fused with the foreground to produce the final deblurred image.

Project Page: Here

Input
Lucy-Richardson [1], [2]
Spatially variant deconvtv [3]+[4]
Proposed Method
Figure 2: Results of the proposed methods compared to other methods. More results can be found in our project web page .
C. Spatially Variant Motion Blur Removal for Videos

This is an application of deconvtv. In this problem, we captured real videos that consist of a moving foreground object and stationary background. With the aid of frame rate up conversion algorithms, we process the blurred video using space-time total variation minimization.

Project Page: Here

Input
[Shechtman, Caspi and Irani 2005]
deconvtv
Figure 3: Motion blur recovery of the proposed methods compared to other methods on a real video sequence. More results can be found in our project web page .

Acknowledgement

Valid XHTML 1.0 Transitional