A Framework for Depth Video Reconstruction from a Subset of Samples and Its Applications
The growth of depth sensing systems in this decade have facilitated a variety of applications in computer vision. Depending on the systematic configurations,
both direct and indirect sensing techniques encounter image processing issues, such as hole filling, depth map super resolution. In this paper, a framework for depth video reconstruction from a subset of samples is proposed.
By redefining classical dense depth estimation into two individual problems, sensing and synthesis, we propose a motion compensation assisted sampling (MCAS) scheme and a spatio-temporal depth reconstruction (STDR) algorithm
for reconstructing depth video sequences from a subset of samples. Using the 3-dimensional extensible dictionary, 3D-DWT, and applying alternating direction method of multiplier technique, the proposed STDR algorithm possesses
scalability for temporal volume and efficiency for processing large scale depth data. Exploiting the temporal information and corresponding RGB images, the proposed MCAS scheme achieves an efficient 1-Stage sampling. Experimental
results show that the proposed depth reconstruction framework outperforms the existing methods and is competitive compared to our previous work, which requires a pilot signal in the 2-Stage sampling scheme. Finally, to estimate
missing reliable depth samples from varying input sources, we present an inference approach using geometrical and color similarities. Applications for depth video super resolution from uniform-grid subsampled data and dense disparity
video estimation from a subset of reliable samples are presented.
 L.-K, Liu, S.H. Chan, and T.Q. Nguyen, "Depth reconstruction from sparse samples: Representation, algorithm, and sampling," IEEE Trans. on Image
Process., vol. 24, no. 6, pp. 1983-1996, Jun. 2015.
 D. Ferstl, C. Reinbacher, R. Ranftl, M. Ruether, and H. Bischof "Image guided depth upsampling using anisotropic total generalized variation," in Proceed. IEEE Int. Conf. on Computer Vision,
Dec. 2013, pp. 993-1000.
 S.H. Chan, D.T. Vo, and T.Q. Nguyen"Subpixel motion estimation without interpolation," in IEEE Int. Conf. on Acoustics Speech and Signal Process., Mar. 2010, pp. 722-725,
Dec. 2013, pp. 993-1000.
 K. He, J. Sun, and X. Tang,"Guided image filtering," IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 35, no. 6, pp. 1397–1409, Jun. 2013.
 S. Hawe, M. Kleinsteuber, and K. Diepold, "Dense disparity maps from sparse disparity measurements," in Proceed. IEEE Int. Conf. Computer Vision (ICCV'11), Nov. 2011, pp. 2126-2133.