# Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image

We consider the problem of dense depth prediction from a sparse set of depth measurements and a single RGB image. Since depth estimation from monocular images alone is inherently ambiguous and unreliable, to attain a higher level of robustness and accuracy, we introduce additional sparse depth samples, which are either acquired with a low-resolution depth sensor or computed via visual Simultaneous Localization and Mapping (SLAM) algorithms. We propose the use of a single deep regression network to learn directly from the RGB-D raw data and explore the impact of a number of depth samples on prediction accuracy. Our experiments show that, compared to using only RGB images, the addition of 100 spatially random depth samples reduces the prediction root-mean-square error by 50% on the NYU-Depth-v2 indoor dataset. It also boosts the percentage of reliable prediction from 59% to 92% on the KITTI dataset.

**Dataset:** http://www.cvlibs.net/datasets/kitti/eval_depth.php?benchmark=depth_completion

**Paper:** https://arxiv.org/abs/1709.07492

**Task:**

Just train on a small subset of KITTI.

1) Replace the feature extractor from RESNET-18 to VGGNet for KITTI

2) Use Nearest Neighbour upsampling instead of bilinear interpolation.

3) Use Uniform random sampling with depth points restricted in numbers to 20000.

Compare results with the paper.