Paper in ICCV Workshop on Geometry Meets Deep Learning Workshop on "Floors are Flat: Leveraging Semantics for Real-Time Surface Normal Prediction"

Citation

S. Hickson, K. Raveendran, Alireza Fathi, K. Murphy, and I. Essa (2019), “Floors are Flat: Leveraging Semantics for Real-Time Surface Normal Prediction,” in IEEE International Conference on Computer Vision (ICCV) Workshop on Geometry Meets Deep Learning, 2019. [PDF] [VIDEO] [arXiv] [BIBTEX]

@InProceedings{ 2019-Hickson-FFLSRSNP,
arxiv = {https://arxiv.org/abs/1906.06792},
author  = {Steven Hickson and Karthik Raveendran and Alireza
Fathi and Kevin Murphy and Irfan Essa},
booktitle  = {{IEEE International Conference on Computer Vision
(ICCV) Workshop on Geometry Meets Deep Learning}},
eprint  = {1906.06792},
howpublished  = {arXiv preprint arXiv:1906.06792},
month = {October},
pdf = {http://openaccess.thecvf.com/content_ICCVW_2019/papers/GMDL/Hickson_Floors_are_Flat_Leveraging_Semantics_for_Real-Time_Surface_Normal_Prediction_ICCVW_2019_paper.pdf},
primaryclass  = {cs.CV},
title = {Floors are Flat: Leveraging Semantics for Real-Time
Surface Normal Prediction},
video = {https://www.youtube.com/watch?v=QrXqmUBlmbc},
year = {2019}
}

Abstract

We propose 4 insights that help to significantly improve the performance of deep learning models that predict surface normals and semantic labels from a single RGB image. These insights are: (1) denoise the ”ground truth” surface normals in the training set to ensure consistency with the semantic labels; (2) concurrently train on a mix of real and synthetic data, instead of pretraining on synthetic and fine-tuning on real; (3) jointly predict normals and semantics using a shared model, but only backpropagate errors on pixels that have valid training labels; (4) slim down the model and use grayscale instead of color inputs. Despite the simplicity of these steps, we demonstrate consistently improved state of the art results on several datasets, using a model that runs at 12 fps on a standard mobile phone.