Abstract
3D scene reconstruction from a sequence of posed RGB images is a cornerstone task for computer vision and augmented reality (AR). While depth-based fusion is the foundation of most real-time approaches for 3D reconstruction, recent learning based methods that operate directly on RGB images can achieve higher quality reconstructions, but at the cost of increased runtime and memory requirements, making them unsuitable for AR applications. We propose an efficient learning-based method that refines the 3D reconstruction obtained by a traditional fusion approach. By leveraging a top-down heightfield representation, our method remains real-time while approaching the quality of other learning-based methods. Despite being a simplification, our heightfield is perfectly appropriate for robotic path planning or augmented reality character placement. We outline several innovations that push the performance beyond existing top-down prediction baselines, and we present an evaluation framework on the challenging ScanNetV2 dataset, targeting AR tasks. Ultimately, we show that our method improves over the baselines for AR applications. Full code and pretrained models will be released on acceptance.
Original language | English |
---|---|
Title of host publication | Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) |
Publisher | Institute of Electrical and Electronics Engineers |
Pages | 5850-5860 |
Number of pages | 11 |
ISBN (Electronic) | 978-1-6654-9346-8 |
ISBN (Print) | 978-1-6654-9347-5 |
DOIs | |
Publication status | Published - 6 Feb 2023 |
Event | IEEE/CVF Winter Conference on Applications of Computer Vision, 2023 - Waikoloa, United States Duration: 3 Jan 2023 → 7 Jan 2023 https://wacv2023.thecvf.com/ |
Publication series
Name | IEEE Workshop on Applications of Computer Vision (WACV) |
---|---|
Publisher | IEEE |
ISSN (Print) | 2472-6737 |
ISSN (Electronic) | 2642-9381 |
Conference
Conference | IEEE/CVF Winter Conference on Applications of Computer Vision, 2023 |
---|---|
Abbreviated title | WACV 2023 |
Country/Territory | United States |
City | Waikoloa |
Period | 3/01/23 → 7/01/23 |
Internet address |