Projects per year
Learning object-centric representations of multi-object scenes is a promising approach towards machine intelligence, facilitating high-level reasoning and control from visual sensory data. However, current approaches for unsupervised object-centric scene representation are incapable of aggregating information from multiple observations of a scene. As a result, these “single-view” methods form their representations of a 3D scene based only on a single 2D observation (view). Naturally, this leads to several inaccuracies, with these methods falling victim to single-view spatial ambiguities. To address this, we propose The Multi-View and Multi-Object Network (MulMON)—a method for learning accurate, object-centric representations of multi-object scenes by leveraging multiple views. In order to sidestep the main technical difficulty of the multi-object-multi-view scenario—maintaining object correspondences across views—MulMON iteratively updates the latent object representations for a scene over multiple views. To ensure that these iterative updates do indeed aggregate spatial information to form a complete 3D scene understanding, MulMON is asked to predict the appearance of the scene from novel viewpoints during training. Through experiments we show that MulMON better-resolves spatial ambiguities than single-view methods—learning more accurate and disentangled object representations—and also achieves new functionality in predicting object segmentations for novel viewpoints. Our implementation and pretrained models are given on GitHub.
|Title of host publication||Advances in Neural Information Processing Systems 33 (NeurIPS 2020)|
|Publisher||Curran Associates Inc|
|Number of pages||11|
|Publication status||Published - 6 Dec 2020|
|Event||Thirty-fourth Conference on Neural Information Processing Systems - Virtual Conference|
Duration: 6 Dec 2020 → 12 Dec 2020
|Name||Advances in Neural Information Processing Systems|
|Conference||Thirty-fourth Conference on Neural Information Processing Systems|
|Abbreviated title||NeurIPS 2020|
|Period||6/12/20 → 12/12/20|
FingerprintDive into the research topics of 'Learning Object-Centric Representations of Multi-Object Scenes from Multiple Views'. Together they form a unique fingerprint.
- 1 Finished
1/01/16 → 31/12/19