SLAMBench 3.0: Systematic Automated Reproducible Evaluation of SLAM Systems for Robot Vision Challenges and Scene Understanding

Mihai Bujanca; Paul Gafton; Sajad Saeedi; Andy Nisbet; Bruno Bodin; Michael F. P. O'Boyle; Andrew J. Davison; Paul H. J. Kelly; Graham Riley; Barry Lennox; Mikel Luján; Steve Furber

doi:10.1109/ICRA.2019.8794369

SLAMBench 3.0: Systematic Automated Reproducible Evaluation of SLAM Systems for Robot Vision Challenges and Scene Understanding

Mihai Bujanca, Paul Gafton, Sajad Saeedi, Andy Nisbet, Bruno Bodin, Michael F. P. O'Boyle, Andrew J. Davison, Paul H. J. Kelly, Graham Riley, Barry Lennox, Mikel Luján, Steve Furber

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

As the SLAM research area matures and the number of SLAM systems available increases, the need for frameworks that can objectively evaluate them against prior work grows. This new version of SLAMBench moves beyond traditional visual SLAM, and provides new support for scene understanding and non-rigid environments (dynamic SLAM). More concretely for dynamic SLAM, SLAMBench 3.0 includes the first publicly available implementation of DynamicFusion, along with an evaluation infrastructure. In addition, we include two SLAM systems (one dense, one sparse) augmented with convolutional neural networks for scene understanding, together with datasets and appropriate metrics. Through a series of use-cases, we demonstrate the newly incorporated algorithms, visulation aids and metrics (6 new metrics, 4 newdatasets and 5 new algorithms).

Original language	English
Title of host publication	2019 International Conference on Robotics and Automation (ICRA)
Publisher	Institute of Electrical and Electronics Engineers (IEEE)
Pages	6351-6358
Number of pages	8
ISBN (Electronic)	978-1-5386-6027-0
ISBN (Print)	978-1-5386-8176-3
DOIs	https://doi.org/10.1109/ICRA.2019.8794369
Publication status	Published - 12 Aug 2019
Event	2019 IEEE International Conference on Robotics and Automation (ICRA) - Montreal, Canada Duration: 20 May 2019 → 24 May 2019 https://www.icra2019.org/

Publication series

Name
Publisher	Institute of Electrical and Electronics Engineers (IEEE)
ISSN (Print)	1050-4729
ISSN (Electronic)	2577-087X

Conference

Conference	2019 IEEE International Conference on Robotics and Automation (ICRA)
Abbreviated title	ICRA 2019
Country	Canada
City	Montreal
Period	20/05/19 → 24/05/19
Internet address	https://www.icra2019.org/

Access to Document

10.1109/ICRA.2019.8794369Licence: All Rights Reserved

SLAMBench 3.0_BUJANCA_DOA02042019_AFVAccepted author manuscript, 2.16 MB

https://spiral.imperial.ac.uk/handle/10044/1/73402
https://ieeexplore.ieee.org/document/8794369Licence: All Rights Reserved

Fingerprint Dive into the research topics of 'SLAMBench 3.0: Systematic Automated Reproducible Evaluation of SLAM Systems for Robot Vision Challenges and Scene Understanding'. Together they form a unique fingerprint.

1 Finished

PAMELA: a Panoramic Approach to the Many-CorE LAndsape - from end-user to end-device: a holistic game-changing approach
Topham, N., Franke, B. & O'Boyle, M.
EPSRC
1/03/13 → 15/10/18
Project: Research

Cite this

Bujanca, M., Gafton, P., Saeedi, S., Nisbet, A., Bodin, B., O'Boyle, M. F. P., Davison, A. J., Kelly, P. H. J., Riley, G., Lennox, B., Luján, M., & Furber, S. (2019). SLAMBench 3.0: Systematic Automated Reproducible Evaluation of SLAM Systems for Robot Vision Challenges and Scene Understanding. In 2019 International Conference on Robotics and Automation (ICRA) (pp. 6351-6358). Institute of Electrical and Electronics Engineers (IEEE). https://doi.org/10.1109/ICRA.2019.8794369

Bujanca, Mihai ; Gafton, Paul ; Saeedi, Sajad ; Nisbet, Andy ; Bodin, Bruno ; O'Boyle, Michael F. P. ; Davison, Andrew J. ; Kelly, Paul H. J. ; Riley, Graham ; Lennox, Barry ; Luján, Mikel ; Furber, Steve. / SLAMBench 3.0 : Systematic Automated Reproducible Evaluation of SLAM Systems for Robot Vision Challenges and Scene Understanding. 2019 International Conference on Robotics and Automation (ICRA). Institute of Electrical and Electronics Engineers (IEEE), 2019. pp. 6351-6358

@inproceedings{4ece12047b3d4d249a14f04f301868a6,

title = "SLAMBench 3.0: Systematic Automated Reproducible Evaluation of SLAM Systems for Robot Vision Challenges and Scene Understanding",

abstract = "As the SLAM research area matures and the number of SLAM systems available increases, the need for frameworks that can objectively evaluate them against prior work grows. This new version of SLAMBench moves beyond traditional visual SLAM, and provides new support for scene understanding and non-rigid environments (dynamic SLAM). More concretely for dynamic SLAM, SLAMBench 3.0 includes the first publicly available implementation of DynamicFusion, along with an evaluation infrastructure. In addition, we include two SLAM systems (one dense, one sparse) augmented with convolutional neural networks for scene understanding, together with datasets and appropriate metrics. Through a series of use-cases, we demonstrate the newly incorporated algorithms, visulation aids and metrics (6 new metrics, 4 newdatasets and 5 new algorithms).",

author = "Mihai Bujanca and Paul Gafton and Sajad Saeedi and Andy Nisbet and Bruno Bodin and O'Boyle, {Michael F. P.} and Davison, {Andrew J.} and Kelly, {Paul H. J.} and Graham Riley and Barry Lennox and Mikel Luj{\'a}n and Steve Furber",

year = "2019",

month = aug,

day = "12",

doi = "10.1109/ICRA.2019.8794369",

language = "English",

isbn = "978-1-5386-8176-3",

publisher = "Institute of Electrical and Electronics Engineers (IEEE)",

pages = "6351--6358",

booktitle = "2019 International Conference on Robotics and Automation (ICRA)",

address = "United States",

note = "2019 IEEE International Conference on Robotics and Automation (ICRA), ICRA 2019 ; Conference date: 20-05-2019 Through 24-05-2019",

url = "https://www.icra2019.org/",

}

Bujanca, M, Gafton, P, Saeedi, S, Nisbet, A, Bodin, B, O'Boyle, MFP, Davison, AJ, Kelly, PHJ, Riley, G, Lennox, B, Luján, M & Furber, S 2019, SLAMBench 3.0: Systematic Automated Reproducible Evaluation of SLAM Systems for Robot Vision Challenges and Scene Understanding. in 2019 International Conference on Robotics and Automation (ICRA). Institute of Electrical and Electronics Engineers (IEEE), pp. 6351-6358, 2019 IEEE International Conference on Robotics and Automation (ICRA), Montreal, Canada, 20/05/19. https://doi.org/10.1109/ICRA.2019.8794369

SLAMBench 3.0 : Systematic Automated Reproducible Evaluation of SLAM Systems for Robot Vision Challenges and Scene Understanding. / Bujanca, Mihai; Gafton, Paul; Saeedi, Sajad; Nisbet, Andy; Bodin, Bruno; O'Boyle, Michael F. P.; Davison, Andrew J.; Kelly, Paul H. J.; Riley, Graham; Lennox, Barry; Luján, Mikel; Furber, Steve.

2019 International Conference on Robotics and Automation (ICRA). Institute of Electrical and Electronics Engineers (IEEE), 2019. p. 6351-6358.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - SLAMBench 3.0

T2 - 2019 IEEE International Conference on Robotics and Automation (ICRA)

AU - Bujanca, Mihai

AU - Gafton, Paul

AU - Saeedi, Sajad

AU - Nisbet, Andy

AU - Bodin, Bruno

AU - O'Boyle, Michael F. P.

AU - Davison, Andrew J.

AU - Kelly, Paul H. J.

AU - Riley, Graham

AU - Lennox, Barry

AU - Luján, Mikel

AU - Furber, Steve

PY - 2019/8/12

Y1 - 2019/8/12

N2 - As the SLAM research area matures and the number of SLAM systems available increases, the need for frameworks that can objectively evaluate them against prior work grows. This new version of SLAMBench moves beyond traditional visual SLAM, and provides new support for scene understanding and non-rigid environments (dynamic SLAM). More concretely for dynamic SLAM, SLAMBench 3.0 includes the first publicly available implementation of DynamicFusion, along with an evaluation infrastructure. In addition, we include two SLAM systems (one dense, one sparse) augmented with convolutional neural networks for scene understanding, together with datasets and appropriate metrics. Through a series of use-cases, we demonstrate the newly incorporated algorithms, visulation aids and metrics (6 new metrics, 4 newdatasets and 5 new algorithms).

AB - As the SLAM research area matures and the number of SLAM systems available increases, the need for frameworks that can objectively evaluate them against prior work grows. This new version of SLAMBench moves beyond traditional visual SLAM, and provides new support for scene understanding and non-rigid environments (dynamic SLAM). More concretely for dynamic SLAM, SLAMBench 3.0 includes the first publicly available implementation of DynamicFusion, along with an evaluation infrastructure. In addition, we include two SLAM systems (one dense, one sparse) augmented with convolutional neural networks for scene understanding, together with datasets and appropriate metrics. Through a series of use-cases, we demonstrate the newly incorporated algorithms, visulation aids and metrics (6 new metrics, 4 newdatasets and 5 new algorithms).

U2 - 10.1109/ICRA.2019.8794369

DO - 10.1109/ICRA.2019.8794369

M3 - Conference contribution

SN - 978-1-5386-8176-3

SP - 6351

EP - 6358

BT - 2019 International Conference on Robotics and Automation (ICRA)

PB - Institute of Electrical and Electronics Engineers (IEEE)

Y2 - 20 May 2019 through 24 May 2019

ER -

Bujanca M, Gafton P, Saeedi S, Nisbet A, Bodin B, O'Boyle MFP et al. SLAMBench 3.0: Systematic Automated Reproducible Evaluation of SLAM Systems for Robot Vision Challenges and Scene Understanding. In 2019 International Conference on Robotics and Automation (ICRA). Institute of Electrical and Electronics Engineers (IEEE). 2019. p. 6351-6358 https://doi.org/10.1109/ICRA.2019.8794369

SLAMBench 3.0: Systematic Automated Reproducible Evaluation of SLAM Systems for Robot Vision Challenges and Scene Understanding

Abstract

Publication series

Conference

Access to Document

PAMELA: a Panoramic Approach to the Many-CorE LAndsape - from end-user to end-device: a holistic game-changing approach

Cite this