Real-time onboard visual-inertial state estimation and self-calibration of MAVs in unknown environments

S. Weiss, M.W. Achtelik, S. Lynen, M. Chli, R. Siegwart

Research output: Chapter in Book/Report/Conference proceedingConference contribution


The combination of visual and inertial sensors has proved to be very popular in robot navigation and, in particular, Micro Aerial Vehicle (MAV) navigation due the flexibility in weight, power consumption and low cost it offers. At the same time, coping with the big latency between inertial and visual measurements and processing images in real-time impose great research challenges. Most modern MAV navigation systems avoid to explicitly tackle this by employing a ground station for off-board processing. In this paper, we propose a navigation algorithm for MAVs equipped with a single camera and an Inertial Measurement Unit (IMU) which is able to run onboard and in real-time. The main focus here is on the proposed speed-estimation module which converts the camera into a metric body-speed sensor using IMU data within an EKF framework. We show how this module can be used for full self-calibration of the sensor suite in real-time. The module is then used both during initialization and as a fall-back solution at tracking failures of a keyframe-based VSLAM module. The latter is based on an existing high-performance algorithm, extended such that it achieves scalable 6DoF pose estimation at constant complexity. Fast onboard speed control is ensured by sole reliance on the optical flow of at least two features in two consecutive camera frames and the corresponding IMU readings. Our nonlinear observability analysis and our real experiments demonstrate that this approach can be used to control a MAV in speed, while we also show results of operation at 40Hz on an onboard Atom computer 1.6 GHz.
Original languageEnglish
Title of host publicationRobotics and Automation (ICRA), 2012 IEEE International Conference on
Number of pages8
ISBN (Electronic)978-1-4673-1404-6
Publication statusPublished - 2012


  • aircraft
  • cameras
  • computational complexity
  • inertial navigation
  • microrobots
  • mobile robots
  • pose estimation
  • robot vision
  • EKF framework
  • MAV self-calibration
  • camera
  • constant complexity
  • fall-back solution
  • ground station
  • inertial measurement unit
  • inertial sensors
  • keyframe-based VSLAM module
  • metric body-speed sensor
  • micro aerial vehicle navigation
  • observability analysis
  • on-board processing
  • onboard Atom computer 1.6 GHz
  • real-time onboard visual-inertial state estimation
  • robot navigation
  • scalable 6DoF pose estimation
  • speed-estimation module
  • unknown environments
  • visual sensors
  • Atom optics
  • Equations
  • Estimation
  • Navigation
  • Optical sensors
  • Robustness
  • Weight measurement

Fingerprint Dive into the research topics of 'Real-time onboard visual-inertial state estimation and self-calibration of MAVs in unknown environments'. Together they form a unique fingerprint.

Cite this