A more useful way to measure robotic localization error
Measuring the displacement between location estimates derived from different camera views can help enforce the local consistency vital to navigation.
Simultaneous localization and mapping (SLAM) is one of the core capabilities of autonomous robots, with a rich literature behind it. “Mapping” means building a map of the robot’s environment, “localization” means identifying the robot’s location on the map, and “simultaneous” means the robot has to do both at once.
More coverage of devices and services announcements
On the basis of SLAM, the robot can plan a trajectory through its environment. But no SLAM algorithm is perfect. How accurate does localization have to be to ensure workable trajectory planning?
In a paper we’re presenting at this year’s International Conference on Intelligent Robots and Systems (IROS), we argue that existing metrics for measuring SLAM error aren’t well suited to the problem of navigation. In particular, minimizing global error can leave open the possibility of local errors that make trajectory planning extremely difficult.
Instead, we propose a new metric, which we call the overlap displacement error (ODE). ODE may sometimes lead to solutions with a slightly larger global metric error, but it enforces greater local consistency, which we believe will enable robust trajectory planning.
Think locally, act globally
Below is an example showing why conventional SLAM metrics are not ideal for purposes of navigation. The figure shows two different maps of a large building with two wings, produced by two different SLAM algorithms applied to the same sensor data.
On the left, we show a SLAM algorithm that is globally very accurate; the error in its estimate of the distance from the edge of the left wing to the edge of the right is only 1 centimeter. However, in the left top corner it gets confused. The absolute trajectory error (ATE) is still very low, but the algorithm produces two different estimates of the location of a doorway (red and blue) that leave the robot believing that the opening is impassably narrow. The robot will thus have trouble planning a path through the opening if it wants to move from one wing to another.
On the right, we see the results of another SLAM algorithm. The robot’s estimate of its trajectory obviously drifts, and the algorithm has a huge absolute mapping error; the error from left to right wing is 100 centimeters. However, the SLAM algorithm appears to be locally consistent with itself, which means that although the map is skewed, it is still completely navigable.
The point to make here is that absolute error is not the most relevant measure for autonomous navigation. What is important is that whenever the robot passes through the same area, it remains consistent with its own knowledge. If it is consistent, then it won't delete walls (or create false openings), and it won't build up false walls (which block off corridors). Self-consistency is precisely the aim of the metric we proposed.
Consider a physical obstacle (the blue circle in the figure below) seen by a robot at two different times, T0 (mustard view) and T1 (green view). A SLAM algorithm maps this obstacle twice (mustard circle and green circle), but due to the localization inaccuracy, the two mapped circles do not overlap with each other as they should. ODE measures the displacement error (red arrow) from the mustard circle to the green circle.
This displacement error can be calculated for every pair of camera views of the same sensing area. If we superimpose a grid on the SLAM map, the error can be aggregated by placing a virtual obstacle on each cell of the grid.
In the figure below, we compare ODE to three conventional metrics: absolute trajectory error (ATE) and two types of relative trajectory error (RTE), linear (m) and angular (deg). On the left, we show a ground truth trajectory (green line), a SLAM algorithm’s estimated trajectory (white line), and a heat map that indicates the error that inaccurate localization would introduce into the map, if we used a 360° range sensor with a two-meter range. The numbers along the estimated trajectory indicate time steps.
On the right, we show the plots for different localization error metrics. Note that ODE focuses on map consistency, with clear peaks at the discontinuity in the trajectory estimate (100th step) and the region where the trajectory imperfectly intersects itself (40th and 220th steps). With the other metrics, however, local inconsistencies are harder to deduce.