Benchmark Problem "Laser SLAM - Bovisa_2008-10-04"
Perform a map building activity with SLAM (online), using the sensor data files provided below. Such files come from the Bovisa_2008-10-04 data-collection session; they include output and calibration information associated to laser range scanners (2 Sick units, 2 Hokuyo units), IMU, odometry, and the ground truth for the specified session.
Please note that the Bovisa_2008-10-04 session also includes other data files, not used for this Benchmark Problem.
This BP belongs to the very first batch of problems published by Rawseeds; other BPs will follow.
Solutions
| Absolute Trajectory Error | Mapping Error | Relative Pose Error | Rough Estimate of Complexity | Self Localization Error | |
|---|---|---|---|---|---|
| Vasco Scan-Matcher |
mean: 50.755 stddev: 30.2636 CI low: 0 CI high: 141.546 |
: 0 |
RPE-T mean in [m2]: 8.93088 RPE-T stddev in [m2]: 35.309 RPE-R mean in [rad2]: 0.104311 RPE-R stddev in [rad2]: 0.391753 |
: 0 |
: 0 |
| GMapping |
mean: 0.746728 stddev: 0.646163 CI low: 0 CI high: 2.68522 |
: 0 |
RPE-T mean in [m2]: 0.090158 RPE-T stddev in [m2]: 0.24999 RPE-R mean in [rad2]: 0.014 RPE-R stddev in [rad2]: 0.018804 |
: 0 |
: 0 |
| GraphSLAM |
mean: 1.6008 stddev: 0.839729 CI low: 0 CI high: 4.11999 |
: 0 |
RPE-T mean in [m2]: 0.076609 RPE-T stddev in [m2]: 0.195128 RPE-R mean in [rad2]: 0.013384 RPE-R stddev in [rad2]: 0.016656 |
: 0 |
: 0 |
Media
Images
Attached Files
- List of corner positions for the Bovisa location (extracted from the executive drawings) - _Bovisa-Corners.tar.torrent
- Set #02 of sensor positions on the robot - _SensorPositions_02.tar.torrent
- Set #01 of file formats - _FileFormats_01.tar.torrent
- CAD drawings of the Bovisa location - _Drawings_01.dxf.bz2.torrent
- Ground truth (trajectory data) from RTK-GPS - Bovisa_2008-10-04-GPS.csv.bz2.torrent
- Rear Sick LRF: data - Bovisa_2008-10-04-SICK_REAR.csv.bz2.torrent
- Front Sick LRF: data - Bovisa_2008-10-04-SICK_FRONT.csv.bz2.torrent
- Odometry: data - Bovisa_2008-10-04-ODOMETRY_XYT.csv.bz2.torrent
- IMU: data - Bovisa_2008-10-04-IMU_STRETCHED.csv.bz2.torrent
- Rear Hokuyo LRF: data - Bovisa_2008-10-04-HOKUYO_REAR.csv.bz2.torrent
- Front Hokuyo LRF: data - Bovisa_2008-10-04-HOKUYO_FRONT.csv.bz2.torrent
- Matlab scripts and data to automatically compute some of the evaluation metrics, given the trajectory data output by a SLAM algorithm. Includes an extended ground truth covering the whole path of the robot, obtained by manual scan matching performed on the data from onboard LRFs (indoor datasets only). - _Rawseeds_Metrics_Computation_Toolkit
Evaluation Methodologies
The provided solutions will be evaluated and scored with respect to the following evaluation methodologies:
- ATE compares the trajectory of a robot, as reconstructed by an algorithm using real sensor data as its input, to the actual trajectory (ground truth). ATE is a mandatory performance measure. Please ... - Absolute Trajectory Error
- ME compares the map of an environment, as reconstructed by an algorithm using real sensor data as its input, to the actual map of the location (ground truth). ME is a recommended performance measure. - Mapping Error
- RPE measures the accuracy of a SLAM result, as reconstructed by an algorithm using real sensor data as its input, by comparing the reconstructed relative transformations between nearby poses to the ac... - Relative Pose Error
- REC provides a basic estimate of how the running time of an algorithm (which uses real sensor data as its input) scales as the quantity of data available to be processed increases. REC is a mandatory... - Rough Estimate of Complexity
- SLE aims to evaluate the overall quality of a SLAM algorithm by actually using its output in a realistic application. The SLAM algorithm, fed with real sensor data from a robot, is used to build a map... - Self Localization Error