Benchmark Problem "Multisensor SLAM - Bovisa_2008-10-04"
Perform a map building activity with SLAM (online), using an arbitrary subset (or all) of the sensor data files provided below. Such files include all those associated to the Bovisa_2008-10-04 data-collection session (sensor data, calibration, and ground truth); typically, they will not be all necessary to solve the Benchmark Problem, depending on the type of algorithm used.
This BP belongs to the very first batch of problems published by Rawseeds; other BPs will follow.
Media
Images
Attached Files
- Matlab scripts and data to automatically compute some of the evaluation metrics, given the trajectory data output by a SLAM algorithm. Includes an extended ground truth covering the whole path of the robot, obtained by manual scan matching performed on the data from onboard LRFs (indoor datasets only). - _Rawseeds_Metrics_Computation_Toolkit
- List of corner positions for the Bovisa location (extracted from the executive drawings) - _Bovisa-Corners.tar.torrent
- Set #02 of sensor positions on the robot - _SensorPositions_02.tar.torrent
- Set #01 of file formats - _FileFormats_01.tar.torrent
- CAD drawings of the Bovisa location - _Drawings_01.dxf.bz2.torrent
- Results of Calibration 02. - _Calibration_02-Results.tar.torrent
- Images from the top SVS camera used for Calibration 02. Checkerboard has 100mm x 100mm squares. - _Calibration_02-Used_for_calibration-SVS_T.tar.bz2.torrent
- Images from the right SVS camera used for Calibration 02. Checkerboard has 100mm x 100mm squares. - _Calibration_02-Used_for_calibration-SVS_R.tar.bz2.torrent
- Images from the left SVS camera used for Calibration 02. Checkerboard has 100mm x 100mm squares. - _Calibration_02-Used_for_calibration-SVS_L.tar.bz2.torrent
- Images from the omnidirectional camera used for Calibration 02. Checkerboard has 100mm x 100mm squares. - _Calibration_02-Used_for_calibration-OMNI.tar.bz2.torrent
- Images from the front camera used for Calibration 02. Checkerboard has 100mm x 100mm squares. - _Calibration_02-Used_for_calibration-FRONTAL.tar.bz2.torrent
- Set of images generated by the top SVS camera from which the images used for Calibration 02 have been selected. Checkerboard has 100mm x 100mm squares. - _Calibration_02-All_images-SVS_T.tar.bz2.torrent
- Set of images generated by the right SVS camera from which the images used for Calibration 02 have been selected. Checkerboard has 100mm x 100mm squares. - _Calibration_02-All_images-SVS_R.tar.bz2.torrent
- Set of images generated by the left SVS camera from which the images used for Calibration 02 have been selected. Checkerboard has 100mm x 100mm squares. - _Calibration_02-All_images-SVS_L.tar.bz2.torrent
- Set of images generated by the omnidirectional camera from which the images used for Calibration 02 have been selected. Checkerboard has 100mm x 100mm squares. - _Calibration_02-All_images-OMNI.tar.bz2.torrent
- Set of images generated by the front camera from which the images used for Calibration 02 have been selected. Checkerboard has 100mm x 100mm squares. - _Calibration_02-All_images-FRONTAL.tar.bz2.torrent
- Ground truth (trajectory data) from RTK-GPS - Bovisa_2008-10-04-GPS.csv.bz2.torrent
- Top SVS camera: data - Bovisa_2008-10-04-SVS_T.tar.torrent
- Right SVS camera: data - Bovisa_2008-10-04-SVS_R.tar.torrent
- Left SVS camera: data - Bovisa_2008-10-04-SVS_L.tar.torrent
- Rear Sick LRF: data - Bovisa_2008-10-04-SICK_REAR.csv.bz2.torrent
- Front Sick LRF: data - Bovisa_2008-10-04-SICK_FRONT.csv.bz2.torrent
- Omnidirectional camera: data - Bovisa_2008-10-04-OMNI.tar.torrent
- Odometry: data - Bovisa_2008-10-04-ODOMETRY_XYT.csv.bz2.torrent
- Set of list files associated to video streams. For each camera it includes: - a .csv file with the timestamps of all the frames in the data stream; - a .lst file with the filenames of all the frames in the data stream. - Bovisa_2008-10-04-LISTS.tar.torrent
- IMU: data - Bovisa_2008-10-04-IMU_STRETCHED.csv.bz2.torrent
- Rear Hokuyo LRF: data - Bovisa_2008-10-04-HOKUYO_REAR.csv.bz2.torrent
- Front Hokuyo LRF: data - Bovisa_2008-10-04-HOKUYO_FRONT.csv.bz2.torrent
- Front camera: data - Bovisa_2008-10-04-FRONTAL.tar.torrent
Evaluation Methodologies
The provided solutions will be evaluated and scored with respect to the following evaluation methodologies:
- ATE compares the trajectory of a robot, as reconstructed by an algorithm using real sensor data as its input, to the actual trajectory (ground truth). ATE is a mandatory performance measure. Please ... - Absolute Trajectory Error
- ME compares the map of an environment, as reconstructed by an algorithm using real sensor data as its input, to the actual map of the location (ground truth). ME is a recommended performance measure. - Mapping Error
- RPE measures the accuracy of a SLAM result, as reconstructed by an algorithm using real sensor data as its input, by comparing the reconstructed relative transformations between nearby poses to the ac... - Relative Pose Error
- REC provides a basic estimate of how the running time of an algorithm (which uses real sensor data as its input) scales as the quantity of data available to be processed increases. REC is a mandatory... - Rough Estimate of Complexity
- SLE aims to evaluate the overall quality of a SLAM algorithm by actually using its output in a realistic application. The SLAM algorithm, fed with real sensor data from a robot, is used to build a map... - Self Localization Error