Benchmark Problems

This page lists all the Benchmark Problems instances included in Rawseeds' Benchmarking Toolkit. Don't know what a BP or the Toolkit are? Please take a look here.

In the list below, the labels associated to the BPs describe their main features. By clicking on a label, you can select the BPs that possess such feature.

Within Rawseeds, we use the expression Benchmark Problem instance to identify a problem using real-world sensor data as its input, related to the typical tasks that an autonomous mobile robot has to perform. A BP instance is a specific instance of a BP: the BP is a description of the kind of problem (e.g., "perform SLAM"), while the specificity comes from the choice of a specific set of data files among those included in Rawseeds' datasets. One important feature of the BPs is the fact that they include a set of well-defined performance measures to assess the solutions for the BP instances. By using such measures, a quantitative evaluation of the quality of the solutions can be done; and, more importantly, different solutions can be compared.

A solution to a BP instance is called a Benchmark Solution; each BP instance admits any number of different BSs. The main components of a BS are: (the description of) an algorithm; the output of the algorithm when applied to the data of the chosen BP instance; and the values of the performance measures defined by the BP, when applied to such output.

* - * - * - * - *

The key components of the Benchmark Problem instances listed below are, of course, the sensor data files. Rawseeds' data files are distributed via BitTorrent ONLY. Don't worry, this will not require any additional work on your part :-) We used BitTorrent to let downloaders help each other getting large quantities of data at high speed, without the need - for us - to pay for ultrawide-bandwidth hosting (that we can't afford). You don't know a thing about BitTorrent? This will quickly let you know what you need to proceed with the downloads. Before reading, try and click on the download links for the files you need: it's likely that your system already knows how to use BitTorrent for downloading...

IMPORTANT: please help other users (and Rawseeds) by redistributing the data files after you have downloaded them. It's sufficient that you leave your BitTorrent client program running, without removing the compressed files from the download directory. Here we explain why and how you should do this. Thank you very much!