TiROD

TiROD: Tiny Robotics Dataset and Benchmark for Continual Object Detection

Abstract

This is the official website for the TiROD dataset and benchmark.

Detecting objects in mobile robotics is crucial for numerous applications, from autonomous navigation to inspection. However, robots are often required to perform tasks in different domains with respect to the training one and need to adapt to these changes. Tiny mobile robots, subject to size, power, and computational constraints, encounter even more difficulties in running and adapting these algorithms. Such adaptability, though, is crucial for real-world deployment, where robots must operate effectively in dynamic and unpredictable settings. In this work, we introduce a novel benchmark to evaluate the continual learning capabilities of object detection systems in tiny robotic platforms. Our contributions include:

Tiny Robotics Object Detection~(TiROD), a comprehensive dataset collected using a small mobile robot, designed to test the adaptability of object detectors across various domains and classes;
an evaluation of state-of-the-art real-time object detectors combined with different continual learning strategies on this dataset, providing detailed insights into their performance and limitations;
moreover, we publish the data and the code to replicate the results to foster continuous advancements in this field. Our benchmark results indicate key challenges that must be addressed to advance the development of robust and efficient object detection systems for tiny robotics.

🌐 Links

📊 Dataset Information

Attribute	Description
Name	TiROD
Number of Images	17.4K
Images Size	640x480x3
Number of Classes	13
Data Format	png
Annotations	COCO format
Download Link	Download Dataset

The distribution of labels across different tasks can be observed in the following figure.

💻 Data Preview

Here are some example frames from each of the 10 CL tasks:

📂 Folder Structure:

TiROD
├── Domain1
│   ├── High
│   │   ├── annotations
│   │   │   ├── train.json
│   │   │   ├── val.json
│   │   │   ├── test.json
│   │   ├── images
│   │   │   ├── train
│   │   │   │   ├── frame1.png
│   │   │   │   ├── ...
│   │   │   ├── val
│   │   │   │   ├── ...
│   │   │   ├── test
│   │   │   │   ├── ...
│   ├── Low
│   │   ├── ...
├── ...
└── docs
    └── README.md

TiROD Cross-Domain Benchmark results

Method	Final mAP ↑	RSD ↑	RPD ↑
Fine-Tuning	10.7	0.17	0.97
LWF	12.6	0.27	0.98
IncDet	12.9	0.18	0.91
SID	16.4	0.41	0.84
Replay	37.8	0.70	0.74
Temporal Replay	25.9	0.50	0.96
K-Means Replay	42.5	0.77	0.95
Latent Distillation	14.5	0.38	0.76
Latent Replay	36.5	0.65	0.90
Latent K-Means Replay	37.8	0.68	0.90
Cumlative Training [mAP]	63%

TiROD Intra-Domain Benchmark results

Method	Final mAP ↑	RSD ↑	RPD ↑
Fine-Tuning	47.0	0.83	0.95
LWF	49.0	0.83	0.94
IncDet	45.9	0.82	0.95
SID	49.2	0.76	0.89
Replay	66.1	0.96	0.96
Temporal Replay	60.3	0.91	0.94
K-Means Replay	66.0	0.96	0.96
Latent Distillation	44.1	0.71	0.86
Latent Replay	62.5	0.93	0.91
Latent K-Means Replay	62.6	0.94	0.94
Cumlative Training [mAP]	70.6 %

To replicate the results, clone this repository and follow the instructions of the Readme.md

There is also a YOLOv8 nano implementation, you can find it in this repository repository

Citation

If you find this project useful in your research, please add a star and cite us 😊

@misc{pasti2026tirod,
      title={TiROD: Tiny Robotics Dataset and Benchmark for Continual Object Detection}, 
      author={Francesco Pasti and Riccardo De Monte and Davide Dalle Pezze and Gian Antonio Susto and Nicola Bellotto},
      year={2026},
      eprint={2409.16215},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2409.16215}, 
}

@inproceedings{pasti2024LatentDistillation,
  title={Latent Distillation for Continual Object Detection at the edge.},
  author={Pasti, Francesco and Ceccon, Marina and Dalle Pezze, Davide and Paissan, Francesco and Farella, Elisabetta and Susto, Gian Antonio and Bellotto, Nicola},
  booktitle={ECCV 2024 Workshops},
  year={2024},
  publisher={Springer}
}

https://github.com/pastifra/Continual_Nanodet

Thanks

https://github.com/RangiLyu/nanodet