ToyCollect - A mobile robot which collects toys from unaccessible places

2020/02/22

I was motivated for this project by my - at project start - two-year old daughter who always hid her toys under the couch. It was quite hard finding out what lay down below. So we built a robot that could be remotely controlled by any Android device via Wifi, drove under the couch, looked at the toys, and moved out what toys it found. It was just under 6.5cm tall, streamed high-quality H264-encoded video in real time, was controlled with a simple touch-based interface, and we even put in a LED lamp so you could actually see something in dark places.

In the meantime, we have built numerous variants of the original robot and are researching on how to make them fully autonomous. Here is a short overview of the different robots, partly with blueprints, as well as the different remote controls.

Robots

TypeRobot plattformCameraMain controllerChassisPowerLocal DL?Construction
v1.0
Pololu Zumo (#1418)1x RPi Camera Rv1.3 (right)1x RPi 1BModified black chassis4x 1.5V 2900mAh Lithium batteries (non-rechargable)No, <1fps2014
Blueprint
v1.1
Pololu Zumo (#1418)1x RPi Camera Rv1.3 (centered)1x RPi 2B+ v1.2Modified transparent chassis4x 3.7V 14500 Lithium-Ion batteriesNo, <1fps2016
Blueprint
v1.2
Pololu Zumo (#1418)2x RPi Camera Rv2.1 (Stereo)2x RPi Zero W3D-printed chassis4x 3.7V 14500 Lithium-Ion batteriesNo, <1fps2018
v1.21 (R2X)
Pololu Zumo (#1418)2x RPi Camera Rv2.1 (Stereo)2x RPi Zero WModular 3D-printed chassisOne of:
  • 4x 3.7V 14500 Lithium-Ion batteries
  • 4x 1.5V Alkaline batteries
  • 4x 1.2V NiMH batteries
  • 4x 1.2V NiCd batteries
No, <1fps2019
Blueprint
v1.3 (K3D)
Pololu Zumo (#1418)1x RPi Camera Rv1.3 with Kúla3D Bebe Smartphone lens (Stereo)1x RPi 3B+Modified transparent chassis4x 3.7V 14500 Lithium-Ion batteriesYes, 8fps2018
Blueprint
v2.0
Dagu Robotics Wild Thumper 4WD (#RS10)2x RPi Camera Rv2.1 (Stereo)1x RPi Compute Module 1 on eval­boardNone (open case)1x 7.2V 2S 5000mAh Lithium-Ion batteryNo, <1fps2016
v2.1
Dagu Robotics Wild Thumper 4WD (#RS10)2x RPi Camera Rv2.1 (Stereo), Asus Xtion depth camera1x RPi Compute Module 1 on eval­boardNone (open case)1x 7.2V 2S 5000mAh Lithium-Ion batteryNo, <1fps2017
v2.2 (OUT)
Dagu Robotics Wild Thumper 4WD (#RS10)2x RPi Camera Rv2.1 (Stereo), Asus Xtion depth camera1x RPi Compute Module 3 on eval­boardNone (open case)1x 7.2V 2S 5000mAh Lithium-Ion batteryYes, 8fps2018


Remote controls

TypePlatformRobot compatibilityCode
v1.0v1.1v1.2v1.21 (R2X)v1.3 (K3D)v2.0v2.1v2.2 (OUT)available?
Touch control with circle plus LED brightnessAndroidYesYes (old TCserver)Yes
Touch control with two circlesAndroidYesYes (old TCserver)on request
Head movement control via Google CardboardAndroidYesYesYesYesYesYeson request
Touch control of robot and robot armAndroidYeson request
2x Wii & Nunchuk controller for robot and robot arm, Google Cardboard as displayLinux & AndroidYeson request
Driving wheel and accelerator/brake (old Play­Station controller), display on 40" 3D-screenLinuxYesYesYeson request
Bluetooth controllerLinux (RPi, TCserver)Yes (new TCserver)Yes (new TCserver)YesYesYesYesYesYesYes
Pretrained Deep-Learning models (autonomous behaviour)Linux (RPi, TCcontrol)Yes (server)Yes (server)Yes (local)Yes (local)Yes (local)Yes (local)Yes

The pretrained deep learning models require video output in uncompressed format and low resolution for efficiency reasons. However, most other remote controls require video output in compressed format (H2.64) and high resolution for quality reasons. Therefore, at present, the use of pretrained deep learning models can only be combined with the Bluetooth controller remote control.

If you find our research helpful and use it for your own research publications or technical reports, please cite one of our papers:

Seewald, A. (2022). Evaluating Two Ways for Mobile Robot Obstacle Avoidance with Stereo Cameras: Stereo View Algorithms and End-to-End Trained Disparity-sensitive Networks. In Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 3, ISBN 978-989-758-547-0, ISSN 2184-433X, pages 663-672. DOI: 10.5220/0010878500003116 Seewald, A. (2020). Revisiting End-to-end Deep Learning for Obstacle Avoidance: Replication and Open Issues. In Proceedings of the 12th International Conference on Agents and Artificial Intelligence - Volume 2, ISBN 978-989-758-395-7, ISSN 2184-433X, pages 652-659. DOI: 10.5220/0008979706520659

Partially funded by FFG Talente 2014,2016,2017,2018,2019