DARKO - Object-throwing robots with people skills
As one of the partners in the EU research project known as DARKO (Dynamic Agile Production Robots That Learn and Optimise Knowledge and Operations), Bosch Research is working on developing agile production robots that can interact with humans much more efficiently than has so far been the case.
One of the main roles in the music video for The Strokes’ song “The Adults Are Talking” is played by a baseball-throwing robot. But how close is this kind of thing to reality right now? And how important are such sophisticated service robots for logistics? Bosch researcher Timm Linder has the answers.
Object-throwing robots play baseball against The Strokes
A countdown is initiated on one of the shiny robots’ breastplates. Its green eyes light up. The smart machine draws its arm back and then hurls a baseball right at the batter. The motions are super smooth – you would be hard pushed to spot any differences with those of a professional pitcher made of flesh and blood. The first batter up is Julian Casablancas, the frontman of the US rock band The Strokes. He and his bandmates go up against the data-driven opponents in the music video for “The Adults Are Talking”.
Will object-throwing robots soon become a reality?
Agile service robots that can interact safely with humans increase the efficiency of industrial processes
For some time now, robots have been able to grasp objects of different shapes and sizes. The ability to throw an object and hit a specific (and potentially moving) target is a much more complex affair though. However, researchers all around the world are actually already working very hard to achieve that. “Robots which can throw objects and are capable of carrying out complex tasks, while, consistently improving their understanding of their environment and interacting safely with humans, could one day make certain manufacturing and logistical processes significantly more cost and time-effecient, says Timm Linder, Research Engineer at Bosch. Throwing could possibly become the most flexible form of material handling in the Factory of the Future, which will in any case feature modular process units which can be continually reconfigured. “Object-throwing robots are faster than mobile robots or conveyor belts over longer distances and more cost-effective than larger robotic arms which would otherwise be necessary.” What’s more, object-throwing robots would also be able to carry out extremely repetitive tasks which humans find not only physically demanding but also mentally draining due to the monotony involved. Employees would therefore be relieved of such tasks and would have greater freedom to concentrate on other areas.
Innovative material handling in the Factory of the Future
DARKO, which was launched at the start of 2021, is a four-year EU research project which aims at developing novel and efficient ways of leveraging throwing robots in production. Seven project partners from a total of five countries are striving to develop agile production robots that can execute highly dynamic motions with the help of elastic actuators. The robots should also have predictive planning capabilities and “be aware” of their human colleagues’ intentions in order to be able to ensure that reliable operations can be achieved in a constantly changing environment. As an industrial partner working alongside five universities, Bosch is actively involved in this cutting-edge research. Furthermore, the spare parts warehouse of BSH Hausgeräte GmbH in Fürth (Germany) provides the lead use case of the project.
The project partners
- Örebro University (Sweden)
- Technical University of Munich (Germany)
- Robert Bosch GmbH (Bosch)
- University of Pisa (Italy)
- EPFL - École Polytechnique Fédérale de Lausanne (Switzerland)
- University of Lincoln (United Kingdom)
- ACT Operations Research (Italy)
Comparable to a human arm
The racks in Fürth hold more than 140,000 parts – from washers to washing machine motors. Employees actually throw some of them over short distances in order to save time. In the future, object-throwing robots could provide support here by collecting frequently required components from the racks and throwing them into moving conveyor trays. To be able to do this, the agile service robots would have to be able to move around freely in the warehouse and avoid other equipment and humans. “The plan is to devise a mobile base which would have a robotic arm mounted on top driven by elastic actuators and developed by one of the academic project partners”, explains Timm Linder. The idea is for it to function similarly to a human arm with tendons and muscles.
The success of the project relies on teamwork and the collaborative efforts of experts from a wide range of different backgrounds. The project partners, which in addition to Robert Bosch GmbH include several university research facilities, are each responsible for a specific step in the development process. “Bosch is developing the algorithms for the mobile base’s motion planning, the 3D environment perception, the detection of humans and the prediction of human motions and intentions – for us, the first steps in ‘mechanical knowledge of human behavior’”, reports the expert.
Smarter cooperation between humans and machines
Machines such as robotic vacuum cleaners and robotic lawn mowers are actually already able to detect obstacles in their way and circumnavigate them, “however, initially this has been achieved mainly through (AI-based) pattern recognition, without a deeper semantic understanding. The next step is for robots to be able to perform intelligent reasoning and take appropriate actions for the task at hand ”, says the engineer. Robotic vacuum cleaners, for example, might then realize that the presence of four chairs and a table meant that it was in a dining area and could conclude: “It would be more efficient for me to do this in one go at the end”. Transposing this idea to the manufacturing area would mean that robots could, for example, learn to understand how people typically move in a factory and then adapt their behavior accordingly, in order to avoid them more intelligently. In addition to realistic simulations which use various different physical models to depict and map human behavior, Bosch researchers also use data on real human movements from research studies to train the algorithms. “We are in the process of profitably combining model-based knowledge with artificial intelligence in the form of deep learning – i.e. data-driven pattern recognition with the help of deep neural networks – and semantic reasoning”, says Linder.
Combination of domain knowledge and artificial intelligence
Deep learning is a key focus of Bosch Research’s work. The researchers’ findings are being presented at renowned international conferences and published in scientific journals on an ongoing basis. The company benefits from the fact that in addition to its knowledge of artificial intelligence it also boasts a wealth of domain knowledge amassed across a wide range of different application areas. “Unlike pure software companies, for example, we have direct access to actual issues in an industrial environment”, says Timm Linder. This helps Bosch stand out from the competition when trying to attract young talented AI experts. “As such, we are exploring and developing solutions for the ‘Factory of the Future’”. The technologies developed by Bosch Research as part of the DARKO project are ultimately extremely important for the future of service robots both in general and in intralogistics in industrial environments.
Dr. Timm Linder, Research Engineer and Project Lead for “Robotics Roadmap Technologies”
Dr. Timm Linder, Research Engineer and Project Lead for “Robotics Roadmap Technologies”
Timm Linder studied Applied Computer Science at the University of Duisburg-Essen. As part of his PhD at the Albert Ludwig University of Freiburg, he focused on the perception of humans using mobile robots, and was involved in an international research project which developed a socially aware service robot for guiding passengers at Amsterdam Airport to their gate. Timm Linder has worked as a Research Engineer at Bosch since 2016. From 2017 to 2021, he was work package leader for “Human-Aware AGV Fleets” in the publicly funded EU project ILIAD, which focused on automated guided vehicles that could operate safely and efficiently amongst humans. This year, Linder was appointed as the Project Lead for Bosch’s internal “Robotics Roadmap Technologies” project which focuses on exploring and developing fundamental algorithms for the navigation and environment perception of mobile robots. Part of this involves the EU research project DARKO (Dynamic Agile Production Robots That Learn and Optimise Knowledge and Operations) in which he leads a work package on“3D Perception and Scene Understanding”.