Sharing the road: LUKAS makes urban transport safer and more efficient
Automated vehicles will soon make road traffic safer than ever before. Using an AI-based simulation that was tested at a real intersection, a project team from Bosch Research has demonstrated that it is already possible for automated vehicles to operate safely in mixed traffic with non-automated vehicles — by ensuring reliable communication between the road participants and with the infrastructure.
Look to the right, look to the left, activate the turn signal — these are just a few of the maneuvers that a driver has to execute to get their vehicle safely across an intersection. They also need to keep track of traffic signs, other oncoming or turning vehicles, and even the irregular behavior of other drivers. That’s in addition to other non-car road users, such as cyclists or people on scooters and motorcycles. Pedestrians, playing children, and unleashed dogs can also affect the situation at the intersection. And what if the driver’s view is obstructed by a building? The result is a veritable universe of possible collisions and accidents.
Planned networking: cooperation for optimized mixed traffic
Automated driving has the potential to make road traffic safer and more efficient in the future. Among other things, achieving this requires collecting and analyzing all the available data, for example, from infrastructure sensors, connected passenger and commercial vehicles, and even from mobile devices such as smartphones belonging to pedestrians or cyclists. The information collected from the different perspectives can be used in traffic situations, particularly complex ones, to help eliminate the effect of obstructions, for example. Any event that is not caught by the driver’s eye or by a camera can be detected and safe maneuvers can be planned for all road users. The aim is for road users to interact and cooperate with each other, whether they are traveling in automated, semi-automated or manual mode.
“In mixed traffic, it is inefficient for each road user to optimize their journey only for their own benefit,” says Dr. Benjamin Völz, a research engineer at Bosch Research. He adds: “One possible solution is cooperative behavioral planning that optimizes traffic flow.” From 2020 to the end of 2023, Benjamin Völz was in charge of the research for this trial within the LUKAS project. The acronym for the project comes from the German phrase meaning “Local environmental model for cooperative, automated driving in complex situations.” The idea: All of the data and information collected by the sensors of the connected road users, the cameras, and the infrastructure sensors are combined on what is known as an edge server, which is directly connected to the 5G network of the intersection. The server then calculates an optimized, cooperative driving maneuver using artificial intelligence methods and transmits instructions to the connected vehicles.
Within the LUKAS project, Benjamin Völz and his team worked together with the Universities of Ulm and Duisburg-Essen and with their partners InMach, IT-Designers, Mercedes Benz, and Nokia to research how to improve efficiency and safety in mixed traffic situations over a period of three years. Bosch Research’s contribution to the project was designing and teaching the algorithms on the edge server, which serves as the central authority for cooperative behavior planning. LUKAS was funded by the German Federal Ministry for Economic Affairs and Climate Action as part of the program for new vehicle and system technologies.
Real-world implementation: test drives at the intersection
During the project, the researchers spent a lot of time performing test drives at a real intersection with actual vehicles and infrastructure in the town of Ulm-Lehr. What makes this street so special is that it is surrounded by buildings that partially or completely obscure the view of the main road, depending on the position of the vehicle. This arrangement and the obstruction effect it creates made this intersection an ideal scenario for testing. The surrounding infrastructure, such as lampposts, is operated by the University of Ulm. The experimental setup in Ulm-Lehr was already configured in a joint predecessor project, in which cameras and laser scanners were installed for data acquisition. For the LUKAS project, Bosch contributed radar sensors for the intersection infrastructure, specifically fifth-generation radar from Bosch Automotive.
During the test, the intersection was traversed by automated vehicles equipped with sensors as well as vehicles without any digital technology on board. Since this was a real traffic situation, all automated test vehicles were operated with human safety drivers who could intervene if something went wrong with the digital system. Unlike automated and connected vehicles, non-connected vehicles cannot send information to the networked environment. Thus they have to be located by the sensors in the immediate vicinity. This is accomplished by the lampposts at the intersection, which are equipped with video, LiDAR, and radar sensors to detect and classify traffic.
Centralized control: with artificial intelligence for cooperative traffic planning
The sensors detect, for example, the vehicles’ GPS position and their speed and deliver the data to the networked, virtual environment. This object information — people and vehicles cannot be identifiable for data privacy reasons — is then sent to the edge server via the 5G network and analyzed to produce what is known as a cooperative traffic plan for the virtual world.
To achieve this, the Bosch Research project team developed a simulation model in which real traffic situations can be simulated. The team used a learning-based AI process known as reinforcement learning, which is a sub-field of machine learning. “We had to go beyond the classical imitation learning in artificial intelligence,” says Benjamin Völz, “because even with optimal traffic flow, vehicles sometimes have to cede the right of way.” There was not much to be gained from imitation learning, which is where people demonstrate behavior and the machine imitates it as closely as possible. There were simply no instances of people voluntarily giving up their right of way to enable a traffic-efficient cooperative maneuver. Especially when you recall that the vehicles sometimes cannot see each other due to the obstruction.
- The simulation model was taught with a feedback signal and a reward model. This means that the digital system is penalized if a calculated maneuver in the simulation results in a collision — receiving “negative points,” in a sense. If things go smoothly, the system is rewarded and receives “positive points,” for example, if road users maintain safe distances to one another. The system always compares several possibilities and tries to find the best possible solution for all participants by maximizing the number of points.
- Reinforcement learning consists of two components: The “actor” decides what to do. The “critic” evaluates the result by calculating the reward. During training, the behavior of the actor is adjusted until the critic no longer has any complaints. “In the calculations, the system privileges optimized traffic flows in seeking out a cooperative situation, as there are always multiple solutions for a situation,” says Benjamin Völz.
- Reinforcement learning employs an artificial neural network known as a graph neural network (GNN), which is constructed like synapses in the human brain and works like them as well, with matrices assuming the function of the synapses. Combining multiple matrices in a neural network allows more complex decisions to be made. Based on the data received from the sensors, the server sends maneuvers to the automated vehicles.
Graph Neural Network
In the LUKAS project, the researchers used what is known as a graph neural network (GNN), where the graph consists of edges and nodes. Vehicles are represented as nodes in the project while the possible interactions are represented as edges. An edge (possible interaction) always connects exactly two nodes (vehicles) and enables researchers to map possible conflicts and relative variables (for example, the distance relevant for a collision). Finally, the network learns recommended actions for each individual vehicle. The number of edges and nodes in the GNN is not limited and can be changed while the system is running. This variable size was an advantage in the LUKAS project, as it could be used to map arbitrarily complex traffic scenarios. Other network architectures, such as convolutional neural networks (CNNs) which contain fixed-size grids, are generally too inflexible.
Each maneuver contains abstract instructions. For example, based on the information gathered about the situation, the server recommends entering the intersection in five seconds at a speed of 30 km/h. To safeguard the overall system and road users, the server sends the information cyclically, even if the maneuver in question has not changed. “This confirms that the server is still live, so to speak, and still deems the maneuver feasible,” says Benjamin Völz. The vehicles verify the maneuver and confirm that it has been executed. Even if the system on the server calculates that a maneuver is required to avoid a collision, for example, and sends the information to the vehicles, a response is always required. If even one of the vehicles does not confirm the maneuver, either by rejecting it or not responding, the maneuver is not initiated.
Loading the video requires your consent. If you agree by clicking on the Play icon, the video will load and data will be transmitted to Google as well as information will be accessed and stored by Google on your device. Google may be able to link these data or information with existing data.
Mission accomplished: cooperative traffic planning is technically possible
“The project demonstrated that cooperative planning in mixed traffic situations is technically possible,” says Benjamin Völz. By using the simulation model in combination with reproducible test drives at the real intersection, he and his team at Bosch Research have made a significant contribution to the project outcome. Other road users, such as cyclists, were not yet integrated in the current experimental setup. But they were already detected by the vehicle sensors and infrastructure sensors, which means that they could be integrated into cooperative traffic planning in the future, as could pedestrians with smartphones that are logged into the network. Based on the project’s successful outcome, advance developments are being planned in the various Bosch divisions so as to expand cooperative transport planning. For example, there are considerations as to how this can be achieved without infrastructure sensor technology, which is possible at intersections but cannot be guaranteed in all road traffic situations. The foundations have been laid for even greater safety in mixed traffic situations.
Benjamin Völz
Benjamin Völz earned his diploma degree from the Faculty of Electrical and Computer Engineering, Dresden University of Technology, and his Ph.D. degree from the Department of Mechanical and Process Engineering, ETH Zurich. He is a research engineer at Bosch Research, focusing on planning for urban automated driving. His research interests include scene analysis, prediction, decision making, and planning for automated vehicles.
