In order for an autonomous car to be safe on the road, it is essential to be able to predict the behavior of other road users. A CSAIL research team at MIT (Massachusetts Institute of Technology) with researchers from the Institute for Interdisciplinary Information Sciences (IIIS) at Tsinghua University in Beijing has developed a new ML system that could one day help driverless cars predict the next nearby movements predict motorists, cyclists and pedestrians in real time. They titled their study: M2I: from factored marginal orbit prediction to interactive prediction”.
Qiao Sun, Junru Gu, and Hang Zhao are the IIIS members who participated in this study, while Xin Huang and Brian Williams represented MIT.
People are unpredictable, which actually makes it very difficult to predict the behavior of road users in urban areas. The currently used AI solutions are too simple: for them, for example, a pedestrian can stay on the same sidewalk without trying to cross it. If they expect pedestrians to cross, the robot simply parks the car. Some only predict the movements of a single road user.
Share to better predict
Trajectory prediction is widely used by intelligent driving systems to infer future movements of nearby agents and identify risky scenarios to enable safe driving. For the team, the existing models are excellent for predicting the marginal trajectories of individual agents, but fail to provide an answer for traffic in urban areas where many users interact, as the prediction space increases exponentially with their number.
MIT researchers have devised what appears to be a very simple solution to this complex problem: they break down a multi-agent behavior prediction problem into several small parts, and then attack each one individually so that a computer can solve this complex task in real time. They called this approach M21. Their behavior prediction framework first estimates the relationships between two road users: which car, cyclist, or pedestrian has the right-of-way, and which agent yields the right-of-way… It then uses these relationships to predict the future trajectories of multiple agents.
M21’s estimated paths proved to be more accurate than other ML models when compared to actual traffic flow in a huge dataset compiled by self-driving company Waymo (the MIT technique even outperformed Waymo’s recently released model). In addition, breaking the problem into sub-problems allowed them to use less memory.
Xin “Cyrus” Huang, a graduate student in the Department of Aerospace and a research associate in the lab of Brian Williams, a professor of aerospace and a member of the Computer Science and Artificial Intelligence Laboratory (CSAIL), co-lead author of the study, says :
“It’s a very intuitive idea, but nobody’s fully explored it yet, and it works pretty well. Simplicity is definitely a plus. We compare our model to other leading models in this space, including Waymo, the leader in this space, and our model achieves the best performance on this difficult benchmark. That has a lot of potential for the future. »
The M21 method
In this work, the researchers exploited the underlying relationships between interacting agents. M21’s algorithm has two inputs: the past trajectories of cars, cyclists and pedestrians interacting in a traffic environment such as a four-way intersection, and a map of road locations, lane configurations, etc
Using this information, a relationship predictor infers which of the two agents has the right to pass first, classifying one as the passer and the other as the giver. Then a prediction model, called the marginal predictor, estimates the trajectory of the passing agent because that agent behaves independently.
A second prediction model, known as the conditional predictor, then estimates what the yielding agent will do based on the actions of the passing agent. The system predicts a number of different trajectories for the dealer and the setter, calculates the probability of each individually, and then selects the six common outcomes with the highest probability of occurring.
The M2I method provides a prediction of the trajectory of these agents for the next eight seconds. It can slow down a vehicle to allow a pedestrian to cross the street and then accelerate as they pass the intersection. In another example, the vehicle waited for several cars to pass before turning off a side street onto a busy main street.
Waymo Open Motion Dataset Tests
The researchers trained the models on the Waymo Open Motion dataset, which contains millions of real-world traffic scenes involving vehicles, pedestrians, and cyclists recorded by sensors and lidar (light detection and ranging) cameras mounted on the company’s autonomous vehicles . They only kept the scenes involving multiple agents.
They then compared each method’s six prediction samples, weighted by their confidence level, to the actual trajectories followed by cars, cyclists, and pedestrians in a scene. Your method was the most accurate. M21 also outperformed the base models on a metric known as the overlap rate; if two trajectories overlap, this indicates a collision. M2I had the lowest overlap rate.
Xin Huang says:
“Rather than just create a more complex model to solve this problem, we took an approach that is more akin to how a human thinks when contemplating interactions with others. A human doesn’t think about all the hundreds of combinations of future behaviors. We make decisions pretty quickly. Another benefit of M2I is that it makes it easier for a user to understand the model’s decision making because it breaks the problem down into smaller parts. In the long term, this could help users trust self-driving vehicles more. »
On the other hand, the framework cannot account for cases where two agents influence each other, for example, when two vehicles are each moving forward at a four-way stop because the drivers do not know which should give way. The team intends to address this limitation in future work. She also hopes to use her method to simulate realistic interactions between road users that will verify self-driving car planning algorithms or generate massive amounts of synthetic driving data to improve model performance.
Article sources: “M2I: From Factored Marginal Trajectory Prediction to Interactive Prediction” by Qiao Sun, Xin Huang, Junru Gu, Brian C. Williams and Hang Zhao. March 28, 2022, Informatics Robotics.