Get the latest updates on the Coronavirus impact on engineers.Click Here
Sensors, Vision

Motion prediction algorithms enhance safety features for automated vehicles

The Southwest Research Institute (SwRI) has developed a machine vision system that predicts pedestrian movements with a convolutional neural network (CNN).
By Southwest Research Institute (SwRI) February 24, 2020
Courtesy: Southwest Research Institute (SwRI)

Southwest Research Institute (SwRI) has developed a motion prediction system that enhances pedestrian detection for automated vehicles. The computer vision tool uses a deep learning algorithm designed to predict motion by observing real-time biomechanical movements with the pelvic area being a key indicator for changes.

“For instance, if a pedestrian is walking west, the system can predict if that person will suddenly turn south,” said SwRI’s Samuel E. Slocum, a senior research analyst who led the project. “As the push for automated vehicles accelerates, this research offers several important safety features to help protect pedestrians.”

Recent accidents involving automated vehicles have heightened the call for improved detection of pedestrians and other moving obstacles. Although previous technologies could track and predict movements in a straight line, they were unable to anticipate sudden changes.

Motion prediction often uses “optical flow” algorithms to predict direction and speed based on lateral motion. Optical flow, a type of computer vision, pairs algorithms with cameras to track dynamic objects. The accuracy of optical flow diminishes, however, when people move in unexpected directions.

To improve accuracy, SwRI compared optical flow to other deep learning methods, including temporal convolutional networks (TCNs) and long short-term memory (LSTM). After testing several configurations, researchers optimized a novel TCN that outperformed competing algorithms, predicting sudden changes in motion within milliseconds with a high level of accuracy.

The temporal design uses a convolutional neural network to process video data. SwRI’s novel approach optimizes dilation in network layers to learn and predict trends at a higher level. Dilated convolutions are structures that store and access video data for spatial observations.

People draw on experience and inference when driving near pedestrians and cyclists. SwRI’s research is a small step for autonomous systems to react more like human drivers.

SwRI’s human motion prediction system overlays sparse skeletal features on pedestrians to predict movement. The blue dot in this image predicts the future direction of pedestrian on right. Deep learning algorithms predict motion based on pelvic biomechanical movement. Courtesy: Southwest Research Institute (SwRI)

SwRI’s human motion prediction system overlays sparse skeletal features on pedestrians to predict movement. The blue dot in this image predicts the future direction of pedestrian on right. Deep learning algorithms predict motion based on pelvic biomechanical movement. Courtesy: Southwest Research Institute (SwRI)

“If we see a pedestrian, we might prepare to slow down or change lanes in anticipation of someone crossing the street,” said Dr. Douglas Brooks, a manager in SwRI’s applied sensing department. “We take it for granted, but it’s incredibly complex for a computer to process this scene and predict scenarios.”

The research team leveraged SwRI’s markerless motion capture system, which automates biomechanical analysis in sports science. Using camera vision and perception algorithms, the system provides deep insights into kinematics and joint movement.

Applications for the project include human performance, automated vehicles and manufacturing robotics. The algorithms can work with a variety of camera-based systems, and datasets are also available to SwRI clients.

– Edited from an SwRI press release by CFE Media. See more Control Engineering machine vision stories.


Southwest Research Institute (SwRI)
Author Bio: Southwest Research Institute (SwRI)