Similarity learning and association between observations of multiple connected vehicles

Abstract

In an example embodiment, a first vehicle platform includes a first sensor that has a first perspective directed toward an external environment and that captures first sensor data reflecting first objects, and a communication unit for receiving second sensor data from a second vehicle platform that reflects second objects included in the external environment. One or more computing devices extracta first set of multi-modal features from the first objects, and a second set of multi-modal features from the second objects in the second image, process the first set of multi-modal features and thesecond set of multi-modal features using separate machine learning logic to produce a first output and a second output, respectively, generate a similarity score using the first output and the secondoutput; and associate the first and second perspectives using the similarity score.

Description

technical field [0001] The present disclosure relates to learning representations of detected objects. In a more specific example, the present disclosure relates to techniques for similarity learning and correlation between observations of multiple connected vehicles. Background technique [0002] Object tracking and traffic situation localization often rely on capturing multiple observations of a road scene of the same object. However, identifying the same object included in these multiple observations is challenging. Existing solutions for associating objects across multiple observations are to extract features of the objects and use feature comparisons to match objects in different views. However, such existing methods generally only consider a limited number of predetermined features. Therefore, feature representations of objects are often incomplete and lead to inaccurate object associations. On the other hand, it is also impractical to use the full feature set to d...

AI Technical Summary

Problems solved by technology

However, such existing methods generally only consider a limited number of predetermined features

Therefore, the feature representation of objects is often incomplete and leads to inaccurate object associations

On the other hand, it is also impractical to use the full feature set to describe objects, since the full feature set cannot be efficiently sent over the vehicle network due to latency and bandwidth constraints.

Furthermore, existing solutions are generally implemented in centralized systems and thus are generally not suitable for distributed systems comprising multiple cooperating vehicles in a vehicle network

Images