The invention discloses a method and a
system for determining binocular scene flow based on semantic segmentation. Firstly, the scene in binocular image is semantically segmented, and the semantic
optical flow is calculated by adding semantic segmentation
label information, then the disparity information is calculated by semi-
global matching algorithm, and then the motion parameters of countless
facet regions are calculated and optimized by combining the semantic
optical flow and disparity information fitting. In the process of optimization, the initial scene flow is obtained by
superpixel segmentation, and then the motion of the superpixel blocks in the semantic tag is optimized, so the motion of the superpixel blocks in the semantic tag is consistent and the edge information of the moving object is protected well.
Semantic information is added into the
optical flow information, so that the edge of the object is protected, and the reasoning process of the
occlusion problem is greatlysimplified. In addition, the motion
inference at the semantic tag level makes the scene flow of pixels on the surface of the same moving object approximately consistent, and ultimately the goal of optimizing the scene flow is achieved.