An apparatus and a method for semantic image labeling

Abstract

Disclosed is a method for generating a semantic image labeling model, comprising: forming a first CNN and a second CNN, respectively; randomly initializing the first CNN; inputting a raw image and a plurality of predetermined label ground truth annotations to the first CNN to iteratively update weights of the first CNN so that a category label probability for the raw image, which is output from the first CNN, approaches the predetermined label ground truth annotations; randomly initializing the second CNN; inputting the category label probability to the second CNN to correct the input categorylabel probability so as to determine classification errors of the category label probabilities; updating the second CNN by back-propagating the classification errors; concatenating the updated firstCNN and the updated second CNN; classifying each pixel in the raw image into one of a plurality of general object categories; and back-propagating classification errors through the concatenated CNN toupdate weights of the concatenated CNN until the classification errors less than a predetermined threshold.

Description

technical field [0001] The present disclosure relates to a device and method for image semantic annotation. Background technique [0002] Markov Random Fields (MRF) or Conditional Random Fields (CRF) have achieved great success in semantic image annotation, which is one of the most challenging problems in computer vision. Existing work can generally be divided into two groups based on the definition of unary term and pairwise term of MRF. [0003] In the first group, researchers improved by exploring a wealth of information to define pairwise functions, including long-range dependencies, high-order potentials, and semantic labeling contexts. labeling accuracy. For example, Kimi Raikkonen ( Kr¨ahenb¨uhl) et al. Accurate segmentation boundaries are obtained by inferring on fully connected graphs. Vineet et al. Fully connected graphs are extended by defining high-order and long-range terms between pixels. The global or local semantic context between tags has also been stu...

AI Technical Summary

Problems solved by technology

Directly combining the above CNN and MRF is inefficient, since CNN typically has millions of parameters, while MRF also needs to infer thousands of potential variables; and worse, incorporate complex binary terms into MRF is impractical, thereby limiting the performance of the overall system

Images