The invention provides a vision-based multifunctional method for identifying hand gestures, which comprises the following steps of: firstly, extracting a hand part region from the background by background subtraction; secondly, carrying out fingertip detection on the extracted hand part region to determine the number of single fingertips contained in the hand gestures; meanwhile, acquiring the outline of the hand part region and describing the outline of the hand part region through improved shape context descriptors; afterwards, introducing a directed acyclic support vector fleet to classify the extracted improved shape context descriptors; after classifying, comparing the number of fingers of a hand form, to which the improved shape context descriptors belong, with the detection result of the fingertip, if the number is matched with the result, outputting the final judgment result; and otherwise, refusing the judgment. The vision-based multifunctional method can be better applied to man-machine interaction, mobile equipment and sing language input.