A real-time visual detection and identification method for high speed rail surface defects comprises the following steps of: (1) image acquisition; (2) image preprocessing; (3) defect preliminary detection, that is, performing logic or operation combination of detection results based on gray scale compensation with detection results based on top-hat operation, and detecting whether an abnormal area exists in an image, if not, finishing the detection, or else continuing the processing; (4) defect accurate positioning, that is, accurately positioning the defect by an algorithm of bonding single defect, an algorithm of filling holes in the defect area, and an algorithm of selecting the main defect, and extracting the defect area through marks; (5) defect classification, that is, extracting and selecting characteristics of the defect area, designing and training a BP neural network, and classifying the defects by the BP neural network. The invention has the advantages of simple principle, high automation degree, high detection speed, and high detection precision.