The invention relates to the technical field of image processing, and provides a surface defect detection method and device. The method comprises the following steps: firstly, extracting each bottom-layer feature of a to-be-detected surface defect image to obtain each feature image pyramid, determining each feature image corresponding to each feature image pyramid according to a center peripheral difference mechanism, normalizing each feature image, and adding the feature images of the same type to obtain each feature saliency map; fusing the feature saliency maps by taking the energy proportion of each feature saliency map as a weight to obtain a synthetic saliency map, then extracting, fusing and sampling high-level features of the surface defect image to obtain a high-level saliency map; and finally, taking the energy proportion of the synthetic saliency map and the energy proportion of the high-level saliency map as a weight, fusing the synthetic saliency map and the high-level saliency map to obtain a total saliency map, and determining defect types and defect positions of the surface defect image according to the total saliency map. According to the invention, the recognition precision of surface defect detection is improved.