The invention relates to a complex curved face grinding and polishing system and method. The system comprises a three-degree-of-freedom manipulator, a visual detection device, a vibrating sensor and a three-degree-of-freedom tool platform. An installation clamp is arranged at the tail end of the manipulator and can install an oilstone, a polishing wheel and a grinding wheel. The visual detection device comprises a laser generating device and a binocular camera, is installed on the manipulator, and detects the outline and roughness of a workpiece. During working, the workpiece is installed on the three-degree-of-freedom tool platform. The visual detection device guides the manipulator to scan the outline of the workpiece, a three-dimensional outline data model is generated, and the roughness is analyzed. Polishing or grinding setting can be conducted on a work station. During grinding, setting can be conducted in three modes that a referential three-dimensional data model is compared with an actually measured workpiece three-dimensional data model, modification is conducted on the actually measured three-dimensional data model with a drawing tool, and grinding data are directly set on the actually measured three-dimensional data model.