Disclosed in the invention is a programmable visual chip-based visual image processing system, comprising an image sensor and a multilevel parallel digital processing circuit. The image sensor mainly includes a pixel array, an analog preprocessing circuit array and an analog-to-digital conversion circuit array; and the digital processing circuit consists of a parallel processing unit array with an M*M pixel level, a parallel processing unit array with M*1 rows, an on-chip artificial neural network and a reduced instruction processor dual-core subsystem. According to the provided system, high quality image collection with high speed and multilevel parallel image processing are realized and several high-speed intelligent visual application can be realized by programming; and compared with a traditional image system, the provided system has advantages of high speed, high integration, low power consumption and low cost. Moreover, the invention brings forward an embodiment for realizing the above-mentioned system as well as several high-speed intelligent visual image processing algorithms based on the embodiment. High-speed motion detection, high-speed gesture identification and rapid face detection are included; and the processing speed can reach 1000 frames per second. Therefore, a requirement of high-speed real-time processing can be met.