The invention discloses an ultra-precise displacement measuring system based on an optical neural network. The system comprises a light source, an optical displacement measuring device, an optical neural network, a detector array and a signal processing device. When a target object moves, the system takes a measurement optical signal output by the optical displacement measuring device as a signalinput, the signal is received by the detector array after being processed by the optical neural network, and finally the signal is converted into displacement information of the target object throughthe signal processing device. The invention further discloses an ultra-precise displacement measuring method based on the optical neural network. The optical neural network is used for processing themeasurement optical signal, so that the displacement of the target object can be directly measured; the phase discrimination process of the electronic signal is not needed; the response speed is extremely high; the size can be zoomed; the energy utilization rate is high; and the system is suitable for ultra-precise measurement occasions with high speed and high dynamic performance requirements. According to the system, the multi-degree-of-freedom pose measurement of the target object also can be realized by increasing the number of input measurement optical signals and the number of detector arrays.