The invention discloses an image restoring method and device. The method comprises the following steps of: step A, obtaining a first layer region to be restored of an image damaged region from inside to outside, and reading structural information and a color of the known image region which is close to the first layer region to be restored; step B, carrying out gradient analysis and color estimation on the first layer region to be restored according to the structural information and the color of the known image region to obtain gradient information and a color of the first layer region to be restored; step C, taking the gradient information and the color of the first layer region to be restored as initial values to be extended to the first layer region to be restored; step D, carrying out texture synthesis restoration on the first layer region to be restored according to the initial values to obtain a first restoring result; and step E, taking the first restoring result as the known image region to circularly execute the steps A to D until the image damaged region is restored. With the adoption of the image restoring method and device, disclosed by the invention, the structural region can be accurately positioned, the block matching accuracy is improved, and the introduction and the reproduction of error information are reduced.