Original document determining method and device, electronic device, and storage medium

Abstract

The embodiment of the invention provides an original document determining method and device, an electronic device, and a storage medium. The method includes: receiving a to-be-processed document and calculating the simhash signature of the to-be-processed document by using a simhash algorithm; obtaining the simhash signature of a first original document from a preset original document library, calculating the similarity between the to-be-processed document and the first original document, obtaining, when the similarity between the to-be-processed document and the first original document is greater than a first preset threshold, the first original document and comparing the full text of the first original document with that of the to-be-processed document; obtaining the same content in thefirst original document and the to-be-processed document, calculating the proportion of the same content in the first original document; and marking the to-be-processed document as a non-original document when the proportion of the same content in the first original document is greater than or equal to a second preset threshold, or marking the to-be-processed document as an original document whenthe proportion of the same content in the first original document is less than the second preset threshold. The original document determining method and device can improve the accuracy of determiningthe original document.

Description

technical field [0001] The present invention relates to the technical field of data processing, in particular to a method, device, electronic equipment and storage medium for judging an original document. Background technique [0002] With the development of Internet technology, the dissemination of information is more convenient and faster. Various good documents often appear on the Internet. When users see a good document, they often like to reprint the good document, or republish it as their own document on the Internet after partially modifying the document. However, good documents are often obtained by the author through his own hard work. Others republish the document after simple modifications. This is undoubtedly a disrespect for the author's labor results. [0003] In order to judge whether the target document published by the user is an original document, in the prior art, word segmentation is often performed on the original document and the target document to obt...

AI Technical Summary

Problems solved by technology

[0004] However, the quality of keywords obtained by word segmentation processing will directly affect the result of similarity calculation. When the target document is subjected to word segmentation processing and the keywords obtained are relatively poor, the similarity calculation results between the original document and the target document will be different from the actual ones. large deviation

Images