Duplicate checking method

Abstract

The invention provides a duplicate checking method. The method comprises the following steps of the first step, using a Word2Vec model to train to obtain a sentence vector of an original sentence anda contrast sentence, wherein the sentence vector is obtained by integrating a word vector and a word characteristic vector; the second step, based on the sentence vector of the original sentence and the sentence vector of the contrast sentence, calculating to obtain the included angle between the sentence vector of the original sentence and the sentence vector of the contrast sentence; the third step, determining the similarity between the original sentence and the contrast sentence, wherein when the included angle is less than or equal to a threshold, it is determined that the original sentence is similar to the contrast sentence; when the angle is greater than the threshold, it is determined that the original sentence is not similar to the contrast sentence. The method comprehensively considers the word vector and the word characteristic vector, compared with the calculation time based on sentence coding, the calculation time of the method is obviously shortened, the introduction ofthe word characteristic vector has a certain solution effect on the situation that checking is difficult after synonym replacing, the problem that word changing, order changing and adding or deletionof words are difficult to check based on complete sentence comparison is solved, and on the whole, the method not only improves the accuracy of duplicate checking, but also improves the efficiency ofthe duplicate checking.

Description

technical field [0001] The invention relates to the technical field of computers, in particular to a method for checking duplicates. Background technique [0002] As people pay more and more attention to science and technology and social development, the academic field presents a trend of diversification, informationization and modernization. Under such circumstances, we urgently need efficient, comprehensive and convenient academic information more than ever. But on the other hand, cases of academic misconduct still exist. Academic misconduct refers to the atmosphere of falsification, bad behavior or anomie in the academic world, or refers to some people plagiarizing the research results of others in the academic field, corrupting the academic atmosphere, hindering academic progress, and violating the scientific spirit and morality. Academic misconduct abandons the principle of authenticity and integrity of scientific experimental data, which has brought serious negative ...

AI Technical Summary

Problems solved by technology

However, most of the existing methods of document plagiarism check aim at whether the texts are the same or how much they are the same. Most of the repeated comparisons are based on the feature word segmentation or the complete sentence as the minimum unit. There are problems such as long calculation time and difficulty in checking duplicates after replacing them with synonyms. Although the calculation time of the sentence-based duplicate checking calculation method is shortened, it is difficult to find situations such as changing the order of words, changing words, or adding or subtracting words. Therefore, document duplicate checking has a broad scope. space for research and application

Images