Open source component duplicate checking method, system and device and readable storage medium

Abstract

The invention discloses an open source component duplicate checking method, which comprises the following steps of: manufacturing a plurality of matching templates, wherein each matching template comprises a plurality of matching frames, and the matching frames of different matching templates are different in size and number; performing local cutting on the open source component through the matching box of each matching template, extracting feature information of each local part of the open source component under each matching template, and storing each piece of feature information; carrying out local cutting on the comparison file by adopting at least one of a plurality of matching templates to obtain feature information of each local part of the comparison file under the matching template; and comparing the feature information of the open source component under the same matching template with the feature information of the comparison file to obtain similarity. According to the method, the code level can be positioned to a certain extent, and hardware consumption and duplicate checking time consumption can be comprehensively considered; and a corresponding matching template can be selected according to a specific application scene. In addition, the invention further discloses an open source component duplicate checking system and device and a computer readable storage medium.

Description

technical field [0001] The invention relates to the technical field of data analysis and processing, in particular to an open source component plagiarism checking method, system, device and computer-readable storage medium. Background technique [0002] In order to avoid intellectual property disputes, open source components need to be checked for plagiarism before using them. At this stage, there are mainly two ways to check the duplicates of open source components: Method 1, matching code line by line, this method can be located at the code level and has a high accuracy of duplicate checking; however, due to the huge amount of code in the open source library , in order to improve the matching efficiency, it is usually operated by loading memory, which leads to the problems of excessive hardware consumption and long matching time. Method 2: Use the hash value of the file itself or md5 to perform file-level matching. This method matches quickly, but because it cannot locate...

AI Technical Summary

Problems solved by technology

At this stage, there are mainly two ways to check the duplicates of open source components: Method 1, matching code line by line, this method can be located at the code level and has a high accuracy of duplicate checking; however, due to the huge amount of code in the open source library , in order to improve the matching efficiency, it is usually operated by loading memory, which leads to the problems of excessive hardware consumption and long matching time

Method 2: Use the hash value of the file itself or md5 to perform file-level matching. This method matches quickly, but because it cannot locate the code level, the accuracy of duplicate checking is low.

Images