33 results about "MinHash" patented technology

The invention discloses a DCGAN (Deep Convolutional Generative Adversarial Network)-based spectral imagery secure retrieval method, and belongs to the field of spectral imageries. According to the method, the features of a spectral imagery are highly expressed by utilizing a DCGAN; and a new encrypted domain spectral imagery secure retrieval method is proposed. Firstly the deep spectral-spatial features of the spectral imagery are jointly extracted by utilizing the DCGAN, and the contents of the spectral imagery are accurately represented; in order to ensure the security in a remote sensing image retrieval process, the deep features are encrypted by adopting a Min-Hash method based on a criterion that the similarity of the encrypted features is unchanged, thereby protecting the deep features; and finally under the non-decryption condition, Jaccard similarity distance measurement is performed on image features directly by comparing the number of same Min-Hash values, and images similar to a query image are returned. Therefore, the information security is ensured while the retrieval is realized.

The invention provides an Eclat-based multivariate time series association rule mining method. the method comprises the steps of 1, generating a perpendicular dataset; 2, generating a MINHASH matrix,wherein the MINHASH matrix needs a designated parameter k; 3, utilizing the MINHASH matrix for estimating a candidate item set in an original data set; 4, according to the minimum support, pruning thecandidate item set to obtain frequent item sets 1; 5, combining two Hash frequent item sets 1 and generating a new frequent item set 2; 6, repeatedly executing the step 5 till combination cannot be performed, and ending an algorithm. The association rule mining speed is remarkably increased, the purpose of obtaining the time series data analysis result in time is achieved, even though the miningprecision is lowered, the mining efficiency can be greatly improved, and the machine memory can be saved.

The invention relates to the technical field of big data, and discloses an intelligent recommendation method and device, computer equipment and a readable storage medium, and the method comprises thesteps: obtaining user information, and carrying out the characterization of the user information to obtain a user vector; calling a product quantization process to segment the user vector to obtain aplurality of sub-vectors, identifying the category to which each sub-vector belongs, and summarizing the categories to obtain a user category set; calling a minimum hash process to perform similaritycomparison on the user category set and each reference category set in a preset index library, and setting the reference category set of which the similarity exceeds a preset similarity threshold as atarget category set; and taking the associated information corresponding to the target category set as recommendation information. According to the method, the fineness and the accuracy of user vector category identification are improved, the operation efficiency of the server is improved, the matching speed between the user information and the reference information in the index database is increased, and the data calculation amount and the data storage amount are reduced.

The embodiment of the invention provides an illegal request identification method and device, and the method comprises the steps: receiving an access request for a certain interface, and analyzing aninterface identification from a uniform resource locator URL of the access request; carrying out minimum hash signature calculation on the URL, so as to obtain a MinHash signature of the URL; and determining the similarity between the MinHash signature of the URL and the MinHash signature of the target illegal URL corresponding to the predetermined interface identifier, and if the similarity is not less than a set similarity threshold, determining that the access request is an illegal request. According to the technical scheme provided by the invention, the URLs are converted into the minimumhash signatures to compare the similarity between the URLs, so that the illegal request identification efficiency is greatly improved.

The invention discloses a multi-label learning design method based on a hashing method. Through the combination of a hashing algorithm and a multi-label learning algorithm based on Bayesian statistics, the correlation between labels is effectively utilized so as to improve the predicting performance of a multi-label learning model, labels and neighbors of the labels are introduced to computation of the posterior probability through the characteristics of the neighbors, the correlation between the labels is fully considered, and the accuracy of the algorithms is improved; the problem that the label space in multi-label learning of large-scale data is higher in dimension and sparse is solved through an MinHash algorithm; the purpose of learning large-scale data is achieved by finding the neighbors through locality sensitive hashing (LSH), the neighbors can be rapidly and efficiently found, and the expandability of the multi-label learning algorithm is improved.

The embodiments disclosed in this application provide a method and system for detecting open-source components in mixed-source software. Wherein, the method includes: obtaining the source code file in the target mixed-source software, that is, obtaining the first source code file, and classifying the first source code file and performing corresponding same-source analysis; wherein, for the first source code file Source code files whose size exceeds the first threshold are subjected to homology analysis based on the Simhash algorithm; for source files whose size does not exceed the first threshold among the first source files, homology analysis is performed based on the Minhash algorithm. Compared with the existing technology, the above scheme can balance the contradiction between the efficiency requirements and the accuracy of open source component detection of mixed-source software, and obtain acceptable open source component detection results under the premise of ensuring detection efficiency.

The invention relates to a method for merging and optimizing based on Lucene index segments, and belongs to the technical field of computer indexing. It includes the following steps: combining the load information of the current node and the segment information of the index, constructing a merge analysis module to judge whether the merge condition is satisfied. According to the dictionary files contained in each index segment, the feature matrix of the index segment in the index is obtained, and then combined with the minHash algorithm and the minimum hash signature algorithm to calculate the signature matrix of the index segment. Combined with the signature matrix of the index segment and the Jaccard similarity principle, the similarity coefficient between each index segment is calculated, and the index segment is divided into different similar sets according to the similarity coefficient. Use the similarity evaluation model to score each similar set, and sort according to the set score, and select one or more sets with the highest score to be merged by the merge thread. The optimization method of the invention can reduce the impact of the merge operation on the performance of the index function and the retrieval function and can effectively improve the speed of retrieval.

The invention discloses a method and a device for acquiring similar object sets, and relates to the technical field of computers. A specific implementation of the method includes: obtaining a set of target objects and a set of objects to be similar; setting a locally sensitive comparison step size r; Feature data, local sensitivity comparison step length r, obtain the similar object set of the target object from the object set to be similar. This implementation method adopts the local sensitivity-minimum hash value algorithm to obtain the similar object set of the target object from the set of similar objects to be similar, which overcomes the Hive SQL distributed method that only compares objects with a certain same attribute and misses most of them. It reduces the complexity of calculation, speeds up calculation, and improves the accuracy of calculation results and the coverage of similar objects.

The invention discloses a malicious program visual detection method based on deep learning. The method comprises the steps of running a malicious program and constructing a data flow graph of the malicious program; extracting feature sub-graphs from the data flow graph to form a sub-graph corpus; mapping the sub-graphs in the sub-graph corpus into character strings; vectorizing the sub-images by adopting a deep learning algorithm; carrying out hash calculation on the sub-graph vectors contained in the malicious program by adopting a minHash algorithm to construct a visual picture matrix of themalicious program; constructing a classification model; and for a to-be-detected program, constructing a visual picture matrix of the to-be-detected program, and classifying the visual picture matrixof the to-be-detected program by adopting the classification model to obtain a classification result. The method is easy to implement, high in reliability, good in practicability and accurate in detection result.

The invention discloses a webshell malicious family clustering analysis method, and relates to the technical field of information security. The method comprises the following steps: step 1, obtaining function call information, parameter values and return value information during Webshell operation; step 2, cleaning, splicing and sequencing the function call information; 3, vectorizing the function call sequence information in the step 2; 4, calculating information entropies of the parameter values and the return values, and sorting according to a function calling sequence; 5, according to the func_seq, the argv_seq and the return_seq obtained in the step 2 and the step 4, building an RNN model to predict the three types of sequences respectively, and learning code family features; 6, after minhash processing is carried out on the original sequence data and the predicted sequence data, mapping the original sequence data and the predicted sequence data into pixel points, and thus, a pixel map is formed; 7, superposing the original pixel image obtained in the step 6 and the predicted pixel image, and drawing a final pixel image; and step 8, clustering the pixel image obtained in the step 7 by using a DBSCAN clustering algorithm.

The invention discloses a method and device for obtaining similar object set and providing similar object set. The method comprises as follows: obtaining input file comprising M objects, N attributes, attribute values corresponding to each attribute; inputting each attribute to first level of pre-created minimum hash function minhash, obtaining the returned value of the first level of minhash of each attribute; according to each attribute, weighted value corresponding to the attribute in the current object and the second level of pre-created minhash function, obtaining the returned value of the second level of the minhash of each attribute; calculating the combined minhash value of each attribute in each object respectively; determining the minimum value of the combined minhash value corresponding to each attribute of the same object as the minhash value of the object; circularly executing the operation to each object for K times, respectively obtaining K minhash values in allusion to each object; inputting K minhash values of each object to the locality sensitive hashing (LSH) computing framework. The method and device are capable of improving the operating efficiency, and improving the validity and accuracy degree of the similar object information.

The invention relates to the technical field of image processing and in particular relates to a latent semantic min-Hash-based image retrieval method comprising the steps of (1) obtaining datasets through division; (2) establishing a latent semantic min-Hash model; (3) solving a transformation matrix T; (4) performing Hash encoding on testing datasets Xtest; (5) performing image query. Based on the facts that the convolution network has better expression features and latent semantics of primitive characteristics can be extracted by using matrix decomposition, minimizing constraint is performed on quantization errors in an encoding quantization process, so that after the primitive characteristics are encoded, the corresponding Hamming distances in a Hamming space of semantically-similar images are smaller and the corresponding Hamming distances of semantically-dissimilar images are larger. Thus, the image retrieval precision and the indexing efficiency are improved.

An embodiment of the invention discloses an index creating method and device. The method includes: extracting feature words of a target text; sorting the feature words to obtain a feature character string; applying the MinHash algorithm to the feature character string to obtain a hash value corresponding to the target text; searching a mapping buffer pool to determine whether an index mapping barrel matched with the hash value exists or not, if yes, creating an index between the hash value and the target text in the index mapping barrel, and if not, establishing the index mapping barrel matched with the hash value, and creating the index between the hash value and the target text. By adoption of the index creating method, index storage quantity is decreased; by creation of indexes of similar texts in the same index mapping barrel, classification of the similar texts is realized, and similar text retrieval speed is increased.

The invention discloses a method and a device for generating text fingerprint information. The method comprises the steps as follows: an initial feature vector of a text is extracted; a weighted value of at least one element in the initial feature vector is endowed with a multiple value of the minimum weighted value; the weighted values of the other elements are endowed with the minimum weighted value; the corresponding element is added to the initial feature vector according to the multiple to form a new feature vector; and the text fingerprint information is generated after minimum hash algorithm is carried out on the new feature vector. The method and the device for generating the text fingerprint information disclosed by the invention can improve the accuracy of fingerprint information, so that information clustering has a relatively excellent effect.

The invention provides a method and a system for identifying homologous binary files in a database. The database comprises multiple binary basic files. The method comprises the steps of obtaining signatures of to-be-identified files and signatures of the basic files according to a min-hash algorithm; for any signature, performing bucket dividing processing on the signature according to a bucket dividing method; according to a reverse indexing method and the signatures, subjected to bucket dividing, of all the basic files, obtaining dictionaries in one-to-one correspondence with buckets, wherein each dictionary comprises at least one key value pair; according to character strings in the buckets of the to-be-identified files, traversing the corresponding dictionaries, and according to valuescorresponding to matching keys, obtaining the homologous binary files of the to-be-identified files. According to the method and the system, the signatures are obtained by adopting the min-hash algorithm and the bucket dividing is performed by adopting a local sensitive hash algorithm, so that the calculation amount can be remarkably reduced; and by adopting the reverse indexing method, an indextable is established for all the signatures, so that the speed of identifying the homologous binary files is increased.