50results about How to "Reduce I/O operations" patented technology

The invention discloses a method and system for obtaining a dynamic feed index. Better query performance can be obtained on the occasion of high concurrent query and large data amount of an index table. An index table cache is established by a storage structure by means of a key value, and a feed index in the index table cache is consistent with a corresponding feed index in a relation type database. The method includes that a client server sends a query request to the index table cache and the relation type database, and takes query results fed back by the index table cache and the relation type database as the obtained feed index.

The invention relates to a dynamic taint analysis device and application of the dynamic taint analysis device to document format reverse analysis, wherein the dynamic taint analysis device comprises a dynamic instrumentation executive logging module and a static snapshoot analysis taint tracking module, wherein the dynamic instrumentation executive logging module is used for calling and executing a tested program by using a binary program instrumentation platform, monitoring opening, analyzing and closing behaviors of a data document including original taint data in a process of executing the tested program, and acquiring snapshoot logs of all command processes, context information and memory access information in the process of executing the tested program according to the process of executing the tested program; the static snapshoot analysis taint tracking module is used for analyzing the snapshoot logs and simulating replay execution of a progress according to information obtained by analysis and recording processing and spread information of original taint data in the data document to obtain a taint data stream path. According to the dynamic taint analysis device, I / O (Input / Output), time and space overhead of dynamic taint analysis in a dynamic execution process can be reduced, an extended instruction set can be supported, and the continuity and incidence relation of original taint data can be obtained.

An embodiment of the invention discloses a data storage control method, a data storage control device, equipment and a storage medium. The data storage control method comprises the steps of: when to-be-stored target data is received, acquiring at least one item of data attribute information matched with the target data according to at least one preset filtering type; processing the data attributeinformation by adopting a filtering processing rule matched with the filtering type to generate a filtering comparison identifier matched with the target data; judging whether the filtering comparisonidentifier is stored in a pre-established identifier database or not; if so, performing data filtering on the target data; otherwise, storing the target data in a target database, and updating the identifier database according to the filtering comparison identifier. According to the data storage control method and the data storage control device of the invention, massive data can be cleaned according to user requirements, and meanwhile, the data is filtered before being stored in the database, so that IO resources are saved, and data storage control performance is improved.

The invention discloses a video storage and retrieval method, and the method comprises the following steps: obtaining a video stream, and converting the video stream into an H264 bare stream; identifying a key frame in the H264 bare stream; calculating the frame number of the key frame and the absolute deviation between the key frame and the initial position; establishing a key frame index queue according to the frame number of the key frame and the absolute offset; writing the key frame index queue into a storage file; the efficiency of video positioning, fast forward and slow backward is greatly improved, and the delay of dragging the video progress bar is reduced.

The invention relates to a maximum frequent item set mining method based on an adjacency list, and the method employs a storage mode combining the adjacency list and a Hash table to reduce the scanning frequency of a database and the spatial scale of traversal, can trim out an item set smaller than a support degree threshold as early as possible, and avoids the generation of all non-empty subsetsof a longer maximum frequent item set. According to the method, the established adjacency list is fully utilized, the original database only needs to be scanned once, and the method has the advantagesof low time complexity, memory consumption and the like.

Embodiments of the invention disclose a data processing method and apparatus. The method comprises the steps of obtaining multiple pieces of to-be-processed data; defining the to-be-processed data in a plurality of preset intervals; determining approximate values of the to-be-processed data in the intervals; and according to the approximate values, determining quantiles corresponding to the to-be-processed data. By utilizing the method and the apparatus, the approximate values are used for replacing the numerical values of the to-be-processed data defined in any interval, so that the quantity of data participating in a calculation process can be reduced. Therefore, the traversal frequency during quantile calculation can be reduced; the I / O operation on a database can be further reduced; and the efficiency of the quantile calculation process can be improved to a certain extent.

The invention discloses an LTE base station big data mining method and device based on memory-mapped files. The method comprises the steps of creating one memory-mapped file F1 for all base stations to store current period original counter information; storing original counter in F1 according to a predefined data structure M; creating a memory-mapped file F2 and copying original counter data in F1 into F2 when it is time for current period integrality statistics; ranking all M structures in F2, reading and comparing all M structures in F2, conducting count counting on the comparison result, and then calculating the integrality of the current index. By the adoption of the method and device, files stored on magnetic discs are processed with the memory-mapped files, I / O operation on files is reduced, original counter integrality calculation efficiency is improved, and platform memory consumption, expansion difficulty and corresponding cost are reduced.

The invention provides a method for dynamically updating the constant values of an application server. The method comprises the steps of: reading and analyzing constant files in a hard disk, and storing data in the constant files into a memory in a key-value mode, wherein the key represents a constant object, and the value represents a constant value; scanning the constant files in the hard disk, judging whether the constant files are changed or not, and if changed, reading the changed constant files; analyzing according to the key-value mode so as to obtain the changed constant objects and constant values; searching all the original constant values, which correspond to the changed constant objects, from the memory according to the changed constant objects; and clearing all the researched original constant values which correspond to the changed constant objects from the memory, and loading the changed constant values into the memory to replace the original constant values. The invention further provides a system for dynamically updating the constant values of the application server. By utilizing the method and the system for dynamically updating the constant values of the application server, the constant values of the application server can be updated in real time.

The invention relates to a dynamic taint analysis device and application of the dynamic taint analysis device to document format reverse analysis, wherein the dynamic taint analysis device comprises a dynamic instrumentation executive logging module and a static snapshoot analysis taint tracking module, wherein the dynamic instrumentation executive logging module is used for calling and executing a tested program by using a binary program instrumentation platform, monitoring opening, analyzing and closing behaviors of a data document including original taint data in a process of executing the tested program, and acquiring snapshoot logs of all command processes, context information and memory access information in the process of executing the tested program according to the process of executing the tested program; the static snapshoot analysis taint tracking module is used for analyzing the snapshoot logs and simulating replay execution of a progress according to information obtained by analysis and recording processing and spread information of original taint data in the data document to obtain a taint data stream path. According to the dynamic taint analysis device, I / O (Input / Output), time and space overhead of dynamic taint analysis in a dynamic execution process can be reduced, an extended instruction set can be supported, and the continuity and incidence relation of original taint data can be obtained.

The invention discloses a super-large scale image feature points matching method and system thereof. The method comprises the following steps: firstly, image matching pairs are obtained by caring out image near neighbor search; images are acted as nodes, the edges of constitution among the image neighbors form an undigraph, the undigraph is caring out priority sorting of breadth, then the images and image pairs are obtained; the feature information of the images are sorted again according to the sorting result, and the feature information of the images is divided and stored as binary files; the binary files with the feature information are read according to the order of sequence; according to the images after being sorted, the characteristics match is being carried out in sequence, and the subsequent useless feature information is released in time; the feature information is iteratively read and the feature matching is being carried out until all images finish matching. The method and system has the advantages that one read and local full both of image feature points in the process of image matching are ensured, memory is not spilled and at the same moment, the efficiency of algorithm is ensured.

The embodiment of the present invention provides a data interaction method and device. The method includes: if a data export instruction is received, searching for the mapping relationship with the relational database management system RDBMS in the target hard disk area by retrieving multiple mapping relationships recorded in the mapping list. The first internal file corresponding to the first external table in ; export the first internal file to the distributed file system HDFS; if a data import instruction is received, then determine the target hard disk by retrieving multiple mapping relationships recorded in the mapping table Whether there is already a second internal file to be imported from HDFS in the area; if not, import the second internal file into the target hard disk area, and generate a second external table in the RDBMS; establish a second external table in the mapping list The mapping relationship between the table and the second internal file. The invention effectively reduces the input / output operation of the RDBMS on the disk where the database table is located, thereby effectively ensuring the original business resources of the RDBMS and improving the processing efficiency and resource utilization of the RDBMS.

The invention discloses a method and system for transmitting packaged files based on a cloud computing environment. The method receives a command to transfer folders; obtains the file types and file sizes of all files in the folder; divides all files into files suitable for packaging and files unsuitable for packaging according to the file types; The files suitable for packaging are put into the packaging queue, and the files whose file size is smaller than the file threshold among the files that are not suitable for packaging are put into the packaging queue; among the files that are not suitable for packaging, the files whose file size is greater than or equal to the file threshold performing transmission; compressing and packaging the files in the packaging queue, and transmitting the packaged files formed after packaging. It greatly shortens the transfer time of the entire folder, improves the file transfer efficiency, and also reduces the I / O operations during the file transfer process, which can greatly improve the data throughput rate and service efficiency of the data center, and increase the service life of the server.

The invention discloses a high-speed digital printing method based on a multi-core processor and the DMA technology. The problem that an existing digital printing machine still cannot carry out real-time processing on a large number of images due to the limitations of the PC processor performance and parallel interface data transmission is solved. Firstly, a multi-core processor is used for quickening processing on printing commands and printing images, then the DMA technology is used for achieving the input / output operation mode that automatic in-batch data exchanging is carried out between a high-speed peripheral device and a main memory so as to reduce CPU interference to the greatest extent, the efficiency of a whole computer system is greatly improved, finally the RAID technology is used for improving the access speed of a hard disk, and therefore the efficiency of a whole digital printing system is promoted.

The invention discloses a key value storage method based on a log-structured merged tree. The method comprises: 1, setting a cache component in a memory for each data storage component, setting a Bloom filter for each cache component, and setting a Bloom filter for each data block of each SSTable; 2, carrying out jump judgment on each key value pair of current data blocks of to-be-merged data storage components Ci stage by stage by adopting the corresponding Bloom filter, wherein if the key value pair jumps to a C<i+M>, no version of key value pair corresponding to keys exists in the C<i+M> and each stage of component in the front of the C<i+M>, a B<i+M+1> comprises a corresponding version of key value pair and then the key value pair is set into a B<i+M>, and if the key value pair is jumped to a B<i+N>, no version of key value pair corresponding to keys in the B<i+N> and each stage of component in the front of the B<i+N> and a C<i+N> comprises a corresponding version of key value pairs; and 3, processing the to-be-merged key value pairs in the Bi by adopting a similarity method.