46results about How to "Reduce the number of IOs" patented technology

The invention discloses a large-scale social network service-oriented graph data storage and query method. A data storage manager stores received graph data in a Key-Value way, wherein the vertex ID (Identity) of the graph data is taken as the Key, a vertex neighbourhood is taken as the Value, and the data of each vertex neighbourhood is stored; and a plurality of edges connected with the vertex neighbourhood are orderly stored in a memory block with a fixed size via a timestamp, a double linked list is formed, and the attribution information and the index information of the vertex are stored in a data structure. When the data storage manager receives an access request that the vertex v is accessed, the data storage manager transmits the vertex v and a k-order neighbourhood of the vertex v to a requester; and the requester caches stored data locally, firstly checks local cache during next query, and sends the access request to the data storage manager if no queried vertexes exist. The graph data storage and query method can meet a scene which is dynamically updated and is suitable for processing data sparseness, and random access.

A method for aggregation optimization of temporal data includes dividing a temporal database file into a data block file and a block index file; scanning the block index file according to the calculated start and end time period, extracting the data blocks that meet the time period condition and other filtering conditions, and sorting according to the offset of each data block in the data block file; scanning the data files inexpensively by sorted data blocks, after the calculation of each data block, gathering the calculation results. The method can be completed by opening a data file for scanning whether it is for reading a single time series data or for aggregating multiple time series data, which greatly improves the overall performance.

An embodiment of the invention provides a method and a device for real-time statistics of logs. The method comprises the following steps of configuring a cluster statistics rule; carrying out rule matching on on-line logs and recording to-be-statistical information which meets the cluster statistics rule; receiving the recorded to-be-statistical information which conforms to the cluster statistics rule at every statistical period; carrying out cluster statistics operation on the received to-be-statistical information; storing a statistical result subjected to the cluster statistics operation; and querying the stored statistical result according to a received query request and showing a query result. The method and the device have the advantages of high statistical performance, low hardware requirements and capability of greatly saving the hardware cost.

The invention discloses a file reading method and system of distributed storage syste, an electronic device and a computer-readable storage medium. That method comprise the steps of: when an IO request of a client is received, determining a read-ahead upper limit accord to a data size of the IO request; Sending a read request with a size of the upper limit of the pre-read to the object storage device each time, and storing a return result of the read request into a memory until all data corresponding to the IO request are read; Returning data corresponding to the IO request stored in the memory to the client. Thus, the file reading method provided by the application can reduce the interaction times between the client and the object storage device, reduce the IO times of the disk, improve the sequential reading performance, thereby improving the performance of the entire storage system and enhancing the product competitiveness.

The present invention provides a virtual IO inter-chip connection circuit for FPGAs. The circuit comprises a clock module, a sending FPGA end circuit, a transmission module and a receiving FPGA end circuit. The sending FPGA end circuit comprises a data coding module for generating coded data; a first asynchronous FIFO which caches coded data written by the data coding module according to a coding clock and an asynchronous FIFO write protocol; and a sending module which sends data of at least two bits and a differential serial clock read from the first asynchronous FIFO until all the coded data is sent. The receiving FPGA end circuit comprises a receiving module for receiving the data of at least two bits and the differential serial clock until all the coded data is received; a second asynchronous FIFO which caches the data of at least two bits written by the receiving module according to the asynchronous FIFO write protocol and the differential serial clock until all the coded data is written; and a data decoding module which decodes the coded data synchronously. According to the invention, the transmission bandwidth and the transmission rate can be increased arbitrarily within a considerable range while the bit error rate can be reduced greatly.

The invention discloses an image display method and device, a storage medium and an electronic device. The method comprises the following steps: target playing time corresponding to a target operatingposition where target operation is performed on a playing progress bar is acquired from a multimedia resource under the condition of detecting the target operation performed on the playing progress bar of the multimedia resource; a target zone corresponding to the target playing time in a target image is determined, wherein the target image comprises a plurality of zones which correspond to a plurality of playing time of the multimedia resource and comprise the target zone, and the plurality of playing time comprise the target playing time; and the image on the target zone is displayed. The technical problem of relatively low image display efficiency in the related art is solved.

The embodiment of the invention discloses a cache file recovery method and device, electronic equipment and a storage medium. The method comprises the following steps of: when an executable file is loaded into the memory, judging whether the executable file is an important cache file or not according to the attribute information of the executable file, marking the executable file if the executablefile is the important cache file, judging whether the residual capacity of the memory meets a preset condition or not, and recycling the unmarked cache file in the memory if the residual capacity ofthe memory does not meet the preset condition. When a user uses the electronic equipment, important cache files and high-frequency access files can be generated in a memory of the electronic equipment, and the unimportant cache files and the non-high-frequency access files are recovered by judging the residual capacity memory of the memory, so that the important cache files are reserved, and the access speed of the files is increased.

The invention discloses an FPGA intra-IO sheet interconnecting digital circuit based on re-encryption algorithm. The FPGA intra-IO sheet interconnecting digital circuit comprises a sending end digital circuit, a receiving end digital circuit and a clock generation module, wherein the sending end digital circuit comprises a plaintext data generation logic, a first asynchronous FIFO, a head synchronous LFSR encryption module, a second asynchronous FIFO, an AES encryption module, a parallel-serial conversion module, a serializer and an OBUFDS module, and the receiving end digital circuit comprises a deseriallizer, an IBUFGDS module, a serial-parallel conversion module, a third asynchronous FIFO write control logic, a third asynchronous FIFO, an AES decryption module, a fourth asynchronous FIFO, a head synchronous LFSR decryption module and a plaintext data reception logic. Encryption and decryption of interconnected data transmission processes between FPGA virtual IO sheets are realized through stable and effective re-encryption algorithm.

The invention discloses a cache writing method. The method comprises the following steps: according to the file index arrangement order in the LRU table, the cached target strip file is read from theSSD; judging whether the number of file fragments of the target strip file is greater than a preset threshold value; if so, the target strip file is aggregated to obtain an aggregated data block; write the aggregated block of data to the HDD via an IO at a time and remove the target stripe file from the SSD. Compared with the mode that each file segment is written into the HDD by IO once at present, the method provided by the embodiment of the invention can make full use of the fast reading and writing performance of the SSD, can improve the writing performance of the storage system and improve the user experience. The invention also discloses a cache writing device, a device and a readable storage medium, which have corresponding technical effects.

The embodiment of the invention provides a big data write-in method and device, a big data read method and device and a big data processing system. The write-in method includes the steps that a write-in request is received; big data are segmented into multiple pieces of subdata according to the write-in request; the subdata are uniform in length, and relative positions of data parts, corresponding to the same field, in the subdata are fixed; the subdata are written in a specified storage medium; storage positions, in the storage medium, of current subdata are added to all the subdata. The big data can be positioned easily, conveniently and rapidly, time consumption is reduced, the speed of storing and reading the big data is increased, and system performance and working efficiency are improved.

The invention relates to the technical field of blockchains, and discloses a multi-version data storage method and device, computer equipment and a storage medium. The method comprises the steps of receiving a modification instruction for specified node data of an MVB tree; copying the first tree node data on the specified node data index path to obtain copied second tree node data; and modifyingthe second tree node data according to the modification instruction, and generating a modified version of the MVB tree according to the node data not in the index path and the modified second tree node data. According to the invention, the storage efficiency of the blockchain data can be improved.

The invention discloses a method for calculating fund estimation values and risk indexes in real time, and the method comprises the steps: S1, obtaining a data source of a transaction system, and building a calculation data model; S2, grouping and loading the data sources according to the service; S3, carrying out parallel computing on the grouped and loaded data; Ss4, aggregating and summarizingresults of the parallel computation; and S5, generating an estimation result set or a risk control result set according to the business scene. The distributed big data framework is utilized, parallelcomputing of data can be easily achieved, the computing capacity of multiple computers can be utilized based on a distributed computer system, and the processing speed is higher than that of a single-machine system.

The invention discloses a backboard interface SMI and UART multiplexing method, and can solve the technical problems that in the design process of an existing router, the number of main control singleboard connectors needing to be designed and the number of layers of a backboard are large, and the design is complex. Based on a main control single board, a backboard unit and an interface single board, the method comprises the following steps that S100, an interface card is powered on, and whether a CPU exists on the single board or not is judged through a Uart interface of a CPU; S200, a body_ on signal of the interface single board and an SMI register are set; S300, the main control logic receives the body _ on signal of the interface single board, and an SMI interface is started to access an interface single board SMI register; And S400, whether the backboard interface is an SMI interface or not according to the value of the register is judged, the backboard interface is set as a Uart interface and the SMI interface is closed if not. According to the backboard interface SMI and UART multiplexing method, the number of connectors of a backboard main control slot position can be reduced, and the design of an interface single board cpu is reduced.

The invention provides an edge side message delivery method and apparatus, and an electronic device. The method comprises the steps of obtaining a working condition message and a to-be-confirmed message sent by an edge side device; performing data processing on the working condition message, and generating a feedback message for confirming reception; and updating a message database according to the to-be-confirmed message and a feedback message confirmed to be received; A data retransmission request is generated based on unacknowledged received messages in the message database. According to the method, whether the working condition message is successfully sent or not is checked on the cloud platform, if not, the cloud platform generates the data retransmission request, even if network flash occurs when the cloud platform sends the data retransmission request to the edge side equipment, the cloud platform can send the data retransmission request again due to the fact that the working condition message is not successfully sent, and the data retransmission request is sent to the edge side equipment. According to the method provided by the invention, the reliability of edge side message delivery is greatly improved, the IO frequency of edge side equipment data storage is reduced, and the system performance required by the edge side equipment is reduced.

The invention provides a bitmap storage method. The bitmap storage method includes the steps that stripes of a redundant array of inexpensive disks raid are evenly distributed according to the number of CPUs, and multiple blocks corresponding to the CPUs one to one are acquired, wherein each block comprises multiple raid stripes; multiple bit pairs are selected from a memory and used for representing the use states of the blocks, and a region bitmap is acquired, wherein the bit pairs comprise first bits and second bits, the first bits are used for representing whether the blocks are written or not, and the second bits are used for representing whether the blocks are initialized. By the adoption of the bitmap storage method, the IO number of the stripe blocks to which a computer has access for the first time after the computer is restarted can be reduced, the first-time IO write speed is higher after the computer is restarted, when the computer is restarted, the initialization and reconstruction progress is kept to a large degree, and multiple restarts are allowed in the reconstruction or initialization process; the storage region of the bitmap is saved, and then the probability that the bitmap in superblocks breaks down is reduced.

The invention provides an alarm information association method and device, electronic equipment and a computer readable storage medium, and relates to the technical field of network monitoring. The alarm information association method comprises the following steps: when alarm information is collected, carrying out vectorization processing on the alarm information to obtain an alarm vector; obtaining an alarm association model matched with the alarm vector from the alarm association probability model set based on the alarm event represented by the alarm vector; acquiring a plurality of pieces of alarm association information of the alarm vector, and establishing an association relationship between the alarm vector and the plurality of pieces of alarm association information according to the alarm association model; generating an alarm association tree according to the association relationship, and pushing the alarm association tree to the monitoring terminal. According to the technical scheme of the invention, the alarm association tree can be generated in real time based on the alarm information and the alarm association model collected in real time, and the alarm monitoring experience and the alarm response timeliness can be improved.

The invention relates to the field of block chains, and discloses a data storage method and device, computer equipment and a storage medium.The method comprises the steps of newly establishing a transaction, wherein the transaction comprises a plurality of adding instructions; before the transaction is finished, adding a newly-added temporary node according to the adding instruction; after the transaction is finished, splitting the newly added temporary node into a plurality of split nodes, and calculating hash values of the split nodes; updating the key value pair list of the target father node, and judging whether the length of the key value pair list of the target father node exceeds a second preset threshold value or not; and if yes, continuing to split the target father node until thekey value pair list lengths of all nodes do not exceed respective corresponding preset thresholds. Through asynchronous processing of transaction processing and node splitting and coupling of node splitting and hash calculation, the transaction processing efficiency can be improved, the IO frequency of paths and index values of a transaction type Merkel B + tree is reduced, and the disk read-write time is shortened.

The invention relates to the technical field of data processing, and provides a caching method, system and device for small block input and a storage medium, and the method comprises the steps: S110,integrating multiple write requests belonging to an object into an IO set belonging to the object; S120, transmitting the integrated IO set to a placement strategy group; S130, integrating the IO setsin the placement strategy group; and S140, performing caching processing on the metadata of the object of the IO set in the integrated placement strategy group. By utilizing the method and the device, the IO quantity of PG processing can be reduced, CPU resources are released, and the data processing efficiency is improved.

The invention discloses a system and method for fast detaching a virtualized disk image. The system comprises a partition table analysis module, a disk analysis module, a file distribution table extraction module, a partition analysis module, a conflict resolution module and a file distribution table write-in module. The partition table analysis module is used for analyzing a target partition and a virtualized disk file physical address. The disk analysis module is used for judging whether a virtualized disk file is within the range of the target partition or not based on the physical address, if not, an error is reported, quitting is carried out, and if yes, the virtualized disk file is analyzed, and a file partition table and a file distribution table are acquired; the file distribution table extraction file is used for judging whether a file in the file partition table is stored in a virtualized disk as it is or not based on the file distribution table and recording file distribution records of files capable of being directly extracted; the partition analysis module is used for analyzing the target partition to acquire the position of a target partition file distribution table; the conflict resolution module is used for comparing the file distribution records with file names in the target partition, and if the file names are identical, file distribution record file names are modified; the file distribution table write-in module is used for writing the file distribution records in the target partition based on the target partition file distribution table and deleting original files.

The invention discloses an access request processing method and device. The method comprises the steps of receiving a plurality of access requests, wherein the access requests are used for accessing astorage bucket; sequentially storing access logs generated by each access request into a cache region, and carrying out merging processing; and writing the current merging result in the cache regioninto the corresponding log storage region according to a preset storage mode in the process of storing the generated access log into the cache region. The technical problem of performance reduction caused by consumption of a large amount of disk IO performance when access requests are frequent in a processing method of the access requests in related technologies is solved.

The invention discloses a transaction log playback method and device of a distributed core system, equipment and a medium. The method comprises the steps that in the desensitization processing process, a log processing sequence number is obtained according to a transaction request timestamp field in a transaction log; wherein if the transaction request time of different transaction logs is the same, the corresponding log processing sequence numbers are the same; according to the log processing parameters, processing the log processing sequence number to obtain a log playback sequence number; wherein the log playback sequence numbers are continuous numerical values, and each log playback sequence number corresponds to one transaction log. Compared with the prior art, the embodiment of the invention is suitable for a scene with a large transaction volume in a distributed core system, the success rate of transaction log playback is improved, and the automatic test efficiency and coverage rate are improved.

The invention relates to the technical field of artificial intelligence, and discloses a thread pool management method, device, equipment and medium suitable for a distributed technology, and the method comprises the steps: calling a target application, determining to-be-cached thread pool configuration data from Spring at a preset sleep time interval, and updating a target local cache, acquiring target thread pool working data according to a preset monitoring time interval, updating the server, determining target thread pool configuration data from a target local cache, and determining a queue early warning judgment result according to the target thread pool working data and the target thread pool configuration data, when the queue early warning judgment result is exceeded, generating alarm information to be stored according to the queue early warning judgment result and issuing the alarm information to the server; calling the client to consume the alarm information to be stored in the server; and calling an early warning platform to determine a thread pool statistical result according to the target thread pool working data of the server. Unified management and monitoring of the custom thread pool in the application are realized.

The invention discloses a system and method for fast detaching a virtualized disk image. The system comprises a partition table analysis module, a disk analysis module, a file distribution table extraction module, a partition analysis module, a conflict resolution module and a file distribution table write-in module. The partition table analysis module is used for analyzing a target partition and a virtualized disk file physical address. The disk analysis module is used for judging whether a virtualized disk file is within the range of the target partition or not based on the physical address, if not, an error is reported, quitting is carried out, and if yes, the virtualized disk file is analyzed, and a file partition table and a file distribution table are acquired; the file distribution table extraction file is used for judging whether a file in the file partition table is stored in a virtualized disk as it is or not based on the file distribution table and recording file distribution records of files capable of being directly extracted; the partition analysis module is used for analyzing the target partition to acquire the position of a target partition file distribution table; the conflict resolution module is used for comparing the file distribution records with file names in the target partition, and if the file names are identical, file distribution record file names are modified; the file distribution table write-in module is used for writing the file distribution records in the target partition based on the target partition file distribution table and deleting original files.

A method and system for maintaining tree-structure based directory relationships is provided. The system includes: a change detection unit, used to receive a change request from outside or inside, and set a change flag and record change information related to the change request according to the change request; a polling task unit, used to check polling change flag; query and tree assembly unit, used to assemble tree structure after deduplicating the change information according to the change information; and calculation unit, used to generate according to the tree structure assembled by the query and tree assembly unit directory tree.

The invention provides a bitmap storage method. The bitmap storage method includes the steps that stripes of a redundant array of inexpensive disks raid are evenly distributed according to the number of CPUs, and multiple blocks corresponding to the CPUs one to one are acquired, wherein each block comprises multiple raid stripes; multiple bit pairs are selected from a memory and used for representing the use states of the blocks, and a region bitmap is acquired, wherein the bit pairs comprise first bits and second bits, the first bits are used for representing whether the blocks are written or not, and the second bits are used for representing whether the blocks are initialized. By the adoption of the bitmap storage method, the IO number of the stripe blocks to which a computer has access for the first time after the computer is restarted can be reduced, the first-time IO write speed is higher after the computer is restarted, when the computer is restarted, the initialization and reconstruction progress is kept to a large degree, and multiple restarts are allowed in the reconstruction or initialization process; the storage region of the bitmap is saved, and then the probability that the bitmap in superblocks breaks down is reduced.

The invention discloses a database query method and system, and belongs to the technical field of data query; the method includes, based on a KV storage engine, storing data queried from a disk by a database according to a SELECT statement defined by a user in the KV storage engine; defining metadata for a data set queried by a user, and recording data information; recording metadata state information of the batch of queried data at different time nodes through a state recorder, and recording metadata changes caused by operation into the state recorder when the data are operated every time. The problems that a traditional database query method is low in large-batch data query efficiency, large-batch data memory occupation is high, system performance is affected, and the traditional database query method is not suitable for scenes where the same batch of data is repeatedly used can be solved. The performance of repeatedly using the query data is remarkably improved, the disk loss is reduced, the function is expanded, and the operation convenience is improved.

The embodiment of the invention provides a data retrieval method and a server. The method comprises the steps: obtaining a core data table in data tables in a database; parsing a query sentence of thecore data table; obtaining a grouping reference column according to a first preset rule and the parsing result; and according to a second preset rule and the grouping reference column, obtaining an index constructed based on the core data table, and performing data retrieval according to the index. The server performs the method described above. The data retrieval method and the server provided by the embodiment of the invention can greatly reduce the IO quantity of the database and the storage space, thereby remarkably improving the efficiency of the overall operation of the system.

The embodiment of the invention provides a big data write-in method and device, a big data read method and device and a big data processing system. The write-in method includes the steps that a write-in request is received; big data are segmented into multiple pieces of subdata according to the write-in request; the subdata are uniform in length, and relative positions of data parts, corresponding to the same field, in the subdata are fixed; the subdata are written in a specified storage medium; storage positions, in the storage medium, of current subdata are added to all the subdata. The big data can be positioned easily, conveniently and rapidly, time consumption is reduced, the speed of storing and reading the big data is increased, and system performance and working efficiency are improved.

The invention belongs to the field of wireless communication, and particularly relates to a reconfigurable electromagnetic metasurface biasing method, which at least comprises an array surface formed by a reconfigurable electromagnetic metasurface unit and a biasing circuit unit, and is characterized by comprising a DAC (Digital-to-Analog Converter) module (9), an array bottom surface unit (8), an array top surface unit (10) and a Pin tube (1), and the array bottom surface unit (8) and the array top surface unit (10) are connected through a lead and a via hole. The reconfigurable electromagnetic metasurface biasing circuit is simplified, the number of biasing lines is from o-n2 to o-n, the number of IO interfaces occupied by a rear-end circuit is reduced, and the influence of the biasing lines on the electromagnetic property of a unit is reduced; and the number and the positions of the current-limiting resistors are reduced, and extra microwave loss caused by the current-limiting resistors is avoided.