341results about How to "Reduce data redundancy" patented technology

The present invention discloses parallel mass data transmitting middleware of Internet of things and a working method thereof. The invention includes: a data collection module, for forwarding data collected from different types of sensors to an application interaction module after analysis; the application interaction module, for achieving communication between the parallel mass data transmitting middleware of the Internet of things and an application system of the Internet of things, analyzing a control command of the application system of the Internet of things, transmitting to the data collection module, and further transmitting the analyzed sensor data to a data management module; the data management module, for performing data screening, fusion, conversion on the mass data analyzed by the application interaction module, and finally storing the calculated data into the application system of the Internet of things; and a developing regulation module, for setting out a protocol plug-in regulation, a communication channel plug-in regulation, a service field metadata plug-in regulation, etc. The present invention can meet high concurrency access and management of the large scale sensing apparatus of different application fields, various communication channels and various communication protocols.

The invention discloses a traffic-road condition information processing method based on a vehicle wireless sensor network. The method is characterized in that: firstly, a wireless communication network is established and the network includes an information center, a vehicle wireless sensor network sub-system formed by limited vehicle mobile knots and an information publishing sub-system based on a broadcasting network; the information center is fixed in a traffic hub in a city as the center of the wireless communication network; secondly, each vehicle mobile knot collects the timely road condition information and integrates data through the vehicle wireless sensor network sub-system according to the instruction of the information center; then the timely road condition information from the information collection and data integration of each road section is reported to the information center through the vehicle wireless sensor network sub-system; finally, the processed traffic road condition information and network control instructions are published by the information center through the information issuing sub-system to each vehicle mobile knot for the road choosing by drivers.

The invention discloses a file processing method, which comprises the following steps of: encoding a file into a plurality of data blocks by utilizing an erasure code algorithm, and storing the plurality of data blocks in at least two storage nodes which at least comprise storage nodes in a distributed file system; wherein one data block is only stored once; and storing the file index of the file,the respective storage address of each data block in the plurality of data blocks and the hash value related to the file in a block chain network. Therefore, the data redundancy is greatly reduced, higher data reliability is obtained with lower data redundancy, and the utilization rate of the storage space is improved. Moreover, the method adopts a mode of combined storage of the block chain network and the out-of-chain distributed storage system, so that the storage pressure of the block chain network is reduced, and the storage capacity is expanded by introducing the distributed file system. The invention further discloses a file processing device, equipment, a medium and a computer program product.

The invention is suitable for the field of storage and provides a file uploading method and device, and a mass storage system. The method comprises the following steps: a terminal device generates summary information corresponding to the file to be uploaded by using a preset summary information generating algorithm according to the file to be uploaded, and uploads the summary information to a storage server; the storage server searches whether the summary information uploaded by the terminal device exists or not; and when the storage server does not find the summary information, the terminal device uploads the file to be uploaded to the storage server. In the embodiment of the invention, the terminal device needs to upload the file to the storage server only when the file to be uploaded does not exist in the storage server, thereby reducing the data redundancy in the storage server, saving the network resources while uploading files and increasing the file uploading speed.

The invention discloses a two-stage single-instance data de-duplication backup method. De-duplication of data at two stages is performed during backup. The method comprises firstly, performing repeated data detection on a file scale, inquiring local logs to judge whether identical files are stored or not, and if the identical files are stored, informing users to complete the backup operation; if identical files are not stored locally, informing backup programs of server ends to inquire databases to judge whether files with identical content exist or not, if the files with identical content are searched, establishing links pointing to the files for clients only, and recording quotes of the files by clients by the server ends; if the files are new, uploading the files and recording information of the files by two ends; further processing the files after the files are uploaded to the server ends by background programs, and splicing small files together to avoid waste of space; storing large files respectively by type, comparing similar files regularly, and performing difference de-duplication at the second stage after grouping.

The invention discloses a method and a system for intrusion detection. Cluster analysis is carried out on a test sample set through a graphical partition method, then the behavior type of cluster obtained through the cluster analysis on the test sample set is determined by utilizing a training sample set, and thus the determination of the behavior type of the test sample set is realized. Since the determination process of the behavior type of the test sample set does not completely depend on the training sample, contained in a training sample set, owned behavior type, a new behavior type can be found by adopting the scheme provided by the embodiment of the invention.

The invention provides a component quality checking device based on line laser three-dimensional measurement and detection method of the device. The quality checking device includes a mobile platform mounted on a guide rail and capable of sliding along the guide rail at a uniform speed. To-be-checked components are fixed on the mobile platform. A camera and a plurality of laser emitters are mounted above the guide rail through a mount. The camera collects images which are casted to the surfaces of the to-be-checked components by the laser emitters and deformed due to surface modulation. The camera transmits the collected image data to a computer for analysis and treatment. According to the detection method, by casting linear stripes emitted by the laser emitters to the surface of the to-be-checked components for linear scanning and by utilizing the camera for collecting image information on the to-be-checked components continuously, measured three-dimensional point cloud data and initially designed three-dimensional image are compared and analyzed in the same coordinate system for judging whether the maximum error is within the maximum permissible error range or not and screening out qualified components. The device provided by the invention is high in detection precision and realizes automatic and flow line production.

The invention discloses a geographic information data fusion system which comprises a geographic information data interface unit, an information exchange platform unit and an information application front-end processor unit. The geographic information data interface unit is used for classifying and storing input geographic information data according to a geographic information data classification table. The information exchange platform unit is used for reading the geographic information data in the geographic information data interface unit according to a preset refresh cycle, converting the geographic information data into geographic information data in a preset format and outputting the geographic information data. The information application front-end processor unit is used for extracting personal information contained in the geographic information data output by the information exchange platform unit, obtaining activity track information corresponding to the personal information from the geographic information data output from the information exchange platform unit, setting access permissions of the personal information, creating fusion geographic information data containing the access permissions and the personal activity track information, setting personal information identifiers for the fusion geographic information data and storing the fusion geographic information data with the set personal information identifiers. By means of the geographic information data fusion system, data fusion sharing application can be achieved.

Vector quantization provides font contour data reduction. Characters are defined in a font collection using segmented outline forms and common references stored to curve segments for similar segments in different characters of the font collection. Compressing the font collection preferably includes applying an entropy encoding data reduction. The common references are stored in a segment storage location. Another aspect includes decompressing a font store containing a compressed collection of outline fonts. The uncompressed curve segments are scaled so as to match a requested size for a letterform.

The invention provides a data inquiry method and system, consensus method and system, apparatus and storage medium. Each node of a block chain network respectively stores a Merkel tree of each block,and a global index table corresponding to the Merkel tree of each block. The version number of the version is a block height, wherein the query instruction includes the queried first block height; a target data is look up in that global index table according to the height of the first block. By configuring the corresponding Merkel tree, The global index table with the block height as the version number, and searching the target data in the global index table through the queried block height, can complete the query of the sub-node data of Merkel tree leaf in the block chain through one read operation, thus greatly improving the data reading performance of the system.

The invention discloses a traffic safety management (TSM) information multihop broadcasting method. The method comprises the following steps of: detecting a traffic safety event vehicle to serve as a source node to broadcast TSM information; calculating a reference waiting time slot Tn of a highway section in which a node is positioned by the node which receives the TSM information according to a weight factor, namely the distance between the node and the source node, dynamically regulating additional forwarding waiting time Delta n according to an estimated value of link quality signal to interference plus noise ratio (SINR), and calculating and acquiring waiting time tn of the node for transmitting a message; and when the waiting time tn is reduced to zero, immediately broadcasting request traffic forward (RTF), after a previous forwarding node receives the RTF, determining a single relay node to transmit a data packet, and immediately forwarding the received TSM information after the node receives the RTF. By the method, link quality information is combined, data forwarding reliability is fully guaranteed, the phenomenon of forwarding collision between multiple nodes in the same highway section is reduced at the same time, and the link load and data redundancy in a media access control (MAC) layer are effectively reduced based on an RTF / authorized traffic forward (ATF) forwarding confirmation mechanism.

In a system and method for facilitating wireless communication between agents in a wireless network, each agent is equipped with an agent's device (10a, 10b) using a main wireless infrastructure (2) and a separate auxiliary wireless infrastructure (1). Initial contact between any first agent's device (10a) and any other agent's device (10b) is made within the auxiliary wireless infrastructure, whereas, once this initial contact has been established, all ensuing communications between the first agent's device (10a) and the other agent's device (10b) take place within the main wireless infrastructure.

The invention provides a data storage method and a system, an apparatus and a storage medium. The method comprises the following steps: a global index table corresponding to a Merkel tree of each block is stored locally at each node of a block chain network; and a Merkel tree of each block is stored locally. Where the version number of the version is the block height, and the global index table isused to query data according to the block height. By configuring the corresponding global index table for the Merkel tree and taking the block height as the version number, the invention provides a data inquiry way for the system to inquire the global index table directly through the block height and complete the inquiry through one reading operation, thereby greatly improving the data reading performance of the system. And further optimizes the robustness of the system in the event of transaction congestion or node crash.

The present invention relates to a method or system of generating a surrogate key using cryptographic hashing. One embodiment of the method of the present invention may have steps such as selecting a field or group of fields that is or are unique among all records in the database, for each record, extracting the data from the fields, concatenating the extracted data into an input message, running the input message through a hash generator, either in batches or one at a time, for testing purposes perhaps, and outputting a surrogate key.

The present invention discloses an optical flow computation method using a time domain visual sensor, and specific implementation steps are given out. Different from a current method for performing optical flow computation by adopting "a frame sequence image (video)", the method disclosed by the present invention uses a visual information acquisition device-time domain visual sensor to perform optical flow computation of a field of view. Furthermore, because each pixel autonomously performs change detection and asynchronous output, motion and change of an object in a scenario can be sensed in real time, a continuity assumption of "no change in brightness and no change in local speed" in a differential optical flow computation method is well satisfied, and the real-time property of optical flow computation is greatly improved while the precision of optical flow computation is significantly improved. Therefore, the method is very suitable for optical flow computation and subsequent tracking and speed measurement of the high-speed moving object.

This invention discloses a method for re-transmitting data used in data re-transmission in the switch of HS-DSCH in the WCDMA system including: a radio network controller (RNC) gets the radio link control packet data unit (RLC PDU), which should be re-transmitted to the user terminal, the RNC sends the RLC PDU should be re-transmitted to the user terminal to the switched high speed media access control MAC-hs queue of the switched target area base station in terms of the sequence number from small to big in batches based on the pre-set or actual transmitted each time in the physical layer RLC PDU number then to re-transmit it to the user terminal.

The invention discloses a light stream optimization based three-dimensional reconstruction method and device. The method comprises the following steps of: collecting a depth picture and a color picture of a multi-viewing angle by utilizing a multi-viewing-angle depth camera, and carrying out three-dimensional spatial alternation so as to obtain a point cloud of each viewing angle; calculating measuring vectors of two groups of point clouds of adjacent viewing angles according to the information of the depth picture and the color picture; establishing a light stream optimization formula according to a multi-channel light stream method by utilizing the measuring vectors of two groups of point clouds of adjacent viewing angles; resolving the light stream optimization formula, thereby obtaining the matching result of two groups of point clouds of the adjacent viewing angles; filling each viewing angle point cloud into a fusion matrix according to the matching result; and processing the fusion matrix, thereby obtaining the reconstruction result. According to the invention, as less as possible of three-dimensional structures of point characterization models can be utilized self-adaptively, thus data redundancy is lowered; meanwhile, the model accuracy is high; and additionally, hardware requirement is not high, and the application range is wide.

The invention relates to a motion planning method based on driving habit learning. The method comprises the following steps: step 1, a cloud processing terminal collects manual driving trajectories, and through an image processing, extracts preview points with road traffic attributes; step 2, the cloud processing terminal delivers the preview points in front of a vehicle in real time according tothe state of a self-driving vehicle; step 3, the self-driving vehicle generates an algorithm based on a set path, generates an alternate trajectory cluster between the vehicle and each preview point,and selects an optimal path according to a cost function. Compared to the prior art, the motion planning method based on driving habit learning has the following advantages: using artificial driving trajectories to generate local planning paths and getting rid of dependence on high-precision maps; new attributes are added to the preview points, and the way of generating is changed, so that the path of the local planning is more in line with the driving habits of people.

The embodiment of the invention discloses a method and a device for synchronizing a compressed dictionary. The method comprises the following steps of: obtaining synchronizing information fields of dictionary entries corresponding to data to be sent of the compressed dictionary; judging whether network equipment at a receiving terminal has the dictionary entries according to the synchronizing information fields, if the network equipment at a receiving terminal does not have the dictionary entries, synchronizing the dictionary entries by using the network equipment at the receiving terminal to ensure that the network equipment at the receiving terminal can uncompress the compressed data to be sent according to the dictionary entries after receiving the data to be sent, which is subjected to redundancy data removal and compression according to the dictionary entries. According to the embodiment of the invention, data redundancy can be decreased by reducing the synchronization for the invalid dictionary entries of the equipment.

The invention provides a multi-cluster cooperative data processing method and system, a memory medium and a computer device. The method comprises the steps of receiving collection data and convertingthe collection data into structured data; establishing a basic data sharing area on a Hadoop cluster and loading the structured data to the basic data sharing area; and cooperatively processing the data in the basic data sharing area through utilization of the Hadoop cluster, a Hadoop online processing cluster and a plurality of distributed database clusters according to a data application scene,thereby providing corresponding data service and finishing a data processing process. According to the method, the system, the memory medium and the computer device, the data redundancy can be reducedthrough establishment of the basic data sharing area.

The invention relates to a system and a method for trajectory tracking during circulation of an E-mail. The method includes: when a sender sends an E-mail, a unique identifier is generated for each receiver, and the receiver accesses the E-mail theme through the unique identifier; when each receiver's operation on the E-mail is detected, the operation trajectory is fed back to the sender; after the information feedback, the sender can accurately learn about whether the corresponding receiver reads the E-mail or not, whether the corresponding receiver forward the E-mail or not, whether the corresponding receiver download an attachment or not, whether the corresponding receiver print the attachment or not, whether the corresponding receiver edit the attachment or not, E-mail reading times, E-mail reading time, E-mail forwarding times, E-mail forwarding time, who the E-mail is forwarded to, attachment reading times, attachment reading time, attachment downloading times, attachment downloading time, attachment editing times, attachment editing time, attachment printing times, attachment printing time and the like. By the method, comprehensive E-mail trajectory tracking can be performed, and E-mail transmitting efficiency and safety are increased by the information feedback.

The invention relates to a wireless sensor network technology and a data acquisition field, and discloses a wireless multi-point equalized low-power-consumption power monitoring system and method. The system comprises a wireless monitoring terminal, common control equipment and upper management software. The innovation point of the system and method lies in that: 1) each wireless monitoring terminal is connected with more than two sensor units, wherein each wireless monitoring terminal uploads an equilibrium value to the common control equipment after equalizing and comparing data acquired by multiple sensor units, thus, the inconvenience in replacing the terminal equipment caused by the damage of the sensor unit is avoided; 2) a special calling mechanism is adopted in each wireless monitoring terminal, and each wireless monitoring terminal is in a low-power-consumption standby state most of time and automatically turns into a working state only after receiving a monitoring command; and 3) the common control equipment selectively sends the received data according to a preset condition. According to the invention, the redundant data in the system is reduced, the safety monitoring efficiency is increased, and the low-power-consumption problem in the wireless monitoring is solved.

A compression dictionary synchronization method and device are provided. The method includes: obtaining a synchronization information field of a dictionary entry in a compression dictionary corresponding to data to be sent; judging, according to the synchronization information field, whether a network device at a receiving side has the dictionary entry; and synchronizing the dictionary entry with the network device at the receiving side if the network device at the receiving side does not have the dictionary entry, so that after receiving the data which is to be sent and is compressed by data redundancy elimination according to the dictionary entry, the network device at the receiving side can decompress the compressed data to be sent according to the dictionary entry. Data redundancy is decreased by reducing useless dictionary entry synchronization between devices.

Said method includes 1, dividing semiconductor processing into real time recording data and state monitoring data.2, actively transmitting real time recording data and PMC and / or TMC to CTC at stated time intervals in technology treating processes, or actively querying to PMC and / or TMC at definite-time by CTC. 3, to technology data only meaning equipment momentary state and used for CTC judging setting system working, CTC querying to PMC and / or TMC as in need. When CTC is in no request, PMC or TMC do not transmit these data information to CTC, the network transmission data quantity is greatly reduced, thereby reducing network transmission load and raising network transmission rate.

The invention discloses a high-performance remote sensing image synthesis method. The method comprises the steps that original remote sensing image data is acquired, and a cloud covered area in the original remote sensing image data is extracted; mask clipping is performed on the remote sensing image data obtained after extraction, and cloudless fragment image data is obtained and stored; a task area to be synthesized is divided into blocks to obtain multiple subtask areas; the cloudless fragment image data in the subtask areas is subjected to substitutive inlaying parallelization processing according to cloudless inlaying rules, and synthesized cloudless image data blocks in all the subtask areas are output; and the cloudless image data blocks acquired from all the subtask areas are gathered to output a synthesized cloudless remote sensing image of the whole task area. According to the method, only a cloudless fragment image is stored, so that the data storage volume is lowered; substitutive inlaying processing is performed according to areas in a multithread parallel mode, thus, efficiency is improved, and a large-range cloudless inlaid image can be acquired; by means of local histogram matching, the problem of hue inconsistency is eliminated while hue adjustment is guaranteed.

The invention discloses a network distributed storage and reading method for a file. The method comprises the following steps of: determining the storage disaster recovery parameter index (n, k, t and sigma) of a storage system (N1, N2,..., Nn); constructing a file subblock matrix Gn*k and a file aggregation matrix Hn*(n-k); equally dividing the file into k data blocks after the file is transformed into a binary format; carrying out information coding on the k data blocks of the file according to the file subblock matrix Gn*k, thereby obtaining n coded data blocks; and storing the n coded data blocks into n different storage nodes of the storage system. Compared with the prior art, the method has the advantages that the complete information of the file can be read and recovered through randomly selecting any n-t storage nodes from the n storage nodes of the storage system, namely, the condition that any t storage nodes of the system are simultaneously broken down can be tolerated, so that the disaster recovery capability of distributed data storage can be remarkably improved, and the dependence on data redundancy and equipment redundancy of the storage system is reduced.

The invention discloses a data processing method. The method comprises the steps of coding a data block with a predetermined size to generate an error correction data block corresponding to the data block, wherein the data block comprises data objects, and the data objects include a keyword, a value and metadata; and generating a data block index and data object indexes, wherein the data block index is used for indexing the data block and the error correction data block corresponding to the data block, and the data object indexes are used for indexing the data objects in the data block, and each data object index is used for indexing only one data object.

The invention relates to the field of three-dimensional digital scene construction, provided are a virtual compound eye camera system for acquiring three-dimensional scene of a building satisfying spatio-temporal consistency and a working method thereof, the system includes a data acquisition module, positioning module and task assignment module, wherein the data acquisition module is composed ofall compound eye cameras facing the building object, and all compound eye cameras facing the building object are programmed according to a predetermined building acquisition grid, and the virtual group is formed into a complete and systematic compound eye system, which is called virtual compound eye; Virtual compound eye is composed of multiple compound eye cameras, which are programmed accordingto the acquisition grid of a given building, and are cooperated with each other to form a virtual compound eye. The virtual compound-eye camera system and the working method thereof can realize real-time shooting and real-time production with consistent time and space, and obtain more accurate and real dynamic three-dimensional virtual scene.