39results about How to "Reduce duplicate data" patented technology

A program product, an apparatus, and a method that identify a plurality of related electronic messages and combine the plurality of the related electronic messages into a composite view are provided. Also provided is a method that searches a first account of a first user to identify a plurality of related electronic messages stored in the first account and searches a second account of a second user to identify at least one other electronic message that is associated with the plurality. By doing so, a user may be provided with a more complete email chain, often leading to an increase in productivity, improved collaboration, and a reduction of duplicate data.

The embodiment of the invention discloses a data operation method, a data operation system, a client and a data server. The method comprises: transmitting a file writing request to the data server, wherein the writing request comprises marks of subdata blocks forming a file; receiving the corresponding relation between the subdata block marks which are returned by the data server and storage servers according to the writing request; and writing the subdata blocks into the corresponding storage servers according to the corresponding relation. The embodiment of the invention can judge whether the subdata blocks are written according to the fact whether the subdata block marks are stored, so that no repeated data can be stored in the system and the storage space of the system is improved.

An apparatus and method for transmitting / receiving data in a communication system are provided, in which an information symbol is repeated, the repeated information symbols are interleaved, the interleaved repeated information symbols are organized into groups of a predetermined size, an nth parity check matrix is calculated by modulo summation of an (n−1)th parity check symbol and all interleaved repeated information symbols of an nth group, and a codeword is generated by multiplying each of the information symbols by the parity check matrix and transmitting the codeword.

The invention discloses a data download system, a device and a method for download management, wherein the download management device comprises a buffer memory, a resolver, a finder and a source return device. The buffer memory is used for buffering each document and a content abstract of each document which are obtained from data source points, and the content abstracts are data which are generated by adoption of special data conversion of content of each document. The resolver is used for obtaining requests for document downloading from equipment of a client-side, and resolving the content abstract of the requested document according to a download link of the requests of the document downloading. The finder is used for finding the requested document in the buffer memory according to the document content abstracts. If the document content abstracts are found, the requested document is transferred to the cline-side equipment. The source return device is used when the finder can not find the requested document in the buffer memory, obtains the requested document from relevant data source points, transfers the requested document to the cline-side equipment, and offers the requested document to the buffer memory for buffering.

The invention relates to the technical field of data processing, in particular to a method, a device and a system for data processing. The method for the data processing includes providing a first interface which receives a file write operation request from an application program; portioning the data of a file into at least one sub-data block, obtaining an unique identification of each of the at least one sub-data block, and comparing unique identifications with a reserved unique identification; establishing a chaining between a sub-data block identical to the reserved unique identification and the data corresponding to the reserved unique identification if the unique identifications are identical to the reserved unique identification; and otherwise, reserving the unique identifications of the sub-data block, which are different from the reserved unique identification, sending a data write operation request including the data of the sub-data block, which are different from the reserved unique identification, to a storage system by means of a second interface, and enabling the storage system to store the data of the sub-data block. According to the method, the device and the system for the data processing, the repeated data are reduced, and the storage space is saved.

Data duplication may be reduced in cloud storage. First snapshots of one or more remote volumes may be received via a network. The first snapshots may be copies of the one or more remote volumes at a first instant in time. Responsive to and / or based on the first snapshots, unique clusters and duplicate clusters may be identified among the valid clusters of the remote volumes. The unique clusters and single instances of the duplicate clusters may be stored in a backup file, such that the backup file is devoid of duplicate clusters. Second snapshots of the one or more remote volumes may be received via the network. The second snapshots may be copies of the one or more remote volumes at a second instant in time, wherein the second instant in time is after the first instant in time. Responsive to the second snapshots, the clusters in the backup file that are no longer valid may be utilized to store the valid clusters in the one or more remote volumes not yet stored in the backup file.

The invention discloses a Bluetooth Mesh network transmission system and method implemented on the basis of routing tables. In the transmission system, a Bluetooth Mesh network comprises a plurality of nodes; each node comprises one routing table, and the routing table stores information of nodes near the current node; and when a certain node receives a message from the Bluetooth Mesh network, a target node is parsed out, and when a current routing table comprises information of the target node, the current node directly sends the message to the target node, so as to implement communication ofthe Bluetooth Mesh network on the basis of the routing tables. The invention aims to modify a Bluetooth Mesh algorithm, adding the routing tables on the basis and implement Bluetooth Mesh transmission based on the routing tables so as to fulfill the aims of reducing duplicated data in the network and improving effectiveness of network communication.

The invention discloses a monitoring data uploading method. The method comprises the following steps: when an application program is started, reporting device information of a device where the application program is located to a background server for one time and storing the device information; judging whether the volatile data of the current device changes; if so, generating a new associating code, and performing associated storage on the associating code and the changed volatile data; during the period when the volatile data is not changed, performing associated storage on all to-be-reportedevent monitoring data and the current latest associating code within the period; and during the data reporting, assembling and compressing related information in the stored device information, the volatile data subjected to associated storage with the same associating code and the event monitoring data to obtain a monitoring data packet and uploading the monitoring data packet to the background server. The invention further discloses a monitoring data uploading apparatus, device and system, and a storage medium. By adoption of the monitoring data uploading method, apparatus, device and system, the repeated uploading of the redundant data can be reduced while the integrity of the monitoring data is ensured, thereby saving the traffic for the user.

Example embodiments provide methods for handling the transition of a mobile device from a first network to a second network, the second network operating according to a different communications protocol from the first network. One example embodiment includes terminating forwarding of the data packets from the first network to the mobile device; rebuilding headers of unforwarded data packets at the first network; and sending the rebuilt data packets from the first network to the second network. Another example embodiment includes resetting a header compression state at the second network; receiving unforwarded data packets from the first network at the second network, the unforwarded data packets being data packets the first network had not yet forwarded to the mobile device; and sending the unforwarded data packets from the second network to the mobile device.

The invention discloses a downlink data transmitting method, system and base station in the communication field, comprising: network side encodes downlink user data, and then packets user data with the same downlink data transmission information and records position information of each user data in the data packet; the network side transmits the data packet to user side and transmits the position information to the corresponding user; each user on the user side extracts the respective user data for coding according to the position information. And it reduces repeated data transmission problem in the downlink user data transmission and raises utilization ratio of network transmission resources.

The invention discloses a method and a device used for synchronizing a mail list to a server, and relates to the technical field of data services. The technical problem that the pressure of short message issuing of the server is heavy in the prior art is solved. The method comprises the steps that the server receives the mail list which is uploaded by a user terminal for the first time, wherein the mail list records the information of each contact of the user terminal; for the information of each contact, the server judges whether the contact information exists in a seed database according to a seed ID in the contact information; if the fact that the seed ID does not exist in the seed database, the server adds the contact information in the seed database. The method and the device, which are provided by the invention, are mainly used for short message sending and receiving.

A data operating method, system, client, and data server are provided. The method includes: sending a write request of a file to a data server, where the write request includes identifiers of sub-data blocks constituting the file; receiving mappings between the identifiers of the sub-data blocks and storage servers returned by the data server according to the write request; and writing the sub-data blocks to the corresponding storage servers according to the mappings. With the present invention, whether the identifiers of the sub-data blocks are saved may serve as a basis for determining whether the sub-data blocks are written, thus ensuring that no duplicate data is stored in the system and increasing the storage space of the system.

The invention discloses a data caching method applied to storage equipment, a control device of the storage equipment and the storage equipment. The storage equipment comprises a first cache (212) anda second cache (213). When the storage device monitors that a first data block is cached in the first cache (212) (S301), a first data block to be deleted is cached in the first cache (212) (S601), or the caching time of the first data block in the first cache (212) exceeds a threshold value (S801), obtaining feature information of the first data block, and then determining whether the first datablock is stored in the second cache (213) according to the feature information of the first data block; and if it is determined that the first data block is stored in the second cache (213) (S303, S603, S803), adjusting the level of elimination priority of the first data block in the second cache (213) (S304, S604, S804). By coordinating the data in the first cache (212) and the second cache (213) in the storage device, the response speed of the IO request is improved.

The invention discloses an online unit load prediction method based on ensemble learning. The method comprises the following steps: firstly, reading historical unit load data of a past month from a database, secondly, preprocessing the data, detecting a gross value according to a Pauta criterion, and processing the detected gross value in a backward filling manner; further performing feature extraction on the preprocessed data, including time features, correlation features and aggregation features, and constructing a feature data set; training a limit gradient lifting model and a lightweight efficient gradient lifting model through the feature data set, and training a long-short-term memory neural network model through a prediction result of an LGBM model and original feature data; and finally, taking output results of the three models as three inputs to train a linear regression model to output a final load prediction result. In a model training stage, network parameters are adjusted by taking a minimum prediction average error as a principle, so that an optimal unit load prediction model is established.

Example embodiments provide methods for handling the transition of a mobile device from a first network to a second network, the second network operating according to a different communications protocol from the first network. One example embodiment includes terminating forwarding of the data packets from the first network to the mobile device; rebuilding headers of unforwarded data packets at the first network; and sending the rebuilt data packets from the first network to the second network. Another example embodiment includes resetting a header compression state at the second network; receiving unforwarded data packets from the first network at the second network, the unforwarded data packets being data packets the first network had not yet forwarded to the mobile device; and sending the unforwarded data packets from the second network to the mobile device.

The invention discloses a high-throughput sequencing database construction technology capable of efficiently capturing the three-dimensional conformation of chromosomes of a frozen biological sample.The invention provides a sample pre-processing method suitable for Hi-C high-throughput sequencing library building. The sample pre-processing method includes the steps of freezing a biological sampleto be tested first, then increasing the temperature in a gradient mode, and then cross-linking and fixing. The FS-Hi-C technology is not only suitable for fresh samples, but also for frozen samples.The results show that the three-dimensional structure of chromatin of a frozen replacement sample and a fresh sample are highly similar, and the quality of a Hi-C library is improved. Compared with atraditional Hi-C pretreatment method, FS-Hi-C is highly consistent in chromatin interaction, A / B compartment and TAD and the like, the quality of the library constructed by the FS-Hi-C method is better, the storage capacity is increased, effective interaction data is more, and the repeated data generated by PCR amplification is reduced. The technology reduces the requirements for sample timeliness. The high-throughput sequencing database construction technology provides an improved method for the application of Hi-C in biology.

The invention discloses a BPMN-based micro-service workflow import method, and relates to the technical field of micro-service workflows, and the method specifically comprises the steps: firstly, reading workflow data based on BPMN in batches, carrying out the classification according to different data protocol standards, and obtaining a name D, and all child nodes N and attributes of a flow through reading the keywords of all workflows; obtaining a structural relationship R, and then converting each workflow data into data based on a (D, N, R) format; verifying whether the workflow executes anewly added operation or an updating operation according to the name D, and finally executing an AQL statement to store workflow data into a graphic database Neo4j; and when the process editing frontend reads the data, executing the AQL statement according to the query condition, reading out the data in the (N, R, P) format from the graph database, recombining the data into workflow data based on the BPMN according to an analysis program, and displaying the workflow data at the front end. According to the method, the query efficiency of the traversal sub-process is improved; and the visual import and export operations also meet the requirements of business personnel.

The invention provides an underwriting method and device, an electronic device and a storage medium. The underwriting method comprises the following steps of obtaining an attribute of a to-be-underwritten insurance policy; matching an underwriting rule set name from the underwriting rule set names of a plurality of underwriting rule sets according to the attribute of the to-be-underwritten insurance policy, and dividing each underwriting rule set according to a determined finite-state machine algorithm; determining at least one underwriting route according to the matched underwriting rule setname; calling a program of an underwriting rule set on the underwriting route according to the determined underwriting route to perform underwriting on the to-be-underwritten policy to obtain an underwriting result of the underwriting route; and obtaining an underwriting result of the to-be-underwritten policy in combination with the underwriting result of the at least one underwriting route. According to the present invention, the efficient underwriting is achieved.

The invention belongs to the technical field of dynamic knowledge updating, and particularly relates to a dynamic knowledge updating method based on three decisions in a variable-precision rough set model, which comprises the following steps: 1, setting a threshold value, and selecting data precision in a rough set; 2, selecting a data relationship in the rough set according to a set precision threshold value, and carrying out data region judgment; 3, generating fuzzy set data in the rough set after judging that the data is finished, and setting an incommitment rule for the fuzzy set data; 4,selecting an approximate space, substituting the generated fuzzy set data into the approximate space, and judging the regional value of the data again; and 5, performing intra-region data comparison on the data generated after judgment, selectively discarding the generated repeated information to obtain updated data, and through three decisions, effectively reducing data updating vulnerabilities caused by two decisions, reducing repeated data of dynamic knowledge updating, and facilitating subsequent updating.

A machine and method of reducing duplicate transmission data employs one more more digests to track field / value pairs that have previously been distributed. Each digest contains a record table and a segment table. The record table includes anonymous identifier records, each of which contain an anonymous identifier and one or more indexes into the segment table. The segment table comprises an array of every existing data field / value pair. Before distribution of update data, each record is matched to an anonymous identifier record in the record table. The segment values in the prospective distribution record are compared to the digest's anonymous identifier record, used to determine which data has already been distributed, and thus will be suppressed in the distribution.

The invention discloses a communication method of a transformer substation and a server. The method is applied to the server. A first port of the server is connected to electronic equipment, and a second port of the server is connected to wireless equipment; the electronic equipment and the wireless equipment which are connected thereto are located on the same layer of the transformer substation together with the server; the method comprises the following steps: receiving a message, and acquiring a port that receives the message; if the port that receives the message is the first port, determining the message as an original message; intercepting the original message to acquire at least one first fragment, wherein the at least one first fragment is the same as at least one stored standard fragment; replacing the first fragment which is the same as the standard fragment by a standard identification of the standard fragment to acquire a recombinant message; adding a recombination identification and a recombination instruction in the recombinant message; and sending the recombinant message to the wireless equipment. The communication method solves the problem of a bottleneck of low data access speed in a network.

Data duplication may be reduced in cloud storage. First snapshots of one or more remote volumes may be received via a network. Responsive to and / or based on the first snapshots, unique clusters and duplicate clusters may be identified among the valid clusters of the remote volumes. The unique clusters and single instances of the duplicate clusters may be stored in a backup file, such that the backup file is devoid of duplicate clusters. Second snapshots of the one or more remote volumes may be received via the network. Responsive to the second snapshots, the clusters in the backup file that are no longer valid may be utilized to store the valid clusters in the one or more remote volumes not yet stored in the backup file.

The invention relates to the technical field of data products, in particular to a data product processing method for data quotient service, which discovers the value of a data product by acquiring information of the data product, performing analysis and function determination so as to subsequently innovate the capability of data service. Data products meeting business requirements are selected from the data products, similarity comparison is carried out on the selected data products, the data product with the highest similarity is selected from the selected data products, the data products are checked, efficient utilization of the data products is realized, data with specific values and meanings are extracted from the data products, excessive repeated data in an original data product is reduced, specific data can be applied to business requirements, the data is optimized and processed, the optimization result is scored, whether the optimized data can meet the business requirements or not is determined, the processed data is prevented from influencing use, and the effect of meeting the actual business requirements is achieved.

The invention relates to a method and a device for converting an SWF into a Canvas cartoon. The method comprises following steps of analyzing an SWF file, traversing the SWF file so as to obtain an element which comprises film editing and graphs, and defining and storing the element to an element pool; compressing the graphs and deriving compressed graph data; compressing the film editing and deriving frame operation instructions of the film editing; and calling and restoring the compressed graph data and the frame operation instructions of the film editing via an application function of a script runtime library to be a Canvas cartoon. In this way, size of the data is reduced; the compressed graph data and the frame operation instructions of the film editing are directly called via the application function of the script runtime library without conditional judgment, so computing time is reduced; and due to the small size of the data, loading time is saved and playing fluency is improved.

The invention discloses a method for uploading monitoring data, which includes: when the application program starts, report and save the device information of the device where the application program is located to the background server once; judge whether the variable data of the current device changes; if so, generate a The new association code is stored in association with the changed volatile data; during the period when the volatile data has not changed, all event monitoring data to be reported during the period is associated with the latest association code; when the data is reported, the The relevant information in the saved device information, the variable data and event monitoring data stored in association with the same association code are assembled and compressed according to the preset format, and the monitoring data package is obtained and uploaded to the background server. The invention also discloses a monitoring data uploading device, equipment, system and readable storage medium. While ensuring the integrity of monitoring data, the present invention reduces repeated uploading of redundant data, thereby saving traffic for users.

The invention discloses a photovoltaic intelligent operation and maintenance data transmission method, which comprises the steps of obtaining data information of all photovoltaic components, a photovoltaic data terminal, a photovoltaic power station and a photovoltaic operation and maintenance scheduling terminal in a preset range, and constructing a whole-network photovoltaic operation and maintenance model based on all the data information; training a whole-network photovoltaic operation and maintenance model by utilizing the data information of all the photovoltaic components; calculating the change trend of the data information of each photovoltaic component by using each photovoltaic data terminal; utilizing a data server of the photovoltaic power station to diagnose and analyze the data change trend of each photovoltaic part in the region, and randomly encrypting abnormal data information of each photovoltaic part by taking the data characteristics of the region as a secret key;and decrypting the abnormal data information of each photovoltaic part in the preset range by using the photovoltaic operation and maintenance scheduling end, and mapping the decrypted abnormal data information to a whole-network photovoltaic operation and maintenance model. According to the invention, the transmission speed and safety of photovoltaic data can be improved.

A machine and method of reducing duplicate transmission data employs one more more digests to track field / value pairs that have previously been distributed. Each digest contains a record table and a segment table. The record table includes anonymous identifier records, each of which contain an anonymous identifier and one or more indexes into the segment table. The segment table comprises an array of every existing data field / value pair. Before distribution of update data, each record is matched to an anonymous identifier record in the record table. The segment values in the prospective distribution record are compared to the digest's anonymous identifier record, used to determine which data has already been distributed, and thus will be suppressed in the distribution.

The invention discloses a passenger flow crowd identification method. The method comprises the following steps: acquiring a moving track, store entering data and passing time of a traveler, and based on the curve smoothness of the moving track and in combination with the store entering time of the store entering data, identifying a temporary crowd in crowds; that is to say, the temporary crowd in invalid data is screened out from the plurality of crowds through the data, so that invalid data caused by invalid passing and temporary tailgating of the temporary crowd is fully considered under the condition of subsequent shopping mall big data analysis, the data analysis dimension is increased, the accuracy of shopping mall big data analysis is improved, and a decision maker can make arrangement and planning in a more targeted manner.

The invention discloses a management device and a method for providing file download addresses, wherein the method includes that resource locating information is generated according to a storage path of a file in a data source node, contents of the file generate a content summary of the file through specific data transformation, a download address is generated at least according to the resource locating information of the file and the content summary of the file, and the download at least comprises the resource locating information of the file and the content summary of the file.