Protocol-independent network redundant flow eliminating method

Abstract

The invention discloses a protocol-independent network redundant flow eliminating method. According to the method, a certain number of data packets are grabbed from a network in advance, the data packets are grouped according to the magnitudes of loads, statistics of the data size cumulative probability of grouping is conducted, a corresponding load threshold value is determined, and a sending end conducts redundant flow elimination on the data packet the load of which is larger than the load threshold value; the loads are partitioned according to the weak Hash values, the strong Hash value of each data block is taken as a fingerprint to be matched with a fingerprint in a fingerprint base, the fingerprint which is not matched with the fingerprint in the fingerprint base and the corresponding data block are updated to the fingerprint base and a data block base, the sending end takes the initial positions of all the data blocks in the data packets and the positions of all the data blocks in the data block base as loads to regenerate data packets and sends the data packets to a receiving end, and the receiving end conducts recovery on the data packets after receiving the data block information in the data packets. The method is used for processing redundant data among the data packets without being affected by the application layer communication protocol and has a good redundant flow elimination effect and a good processing time effect.

Description

technical field [0001] The invention belongs to the technical field of network flow management, and more specifically relates to a protocol-independent method for eliminating redundant network flow. Background technique [0002] With the rapid development of Internet technology applications, network traffic such as web browsing, file sharing, video distribution, and entertainment games has shown a rapid growth trend. A large number of identical or similar content on the Internet has been repeatedly requested by users at the edge of the network, resulting in data in the network. The repeated transmission on the network leads to the generation of a large amount of redundant traffic, which not only consumes network bandwidth, but also reduces the efficiency of the Internet. Network redundant traffic elimination has become a new research hotspot in network traffic management technology. [0003] Most traditional redundancy elimination schemes operate at the application layer an...

AI Technical Summary

Problems solved by technology

Object caches, including web proxy caches and peer-to-peer media caches, can be used to respond to frequent and repeated requests instead of origin servers where resources reside. Level caching doesn't work well

Images