High-performance file transmission system and method thereof

Abstract

The invention discloses a high-performance file transmission system and a method thereof; the method is realized by using the system provided by the invention; the method comprises a process of receiving client events by a high speed transmission TCP server; the process further comprises: initializing a monitoring socket and setting a monitoring mode; initializing a receiving event for the monitoring socket, activating the receiving event and adding the receiving event to a libevent event trigger network library; when the client initiates a connection request, automatically calling functions of receiving the client event by the high speed transmission TCP server; receiving the client to generate a client socket and packaging an operation type of controlling connection handle and requests to an event notification, and adding the event notification to a task queue. The high-performance file transmission system and the method provided by the invention use a full asynchronous mode on the architecture design of the FTP server and use memory cache simultaneously to reduce times of locating and reading / writing of a magnetic needle of a magnetic disc and increase the ratio of the utilization rates of magnetic discs I / O and the network I / O, thereby transferring the bottleneck to the network I / O. Furthermore, the system realizes high speed transmission and assures the reliability of thetransmission by using good elasticity of the UDT protocol server.

Description

technical field [0001] The invention relates to the technical field of network data transmission, in particular to a high-performance file transmission system and method. Background technique [0002] At present, the BGI Institute has world-class gene sequencing capabilities, and the data generated every day reaches dozens of terabytes (1024GB); it puts forward higher requirements for data transmission. In addition, data synchronization between sub-centers distributed across China and overseas and the headquarters also imposes very high requirements on bandwidth utilization. [0003] Although after more than 30 years of development, submarine optical cable technology has become increasingly mature, but the signal delay introduced by long-distance network transmission is still unavoidable. Especially in long-distance transmission (such as the transmission of optical signals from Beijing to New York will introduce a delay of at least 60ms), the traditional TCP protocol seriou...

AI Technical Summary

Problems solved by technology

[0003] Although after more than 30 years of development, submarine optical cable technology has become increasingly mature, but the signal delay introduced by long-distance network transmission is still unavoidable

Especially in long-distance transmission (such as the transmission of optical signals from Beijing to New York will introduce a delay of at least 60ms), the traditional TCP protocol seriously deteriorates the performance of transmission technology due to signal delay.

And with the increase of the bandwidth delay product (BDP, Band Delay Product), the TCP protocol begins to become inefficient (this is because the AIMD algorithm completely reduces the congestion window of the TCP protocol, but cannot quickly restore the available bandwidth; in theory, Traffic analysis shows that TCP is more vulnerable to packet loss attacks when BDP is increased to a high level), and cannot complete efficient transmission tasks

[0004] Therefore, the traditional TCP protocol-based file transfer system and method will not be able to meet the growing demand for large data volume, long-distance, and real-time transmission in terms of speed and reliability, and will become a bottleneck restricting the rapid development of network data transmission technology.

Images