Method and device for extracting communication features and generating traffic

Abstract

The invention discloses a communication characteristic extraction and traffic generation method and device, and relates to the technical field of computer communication. The device comprises an application characteristic obtaining module, an application realizing module, and a traffic generation engine module, wherein the application characteristic obtaining module is used for establishing a communication module by analyzing communication characteristic data of a Map / Reduce type application, and inputting obtained data to the communication model to generate a result; the communication module is composed of a phase segmentation model and a phase traffic model; the application realizing module is used for generating a Map list according to the result and the communication model, obtaining a data rate calculation function according to the Map list and data rate request information, and obtaining a data rate via the data rate calculation function; and the traffic generation engine module is used for obtaining a generated traffic according to the data rate. According to the communication characteristic extraction and traffic generation method and device disclosed by the invention, in a communication information statistic aspect, through analysis on a target application, the dimensionality of characteristic parameters is reduced; and therefore, the cost needed for calculating and storing resources is relatively low, the realization is easy, and the method and the device are applied to an application scene.

Description

technical field [0001] The invention relates to the technical field of computer communication, in particular to a method and device for extracting communication features and generating traffic. Background technique [0002] At present, in the field of big data communication characteristic analysis, how to use limited computing and storage resources to generate network traffic that conforms to the communication characteristics of big data distributed systems has become a bottleneck technology for communication optimization in distributed systems. Existing solutions are all It is to statistically record the data transmission of each node in the network within a limited period of time under the network structure that has been determined to be deployed. The second is traffic fitting modeling, that is, according to the statistical records within a period of time, function fitting is performed, and by looking for a mathematical model that conforms to the communication characterist...

AI Technical Summary

Problems solved by technology

[0002] At present, in the field of big data communication characteristic analysis, how to use limited computing and storage resources to generate network traffic that conforms to the communication characteristics of big data distributed systems has become a bottleneck technology for communication optimization in distributed systems. Existing solutions are all It is to statistically record the data transmission of each node in the network within a limited period of time under the network structure that has been determined to be deployed. The second is traffic fitting modeling, that is, according to the statistical records within a period of time, function fitting is performed, and by looking for a mathematical model that conforms to the communication characteristics of the actual records, guide and make each Communication nodes that send information generate communication traffic, but the traffic generated by the above two methods only depicts the communication characteristics in a specific network, and it is difficult to evaluate the network performance of different scales or different topologies. The specific analysis is as follows

[0003] First of all, by recording network traffic and directly reproducing it, although it is possible to accurately generate communication traffic that conforms to the communication characteristics of the system, it is not operable when facing a large-scale network, such as a network with a scale of millions of nodes. It is necessary to record the time-varying communication flow value information of each node in the network for a long time, and it needs to consume large storage resources to store the communication flow information of each communication node sending information in the network at each moment. At the same time, the biggest problem of this method is , the communication characteristics it can describe are limited to the system scale when the traffic is obtained, and cannot be used to describe the communication characteristics of a larger-scale system. Secondly, although the method of function fitting based on statistical records saves storage resources, the actual The communication behavior of the big data platform application has certain fluctuations, ignoring the network performance changes caused by the actual deployment parameters of the application. It is also impossible to accurately generate traffic consistent with the actual big data distributed system

[0004] To sum up, the existing communication traffic generation methods for big data distributed systems cannot take into account the cost of computing storage resources and the accuracy of traffic. In addition, the existing methods are suitable for different network topologies, different network node scales, For different types of distributed platforms, the statistical recording process needs to be performed again, that is, the statistical recording results are not universal, which increases the scalability of the method and the complexity of actual deployment and implementation

Images