Data transmission method and device

A data transmission method and a technology of a data transmission device, which are applied in the field of data transmission, can solve problems such as frequent fluctuations in data volume, unpredictable data, and difficulty in ensuring safe and efficient data transmission, so as to improve the data transmission rate and ensure the safety and stability of the transmission process Effect

Pending Publication Date: 2019-08-20
厦门稿定股份有限公司
View PDF3 Cites 7 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0002] In the process of business operation, a large amount of log data is usually generated. In order to facilitate subsequent viewing and statistics of the log data, the log data needs to be collected and transmitted in a unified manner, and stored in a unified log data storage service. However, , in the process of data collection and transmission, due to frequent fluctuations and unpredictable data volume, once the data volume is large, it will be difficult to ensure safe and efficient data transmission

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data transmission method and device
  • Data transmission method and device
  • Data transmission method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0029] Embodiments of the present invention are described in detail below, examples of which are shown in the drawings, wherein the same or similar reference numerals designate the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary and are intended to explain the present invention and should not be construed as limiting the present invention.

[0030] In the process of data collection and transmission, due to the different sources of data (log data may exist in log files, databases, queue services, etc.), the data protocols are different, which makes it difficult to manage the data transmission process in a unified manner. Currently, although Logstash / Flume can However, the required configuration is complex and requires high usage costs. When running multiple Logstash / Flume services, manual intervention is required for management, which will also bring high operation and...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a data transmission method and device, and the method comprises the steps: carrying out the customized collection of different data sources to obtain different collected data, and enabling the different collected data to be analyzed into data packets in a unified data format; writing the data packets in the unified data format into a data pipeline for caching; reading a datapacket cached in the data pipeline, and writing the read data packet into a downstream data storage service to realize data transmission; and in the data transmission process, limiting the data reading flow rate according to the preset flow peak value of the downstream data storage service. Therefore, the data is locally buffered through the data pipeline, and the data transmission flow is dynamically adjusted through the flow control assembly, so that the whole transmission process is safe and stable, and the data transmission rate is effectively improved.

Description

technical field [0001] The present invention relates to the technical field of data transmission, in particular to a data transmission method, a data transmission device, a computer-readable storage medium and computer equipment. Background technique [0002] In the process of business operation, a large amount of log data is usually generated. In order to facilitate subsequent viewing and statistics of the log data, the log data needs to be collected and transmitted in a unified manner, and stored in a unified log data storage service. However, , in the process of data collection and transmission, due to frequent fluctuations and unpredictable data volume, once the data volume is large, it will be difficult to ensure safe and efficient data transmission. Contents of the invention [0003] The present invention aims to solve one of the technical problems in the above-mentioned technologies at least to a certain extent. Therefore, an object of the present invention is ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/18G06F16/172
CPCG06F16/181G06F16/172
Inventor 廖峻阳陈锦煊林杰兴
Owner 厦门稿定股份有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap