Time series data processing method, device, system, server and readable storage medium

Abstract

The invention discloses a time series data processing method, device, system, server and readable storage medium. The method includes: receiving multiple pieces of data to be processed sent by multiple devices, the multiple pieces of data to be processed respectively include time series data collected by the multiple devices and identification codes of the multiple devices; For the grouping results of a device, import the multiple pieces of data to be processed into corresponding task queues for sequential processing, correspondingly obtain multiple pieces of result data; and store the result data in sequence in the first cache queue. According to the time-series data processing method provided by the present invention, based on the grouping results of devices, high-performance time-series processing of large batches of data can be realized in parallel, and the advantages of multi-core computing can be brought into play. At the same time, a single machine can handle the data processing of tens of thousands of devices, avoiding the construction of complex computer clusters, and significantly reducing deployment costs.

Description

technical field [0001] The present invention relates to the field of data processing, in particular to a time series data processing method, device, system, server and readable storage medium. Background technique [0002] With the development of Internet of Things technology, the processing volume and processing difficulty of Internet of Things data are increasing. At present, the data processing methods commonly used in the field of Internet of Things are mainly divided into the following two categories: [0003] The first category: Big data cluster computing methods based on typical flow / batch processing frameworks such as Hadoop, Spark, and Storm, which are mainly suitable for high-speed processing of massive data, have the advantages of high parallelism and high throughput, and can support thousands of machines Data processing for IoT devices. But at the same time, the big data cluster processing framework has high requirements on the resources required for its own op...

AI Technical Summary

Problems solved by technology

However, when the data cannot be guaranteed to be processed in chronological order, for example, the data at the 4th moment is processed before the data at the 3rd moment, an early warning prompt is only triggered when the data at the 3rd moment (20°C) is processed, resulting in transportation Delay in detecting failure of cabin temperature control system

[0006] Serialization can ensure the orderly processing of data, but it is bound to fail to take advantage of multi-core computing, resulting in a large waste of resources

In addition, the existing dedicated time series databases generally only focus on storing time series data and providing data query services, and cannot directly complete the aforementioned logical processing

Images