Industrial real-time data compression method with high compression ratio

Abstract

The invention discloses an industrial real-time data compression method with a high compression ratio. The method is characterized by: reading in n data points and using a minimum square fitting method to fit a linear equation; if the n data points do not satisfy that linear equation deviation is less than a compression threshold, storing a starting point, then reading in the n data points after the starting point to fit the linear equation and restarting; if the n data points satisfies, reading in and determining whether the subsequent data point data value deviation is less than or equal to the compression threshold Delta; if the deviation is less than the Delta, abandoning, continuously reading in and determining the subsequent data points till the deviation is larger than the compression threshold Delta; if the deviation is larger than the compression threshold Delta, recording a fitted value of the previous data point, taking the previous data point as a start of reading in the data, re-reading in the n data points so as to carry out fitting determination and storing till all the industrial real-time data is compressed. In the invention, a direction of the linear equation accords with a variation trend of the subsequent data points. In a certain compression threshold scope, a data compression ratio is good and the ratio is larger than the ratio obtained by using the traditional compression methods.

Description

technical field [0001] The invention belongs to the technical field of data compression, and more specifically relates to an industrial real-time data compression method with a high compression ratio. Background technique [0002] 1. Overview of Data Compression [0003] Data compression technology has been widely used in image, audio processing and other fields. The technology is becoming more and more advanced and mature and has formed an international standard, such as JPEG compression technology in the field of image processing, MP3 compression technology in audio processing, etc. However, due to the continuous increase in the capacity of modern storage devices, it is less used in the field of industrial automation. Massive real-time and historical data will be generated in power systems, fault detection and diagnosis systems, process control, process monitoring, and multi-channel data acquisition systems. In the automatic system, the data compression has not been widel...

AI Technical Summary

Problems solved by technology

However, due to the continuous increase in the capacity of modern storage devices, it is less used in the field of industrial automation. Massive real-time and historical data will be generated in power systems, fault detection and diagnosis systems, process control, process monitoring, and multi-channel data acquisition systems. In automated systems, data compression has not been widely valued and applied

In a multi-channel automatic test system, the data is generally collected by the data acquisition module. The collected signal is generally a sensor signal. The current data acquisition module has a high sampling frequency. For example, the total sampling rate is 100KHz. If the system has 16 channels, A single channel can collect 62 times per second, and the current higher acquisition modules ranging from dozens of MHz to tens of GHz are much higher than this number, so that a large amount of high-precision floating-point data can be generated per second. For massive storage data, people's solution is simply to increase storage devices, but rarely apply data compression technology to compress a large amount of redundant data in order to reduce the amount of data and save storage devices

[0004] 2. Existing industrial real-time data compression methods

[0016] Among the above methods, the steady-state threshold method is more suitable for relatively steady-state changing data, and the effect on data with large real-time changes is not very good; the revolving door compression algorithm mainly uses the compression data formed by the current data point and the previous storage point. Limit the coverage area to determine whether the data should be retained. In this algorithm, multiple data points need to be repeatedly judged, so that the compression time is too long; the linear extrapolation algorithm has a better compression effect on data with a smaller compression limit, and the compression limit Larger is less effective

Images