Data outlier identification method and system for automatic environment monitoring network

Abstract

The invention discloses a method and a system of identifying data abnormal values of an environment automatic monitoring network. The method comprises the following steps of: carrying out unified definition on a data identification system of the environment automatic monitoring network according to four data sources including automatic identification of a base station system, manual examination and verification of the base station, specialist identification of a real-time data platform and manual examination and verification of the platform so as to obtain data structures of a source type identifier, a mode identifier, a mode tracking data identifier, a deduction identifier and a deleting identifier; and identifying abnormal values of a base station layer and a platform layer according to the data structures of the source type identifier, the mode identifier, the mode tracking data identifier, the deduction identifier and the deleting identifier, and adopting an improved data identifier synchronization algorithm to implement synchronization of the abnormal value of the base station layer and the abnormal value of the platform layer. The mode tracking data is added and the deduction identifier is identified, and the abnormal value identification algorithm of the real-time data platform is provided, so that the method and the system have the advantages of high traceability efficiency, high precision, completeness, reliability, science and accuracy, and can be widely applied to the field of environment monitoring.

Description

technical field [0001] The invention relates to the field of environmental monitoring, in particular to a data abnormal value identification method and system for an automatic environmental monitoring network. Background technique [0002] The environmental automatic monitoring network is an important data source for the scientific characterization of environmental quality. Its purpose is to obtain high-quality environmental monitoring data and infer the existing quality characteristics of the entire environment. In addition to the elements of general environmental monitoring, the automatic environmental monitoring network also has the characteristics of long-term real-time uninterrupted monitoring of base stations, synchronous docking of data between base stations and real-time data platforms, and automatic analysis and statistics of massive data by real-time data platforms. According to the unique technical characteristics of the automatic environmental monitoring network,...

AI Technical Summary

Problems solved by technology

This method can effectively prevent outliers from being included in data statistics, but because there is no vertical tracking of the causes of outliers (such as instrument quality control and external environmental factors, etc.) when tracing the source of quality control work, platform data managers cannot track them. Judging the true state of data quality

In order to solve this problem, data management personnel need to query the work log of the base station and ask the base station maintenance personnel. The traceability of quality control work is inefficient and it is easy to make mistakes due to inquiries

[0006] (2) Missing or inconsistent sources of identification

[0008] However, there is still no relevant technology to unify the four types of data identification. As a result, data reviewers at different levels often only consider data identification from one source, which is not comprehensive and reliable enough.

[0009] (3) Lack of statistical algorithms in the real-time data platform for automatic environmental monitoring

However, most of the environmental automatic monitoring data do not conform to these distributions, and it is difficult to directly use these research results to test and identify

Therefore, the real-time data platform for automatic environmental monitoring lacks statistical algorithms that can check and audit the reliability and accuracy of statistical data

[0011] In addition, the current outlier identification technology directly deletes the outliers of the monitoring data, and it is difficult to accommodate the method of statistical theory to identify outliers

Because statistical theory can calculate the abnormal value in the monitoring data, but it does not mean that the abnormal value is absolutely wrong in the real environment. If the data calculated as abnormal is directly deleted, it will not respect the objective reality, and it is not scientific and accurate enough.

Images