Power operation and maintenance data cleaning method based on isolation forest algorithm and neural network

Abstract

The invention provides a cleaning method of power communication operation and maintenance data, and more particularly to a power operation and maintenance data cleaning method based on an isolation forest algorithm and a neural network. The method includes: firstly, using the improved isolation forest algorithm to construct an isolation forest model iForest solving a target problem; then definingan evaluation system of the isolation forest algorithm on abnormal data; and carrying out prediction correction on abnormal data attributes, which are detected through an isolation forest, through training the BP neural network. According to the method, optimization is carried out on a power communication operation and maintenance data cleaning method based on the isolation forest algorithm and the neural network, abnormality detection precision is improved, data correction errors are reduced, and effective optimization is realized for a power operation and maintenance data cleaning program onaspects of abnormal-data positioning accuracy, a data correction accuracy rate, training time, resource occupation and the like.

Description

technical field [0001] The invention provides a method for cleaning power communication operation and maintenance data, and more specifically relates to a method for cleaning power operation and maintenance data based on an isolated forest algorithm and a neural network. Background technique [0002] With the vigorous development of power communication networks, the volume of power operation and maintenance data is increasing, and the power sector has higher and higher requirements for data reliability. In the process of transmission and storage of power operation and maintenance data, due to the influence of external interference and transmission errors, bad data problems such as noise, data loss, and data errors will inevitably occur; power data contains multidimensional attributes and is obtained by different devices. Anomaly detection of data presents challenges. Traditional data correction methods such as calculating the mean and regression analysis cannot accurately l...

AI Technical Summary

Problems solved by technology

However, this method does not correct missing values ​​and invalid values. At the same time, due to the limited hardware processing capacity, it is not suitable for processing large-scale power operation and maintenance data with complex attributes; patent 201510129479.3 performs data cleaning based on the ETL mechanism in the data warehouse, and the cleaning range is large. , the algorithm execution efficiency is high

However, because the power operation and maintenance data contains multi-dimensional attributes, the data volume and scale are huge, and the attributes are complex, the above schemes are still insufficient in terms of cleaning accuracy and data quality.

Images