Massive unstructured distribution network data integration method based on knowledge graph technology

Abstract

The invention discloses a mass unstructured distribution network data integration method based on knowledge mapping technology. The method includes that a data collection unit collects unstructured distribution network data of each informatization system, and quality analysis and data cleaning processing are performed on the unstructured distribution network data of each informatization system; according to the processed unstructured distribution network data of each informatization system, data local index based on local knowledge mapping is constructed; the data local index based on the local knowledge mapping is sent to a data management center through a big data connector; the data management center constructs data global index based on global knowledge mapping. Collection, quality analysis and data cleaning of distributed multi-source heterogeneous data are advanced to each informatization system, so that data fusion calculation quantity, storage pressure and data scheduling burden of the data management center are lowered; the data global index based on the global knowledge mapping is utilized to integrate data sources, so that convenience is brought to data inquiry and extraction, and workload of the data management center is reduced.

Description

technical field [0001] The invention relates to the technical field of data fusion and integration, in particular to a massive unstructured distribution network data integration method based on knowledge map technology. Background technique [0002] The power grid includes information systems such as marketing systems, production systems, distribution data collection and monitoring systems, and energy meters. Massive data, combined with business system data such as marketing system and production system, and effectively identify and filter the data, finally output relevant data that is beneficial to power operation and improves customer service quality and service level. [0003] The distribution network data collected from various information systems can be divided into two types, one is structured data, such as data or symbol data, and the other is unstructured data, such as user voice, image, text, etc. The existing unstructured distribution network data integration meth...

AI Technical Summary

Problems solved by technology

[0004] However, on the one hand, this method generally performs centralized data cleaning in the data center, resulting in a large amount of data cleaning and slow integration speed, which cannot meet the integration requirements of massive unstructured data; on the other hand, the unstructured data of each information system There are differences in business logic, data format, and storage of digitalized data. Therefore, when the data is transmitted to the data center platform, it is not only not conducive to the classified storage of massive data, but also brings inconvenience to data extraction and query, which greatly increases the data center. workload of the platform

Images