Multi-source heterogeneous water environment big data management system

Abstract

The invention provides a multi-source heterogeneous water environment big data management system, and relates to the technical field of computers. The system comprises a water environment big data cleaning unit for cleaning water environment data, a water environment big data exchange unit for exchanging the cleaned water environment data, and a metadata management unit which is used for tracking and analyzing the water environment big data. On the one hand, by establishing the water environment data cleaning model, the cleaning process and the cleaning rule are provided, and the problem of conflicts during water environment data application is solved; on the other hand, a distributed technology is adopted at the center end of the data exchange platform, an NoSQL database and a relational database are supported, the type of stored data is effectively expanded, and the data reading and writing efficiency is improved; besides, by defining the metadata of the water environment big data, unified coding, management and retrieval of the metadata in the multi-source heterogeneous water environment management system are realized, and a metadata support is provided for the management system.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a multi-source heterogeneous water environment big data management system. Background technique [0002] Under the vigorous promotion of my country's water special project, various water environment management technologies have been carried out in various river basins, including the cleaning and exchange of water environment big data and the management of big data. [0003] The current water environment monitoring data presents the following characteristics: first, the large amount of data, and the rapid increase of large texts, pictures, and video information of environmental protection data; second, the wide range of environmental monitoring data, including pollution source sewage monitoring data, pollution source census data, and environmental statistics Data, surface water quality monitoring data, ecological data, etc. Many of these data are updated every day, and some are ...

AI Technical Summary

Problems solved by technology

[0004] In terms of data use, although a large number of water quality monitoring stations, hydrological monitoring stations, and online monitoring systems for discharge outlets have been built in most river basins, the use of data is relatively simple, and there is a lack of data analysis and application functions, making it difficult to form a platform for big data analysis. Joint efforts, lack of effective multi-information fusion and systematic comparative analysis, make it difficult to effectively mine the value of data, and cannot provide support for scientific decision-making. With the growing trend of "composite" environmental data, there is an urgent need to plan and integrate existing data. Management, based on the business needs of actual management, strengthen the research of multi-dimensional analysis technology of data, and give full play to the decision-making support ability of data

Images