Spatial big data distributed storage system based on hybrid database

Abstract

The invention discloses a spatial big data distributed storage system based on a hybrid database. The system comprises a spatial data storage subsystem and a spatial data management subsystem, whereinthe spatial data storage subsystem is mainly responsible for providing streaming data processing capability, traditional spatial database query and analysis capability and distributed system big dataprocessing capability to the outside; and the spatial data management subsystem is mainly responsible for data scheduling, monitoring, data distribution and node management. A spatial big data distributed storage strategy is adopted, real-time / near-real-time streaming big data is stored and managed, data in a distributed database can be quickly extracted to support traditional application, the application requirement for high concurrency and even high timeliness of the big data is met, and the storage requirement of the big data and efficient retrieval of the spatial data are effectively met.

Description

technical field [0001] The invention belongs to the field of big data distributed storage management, and relates to a space big data distributed storage system based on a hybrid database. Background technique [0002] The development of new technologies brings new challenges to spatial data storage and management. The booming development of Internet of Things, mobile Internet and cloud computing technologies and applications has transformed spatial data in terms of data volume and application mode; in addition, the development of sensor technology has significantly improved the spatial and temporal resolution of collected data. As a result, the scale of acquired data increases exponentially and rapidly. Facing the data in terabytes or even petabytes, it also brings enormous pressure to spatial data storage and processing. [0003] At present, various geospatial information acquisition means are diverse, the data scale is huge, the update frequency is fast, and the data app...

AI Technical Summary

Problems solved by technology

From the perspective of data storage, NoSQL databases are very suitable for the storage of spatial big data, but there are certain differences between the application modes in the Internet field and the GIS field. It is generally not recommended to modify data in NoSQL databases, and careless modification operations may even cause problems. Mass migration of related storage data leads to a sharp decline in performance; NoSQL databases provide very limited query capabilities, and MongoDB cannot meet the needs of common GIS queries; in a big data application environment, a single spatial index algorithm can no longer meet the efficient retrieval of spatial data

Images