The invention provides a real-time
dynamic data management system based on HBase. The real-time
dynamic data management system comprises data reading optimization, a
composite index mechanism and parallel
query optimization. According to the invention, a persistent data storage
mechanism based on HDFS and HBase is designed, so that the problem of
mass storage of multi-structure data is solved; a
composite index storage model is designed based on an HBase table structure, and realizing mapping management of index cache addresses and index table design of cache data storage;
queue division is performed on the data based on a multi-source buffer area structure to realize parallel storage, and the concurrent writing pressure of
mass data is relieved by combining strategies such as a consistency hash
algorithm, a multi-thread technology, a line key optimization design and the like; And a multi-query parallel mechanism is designed based on an HBase query real execution module and a multi-line technology, so that the query speed is increased. According to the real-time
dynamic data management system based on the HBase, the data writing and reading performance is effectively improved, theproblem of
mass storage of multi-structured data is solved, and the high-dynamic and real-time environmental requirements are met.