Multi-rule combined compression method based on database row and column mixed storage

Abstract

The invention discloses a multi-rule combined compression method based on database row and column mixed storage. A mixed storage compression mode for organizing data in a database according to tuple rows and compressing the data in the database according to property columns is provided by combining the current software and hardware development tendency and a severe performance bottleneck in database industry, and has the characteristic of high compression rate of column storage and the advantage of convenience in random positioning and accessing of row storage. Furthermore, a rule encoding method in a plurality of property columns is provided according to different data distribution characteristics; particularly, an inter-column compression rule is provided according to a possible relation among the property columns in a database sheet; by a rear-end general compression algorithm, a multi-level combined compression function is efficiently supplied to upper database application; and the maximum encoding and decoding speed under an appointed compression rate condition is guaranteed.

Description

technical field [0001] The invention relates to data storage technology, data compression technology, and data retrieval technology, in particular to a multi-rule composite compression method based on mixed storage of rows and columns in a database. Background technique [0002] Query processing and data storage are the two core elements of the database, and the two complement each other to ensure that the database can provide users with efficient data management and retrieval services. However, with the deepening of the information revolution, real-world applications generate massive amounts of new data all the time, and users are more inclined to retain historical data for a longer period of time. The limitation of data storage capacity has become a serious problem that must be faced without delay. On the other hand, the development speed of storage hardware has lagged far behind that of other computer system hardware, and the storage system has become a serious bottleneck...

AI Technical Summary

Problems solved by technology

The data in row storage is stored and read continuously in the form of tuple rows, but since the attributes of each tuple row in the data table have basically no data association, good compression effects cannot be achieved

In contrast, column storage stores the data of each attribute column in the data table separately and continuously, which can greatly improve the similarity of continuous data to achieve a higher compression rate, but at the same time breaks the organization of data tuple rows and will Causes the database to be extremely inefficient when doing traditional row queries

Images