Redis-based mass data classified storage method and system

Abstract

The invention provides a Redis-based mass data classified storage method and system, and the method comprises the steps: S1, classifying data, and defining a data category ID for each category of data; S2, for each data category, calculating the number N of hash buckets according to the data scale of the corresponding actual service; S3, taking the data identifier ID, the data category ID and the number N of the data buckets as input parameter factors, and performing hash calculation to obtain a hash key and a field; S4, using the data content corresponding to the data identification ID as a hashvalue; and S5, storing the hashkey, the field and the hashvalue into the redis. According to the method and the device, the user data identifier is converted into a digital form and is stored in redis in the form of hash type data, so that the purpose of reducing memory occupation is achieved.

Description

technical field [0001] The present invention relates to the field of computer technology, in particular to a Redis-based massive data classification storage method and system. Background technique [0002] At present, when processing huge user data, there are generally the following common processing methods: [0003] Store user data directly in the database: This method has almost no limit to the growth of user data, and the cost of the hard disk is cheaper than memory, but the efficiency and response time of query data are far inferior to the memory database redis. For real-time advertising trading systems In terms of advertising, it will increase the response timeout rate of advertising bidding, and with the increase of user data, the query time will increase, which will further increase the response timeout rate; [0004] Store user data directly in redis in the form of key-value key-value pairs. This method will take up 8 times more memory space than the method used in...

AI Technical Summary

Problems solved by technology

[0003] Store user data directly in the database: This method has almost no limit to the growth of user data, and the cost of the hard disk is cheaper than memory, but the efficiency and response time of query data are far inferior to the memory database redis. For real-time advertising trading systems In terms of advertising, it will increase the response timeout rate of advertising bidding, and with the increase of user data, the query time will increase, which will further increase the response timeout rate;

[0004] Store user data directly in redis in the form of key-value key-value pairs. This method will take up 8 times more memory space than the method used in this patent, and the server cost needs to increase a lot, and it is difficult to meet the user data volume. Continuous growth, and at the same time, more memory fragments are generated when user data is added, deleted, or modified in business operations

Images