Data processing method and device, electronic equipment and storage medium

Abstract

The invention discloses a data processing method and device and electronic equipment, and the method comprises the steps: obtaining a to-be-processed aggregation message from a cache queue, the aggregation message comprising a target identifier and a task category of a target object, and the task category representing the category of a task processing item executed on the target object; according to the target identifier and the task category, executing the task processing item on the target object to obtain a processing result; and storing the processing result according to a preset storage rule. According to the method, the data processing amount in the system can be reduced, so that the database pressure is relieved, and the data reading and writing speed is increased.

Description

technical field [0001] The present disclosure relates to the technical field of Internet data processing, and more specifically, to a data processing method, device, electronic equipment, and computer-readable storage medium. The present disclosure also relates to a data acquisition method. Background technique [0002] In the Internet field, users can publish a variety of objects through terminal devices, such as video, text, and images; after the terminal device sends the object to be published by the user to the server, the server can aggregate the objects. Aggregate objects into the same category; in this way, when other users use terminal devices to browse objects, the server can display objects under the category that the user is interested in to the user for viewing, so as to improve user stickiness. [0003] In the process of implementing the present application, the inventor found that when performing aggregation processing in the existing method, the object is ofte...

AI Technical Summary

Problems solved by technology

[0003] In the process of implementing the present application, the inventor found that when performing aggregation processing in the existing method, the object is often aggregated after the object is successfully released, and the obtained aggregated data is synchronously written into the database, for example, mysql In the database; however, during the peak hours of user activity, for example, at night or on holidays, due to the sharp increase in the number of objects released, the corresponding aggregation processing will also increase accordingly, and the existing objects are synchronously sent to the database after the object is successfully released. The way of writing aggregated data will cause the data request volume of the database to increase suddenly during peak hours, which will put a lot of pressure on the database, and since the database is generally deployed according to the maximum data request volume, there may also be non-peak hours The problem of wasting database resources

[0004] The above is an example of the aggregation processing of the object after the object is successfully released to illustrate the problems existing in the existing method; in practice, after the object is successfully released, when the relevant information of the object needs to be updated, the existing method also The above problems exist, therefore, it is necessary to provide a data processing method to aggregate data according to business characteristics, avoiding possible problems of excessive pressure on the database and waste of resources

Images