Data processing method and apparatus

Abstract

The invention provides a data processing method and device. The method comprises the steps that for all dimensionality processing in multi-dimensional processing, whether the absolute value of the difference between the dimensionality identification number and a preset thread number is smaller than or equal to a preset threshold is judged; when the absolute value of the difference between the dimensionality identification number and the preset thread number is smaller than or equal to the preset threshold, all pieces of processing corresponding to all dimensionality identifications are distributed to a preset number of multiple threads to be executed, and therefore the pieces of processing corresponding to the same dimensionality identification are completed on multiple threads in parallel; furthermore, when the absolute value of the difference between the dimensionality identification number and the preset thread number is larger than the preset threshold, all pieces of processing corresponding to all dimensionality identifications are distributed to corresponding single threads in a preset number of threads to be executed, and therefore the pieces of processing corresponding to the same dimensionality identification are completed on the same thread. In this way, all pieces of concurrent processing of event flow data can be completed at the same time, so that the real-time processing efficiency is improved, and use of a user is facilitated.

Description

technical field [0001] The present application relates to the Internet field, and more specifically relates to a data processing method and device for multi-dimensional processing of event stream data. Background technique [0002] With the rapid development of the Internet, online data interaction is becoming more and more common, and it has become the main way of data interaction in people's lives. [0003] In the process of online data interaction, due to the need for efficient and real-time processing of massive online interactive data, event streaming technology is generally used to process massive interactive data. The so-called event flow means that the same kind of events occurring at different times in the same data source constitute a strictly ordered flow in time. That is, the same type of data is formed into multiple events, and when multiple events are triggered in a certain order, an event flow is formed. Statistical processing on multiple dimensions can be p...

AI Technical Summary

Problems solved by technology

[0009] The purpose of this application is to provide an improved data processing technology to solve the problem of low real-time processing efficiency caused by the different number of threads actually used by concurrent processing.

Images