Live broadcast platform training data processing method and device, storage medium and equipment

Abstract

The invention relates to the technical field of live broadcast data processing, in particular to a live broadcast platform training data processing method and device, a storage medium and equipment. The live broadcast platform training data processing method comprises the steps of obtaining user behavior data generated by a live broadcast platform in live broadcast, and extracting feature data ofa user from the user behavior data; determining a feature form of the feature data according to the data type and the data distribution of the feature data, calling a corresponding compression algorithm to compress the feature data according to the feature form, and outputting compressed data and a corresponding compression mark; and decompressing the compressed data by using the compression markto obtain decompressed data, and inputting the decompressed data into a machine learning model for training. According to the scheme provided by the invention, the feature data is compressed and thentransmitted and processed, and different compression algorithms are selected according to different feature forms, so that the processing efficiency of the live broadcast platform training data is improved.

Description

technical field [0001] The present application relates to the technical field of live data processing, and in particular, the present application relates to a method, device, storage medium and equipment for processing training data of a live broadcast platform. Background technique [0002] The rapid development of the Internet has brought a lot of convenience to people's life, but also brought massive data information. In the live broadcast scenario, the recommendation system can provide users with interested live broadcast rooms or live broadcast content based on the information recommendation model. However, the data generated in the live broadcast scene is huge and complex, and the huge and complex data puts a huge pressure on the hardware that performs data transmission during the storage and transmission process. Therefore, it is necessary to reduce the training data of the information recommendation model through some compression techniques to reduce Training data o...

AI Technical Summary

Problems solved by technology

However, the data generated in the live broadcast scene is huge and complex, and the huge and complex data puts a huge pressure on the hardware that performs data transmission during the storage and transmission process. Therefore, it is necessary to reduce the training data of the information recommendation model through some compression techniques to reduce Training data occupies storage space and processing resources, and reduces the computational complexity of the model

[0003] At present, common model compression techniques include: converting various data types into string types, and then forming data packets, but this kind of live platform training data processing scheme adopts the same compression strategy for all data types, resulting in Data processing is inefficient

Images