Graph-based large-scale embedding model training method and system for click rate prediction

Abstract

The invention discloses a graph-based large-scale embedding model training method and system for click rate prediction, the system comprises a dense parameter module and a client module, a hybrid communication architecture is adopted, a click rate prediction input data set is distributed to different working nodes, each working node maintains a client, and the client module is used for providing an input data set; all local model parameters are directly stored in a GPU memory; and each working node holds a model parameter copy, and synchronization is carried out during training. According to the method, the importance of category characteristic values corresponding to click rate prediction input data is represented by adopting an Embedding model parameter, the click rate prediction data and an Embedding model vector are represented as a binary graph model, and model parallel training is executed by utilizing graph locality and degree deflection characteristics; and graph-based partition and bounded synchronization are designed, so that the expandability and parallel computing efficiency of training a large embedding model are improved.

Description

technical field [0001] The invention belongs to the technical field of distributed machine learning, and relates to a large-scale embedding model training method and system, in particular to a graph-based large-scale embedding model training method and system for click-through rate prediction. Background technique [0002] Embeddings are often used to tackle representation learning problems on high-dimensional data, such as words in text corpora, users, and items in recommender systems. Deep Embedding technology uses continuous vectors to represent discrete variables and has a large number of practical applications, such as click-through rate (CTR) prediction systems, graph processing, and information extraction. However, with the continuous expansion of the scale of the deep embedding model and the increase of the amount of input data, it is more challenging to build a large embedding model training system in terms of effectiveness and efficiency. For example, Facebook's p...

AI Technical Summary

Problems solved by technology

[0009] In summary, the existing click-through rate prediction technology cannot be applied to deep learning models, and there is expensive network communication overhead in large-scale distributed training scenarios; existing graph processing algorithms are not suitable for the embedding model used for click-through rate prediction. Training; the existing traditional consensus protocols and learning and training systems do not take into account the update dependencies between embeddings; high overhead and low efficiency

Images