Data export method and related product

Abstract

The invention relates to the technical field of data transmission optimization, and particularly discloses a data export method and a related product, the method comprising: a first device receives adata export request sent by a user terminal, the data export request comprising a data identifier of to-be-exported data and a memory of the to-be-exported data; if the memory of the to-be-exported data is greater than a first threshold, the first device splits the data export request into N sub-data export requests and adds a device identifier of a second device in a second device cluster to eachsub-data export request, and the device identifiers between any two sub-data export requests are different; and the first device broadcasts the N sub-data export requests to the second device cluster. By adopting the embodiment of the invention, the data export efficiency can be improved.

Description

technical field [0001] This application relates to the technical field of data transmission, in particular to a data export method and related products. Background technique [0002] With the maturity of informatization, the data collection work is becoming more and more perfect, and there are more and more types of data, so that a large amount of data corresponding to a user can be stored in the database, and the merchant can take the user's data from the database. Through big data Technology recommends products for users. Due to the large amount of data for each user, the export process is extremely difficult. Currently, there are two export methods: 1. Split the big data and export in batches. This export method will consume too much business time, and The user is required to perform the same export operation repeatedly, which reduces the user experience and affects the development of subsequent business applications. 2. "Export once, wait continuously", that is, initia...

AI Technical Summary

Problems solved by technology

Due to the large amount of data for each user, the export process is extremely difficult. Currently, there are two export methods: 1. Split the big data and export in batches. This export method will consume too much business time, and Users are required to perform the same export operation repeatedly, which reduces user experience and affects the development of subsequent business applications

2. "Export once, wait continuously", that is, initiate a data export request to the database server, and the requesting user exports from the database to the specified storage path. After receiving the request, the database server transmits the data according to the data it can bear If the data is exported to the specified storage path, due to the large amount of data, it will occupy too much memory resources, the waiting time is too long, and the performance of distributed services will not be fully utilized.

[0003] Therefore, the current big data export technology is single, the export time is too long, and the export process is complicated. When exporting a large amount of data on a single server, it may cause server memory overflow, the risk of server downtime, and poor user experience.

Images