A Load-Driven Distributed Graph Data Segmentation and Replication Method

Abstract

The invention proposes a workload-driven distributed graph data segmentation and replication method. In the present invention, the data in the graph data set are respectively stored in the server cluster through the hash table method; combined with the user's workload information, the active vertex set is constructed in the vertex data set combined with the load determination of the source server, and then each active vertex is combined with the server cluster According to the score function score, construct the target server score list, determine the target migration server through the target server load judgment; combine the active vertex set and the corresponding target migration server, judge whether the active vertex is a high-read vertex through the threshold, if the active vertex is not high The read vertex further dynamically adjusts the copy data of the active vertex through the maximum replication factor. The invention utilizes the characteristics of dynamic change of the workload to dynamically adjust the position of the graph vertex data, thereby improving the query efficiency and ensuring low delay and high throughput of the query.

Description

technical field [0001] The invention belongs to the field of computers, and in particular relates to a load-driven distributed graph data division and replication method. Background technique [0002] In recent years, with the continuous expansion of social network and Wanwei graph data, and the continuous expansion of multi-user access requirements, it is difficult for a common single database server to meet multi-user high-quality access services under existing resources. The traditional solution is to use vertical expansion and complete replication of data, and the resulting high cost is unsatisfactory, so a distributed data storage method emerges as the times require. Distributed data storage is based on cheap server clusters for horizontal partition expansion and partial replication backup, and provides concurrent graph data processing. In the face of distributed parallel data storage, how to reasonably store graph data in different nodes, so that users can improve the...

AI Technical Summary

Problems solved by technology

[0005] Another problem is the disaster recovery of distributed graph data. In order to prevent data loss due to system operation errors or system failures, we need to perform data backup, that is, data replication

The traditional solution is to perform a full backup of all data according to a fixed replication factor, resulting in expensive network and storage overhead

Images