A graph data storage and query method for large-scale social networks

Abstract

The invention discloses a large-scale social network-oriented graph data storage and query method. The data storage manager of the invention stores the received graph data in a Key-Value manner, uses the vertex ID of the graph data as the Key, and uses the vertex neighbor The field is Value; data storage for each vertex neighborhood: store multiple edges connected to the vertex neighborhood in a fixed-size memory block in order with time stamps, and form a doubly linked list, and attribute information of the vertex and index information are stored in a data structure. When the data storage manager receives an access request to visit vertex v, the data storage manager transmits the vertex v and its k-order neighbors to the requester; the requester caches the returned data locally, and the next query, first check The local cache, if there is no queried vertex, sends the access request to the data storage manager. The invention can satisfy dynamic update, and is suitable for processing sparse data scenarios and random access.

Description

technical field [0001] The invention relates to a large-scale social network-oriented graph data storage and query method, which belongs to the field of software technology. Background technique [0002] At present, the mainstream method of graph data storage is to preprocess the graph data into records of edges and vertices, and store them in large files in the distributed file system in the form of sequential data sets. When accessing graph data, the large files storing the graph data are accessed in a sequential scan. This organization method cannot provide effective data storage and access performance for multi-round iterative graph computing applications. In order to improve the access performance of graph data, the memory management technology of graph data has become an important trend, such as Trinity, Giraph, etc. [0003] Neo4j is a graph database using the Key-Value storage model. The most basic storage units are vertices and edges. When the breadth traverses BFS...

AI Technical Summary

Problems solved by technology

The data division of Redis is completed by the client, and the server program cannot proxy the request of different map data

[0009] From the above research work, it can be seen that the data access requirements of graph computing and the characteristics of graph data pose challenges to the storage of graph data. Traditional data storage methods such as file systems or distributed file systems and graph databases with Key-Value storage models are difficult. Efficient support for graph data access and update, and thus cannot effectively support graph calculations

Images