72 results about "Consistent hashing" patented technology

The present disclosure discloses a method, a system and a server of removing a distributed caching object. In one embodiment, the method receives a removal request, where the removal request includes an identifier of an object. The method may further apply consistent Hashing to the identifier of the object to obtain a Hash result value of the identifier, locates a corresponding cache server based on the Hash result value and renders the corresponding cache server to be a present cache server. In some embodiments, the method determines whether the present cache server is in an active status and has an active period greater than an expiration period associated with the object. Additionally, in response to determining that the present cache server is in an active status and has an active period greater than the expiration period associated with the object, the method removes the object from the present cache server. By comparing an active period of a located cache server with an expiration period associated with an object, the exemplary embodiments precisely locate a cache server that includes the object to be removed and perform a removal operation, thus saving the other cache servers from wasting resources to perform removal operations and hence improving the overall performance of the distributed cache system.

A distributed storage system has MDSs (metadata servers). Directories of file system namespace are distributed to the MDSs based on hash value of inode number of each directory. Each directory is managed by a master MDS. When a directory grows with a file creation rate greater than a preset split threshold, the master MDS constructs a consistent hashing overlay with one or more slave MDSs and splits directory entries of the directory to the consistent hashing overlay based on hash values of file names under the directory. The number of MDSs in the consistent hashing overlay is calculated based on the file creation rate. When the directory continues growing with a file creation rate that is greater than the preset split threshold, the master MDS adds a slave MDS into the consistent hashing overlay and splits directory entries to the consistent hashing overlay based on hash values of file names.

The invention discloses a file processing method based on a distributed file system, the system, and a client. The method comprises the following steps: the client sends a file writing request to a meta node; the client receives a file identifier, a data block identifier of each data block, the copy quantity of data blocks and a selectivity factor from the meta node; the client performs hashing operation on the file identifier of the file, the data block identifiers of the data blocks and the selectivity factor to obtain a hashing value; the client finds a consistent hashing ring according to the hashing value, obtains the value of a minimal virtual node greater than the hashing value and clockwise selects virtual nodes consistent with the copies in quantity from the virtual node on the consistent hashing ring; and the client sends the data blocks to different storage devices for storage when the selected virtual nodes meet the pre-condition. According to the invention, the occupation of the storage space of the meta node is reduced, the location speed during data writing can be improved, and the load balance of the system is implemented.

A global namespace (GNS) is constructed across heterogeneous network-attached storage (NAS) devices by organizing metadata servers into a cluster layer having cluster nodes and a consistent hashing overlay layer having overlay nodes. The global namespace is distributed based on workload at the cluster and overlay nodes, and a portion of the GNS namespace may be migrated from one layer to the other. Cluster nodes may be added to the cluster layer, overlay nodes may be added to the consistent hashing overlay layer. Further, a cluster node may become an overlay node or an overlay node may become a cluster node. To access the global namespace, a request for namespace metadata managed by a cluster node is serviced by the cluster node, but if the requested namespace metadata have been migrated to the consistent hashing overlay layer, the cluster node will forward the request to the responsible overlay node for servicing.

The invention provides a mass information storage system. The system comprises a storage system, a metadata server, a coordinator and a data distribution and balance module, wherein the storage system has a data storage function; the metadata server is used for receiving and saving metadata from the storage system; the coordinator is used for receiving data which needs to be written by a client into the storage system, organizing the received data to a key value form and sending to the data distribution and balance module, and the coordinator is further used for receiving an inquiry command of the client, instructing a node of the server to implement an inquiry operation, receiving an inquiry result and returning to the client; and the data distribution and balance module is used for distributing the data to all the nodes of the server of the storage system by applying a consistent hashing algorithm.

The invention discloses a multiple-tier distributed cluster system and solves the possible content transfer of common load balance technology that is based on a polling strategy or content hash. The system solves the problems of cluster expansibility, content transfer and the like by adopting tiered load balance technology and combining the polling strategy and consistent hashing algorithm. The technical scheme of the system is as follows: the system comprises a transport tier load balancer at the front end of the system, a URL (Uniform Resource Locator) consistent hashing balancer at the middle tier of the system, and a real server at the rear end of the system, wherein the transport tier load balancer receives request data transmitted by a client side and conducts load balance; requests of the client side are distributed to the URL consistent hashing balancer at the middle tier; the URL consistent hashing balancer receives the request data of the client side distributed by the transport tier load balancer and distributes the same URL to the same real balancer for service through the consistent hashing algorithm; and the real server stores the data.

A distributed data storage method and system based on multilayer consistent hashing are provided. The system includes a plurality of storage nodes for providing data storage and redundancy protection;a plurality of management nodes for maintaining storage node attributes and virtual groups-storage node mapping information; a plurality of monitoring nodes for maintaining storage node state and processing state changes such as storage node addition, deletion, and failure; and one or more clients that provide storage system access points for applications or users. Storage nodes are organized ina tree structure, with each storage node in each layer of the tree being assigned with multiple identities and a hash space maintaining consisteny hash space. Instead of sharing a hash space between all storage nodes, multiple hash spaces in the storage node architecture tree that are consistent across each layer. The system guarantees the reliability and availability of the stored procedure and avoids the problem of load imbalance when accessing the system.

The invention provides a block chain transparent slicing method, a device and a system. The method comprises the following steps: obtaining a plurality of slicing node information serving the block chain; obtaining the corresponding relationship between the slicing node and the slicing according to the slicing node information through a consistent hash algorithm; obtaining the corresponding relationship between the slicing node and the slicing node according to the slicing node information; constructing a ReDiR tree corresponding to the fragments on a DHT network according to the correspondingrelationship between the fragment nodes and the fragments and the fragment nodes; obtaining contract address information of the transaction data, and obtaining the correspondence relationship betweenthe transaction data and the segments through a consistent hashing algorithm according to the contract address information; obtaining a slice node in a corresponding slice according to the corresponding relationship between the transaction data and the slice; sending the transaction data to the slicing node.

The invention provides a back-to-source method and a back-to-source device for an access request. The back-to-source method and the back-to-source device relate to the field of Internet, and solve theproblems of large back-to-source quantity and low resource utilization ratio of a father node scheme in the prior art. The back-to-source method comprises the steps of: receiving an access request ofa user by an edge node, performing consistent hashing operation on an URL (Uniform Resource Locator) carried in the access request and then determining that the URL accesses a corresponding father node server group; and requesting a source file corresponding to the URL from the father node server group. The back-to-source method provided by the invention is suitable for a CDN (Content Delivery Network), and achieves file back-to-source with flexibility, balance, and a high storage disaster tolerance capacity.

A method for scheduling distributed buffer resources is provided, where in the distributed buffer system, each buffer node is distributed on a virtual circle that is based on a consistent hashing algorithm, and the method includes: monitoring a load value of each buffer node in the distributed buffer system; judging whether a load exception exists in the current distributed buffer system according to the load value; and if the load exception exists in the current distributed buffer system, adjusting a layout of the buffer nodes in the distributed buffer system.

The invention discloses a method for improving QUIC protocol request scheduling efficiency. According to the method, a mapping method based on a connection ID and a consistent Hash ring is adopted torealize the dynamical selection of the connection IDs and efficient scheduling of QUIC protocol requests; virtual server nodes are mapped onto the consistent Hash ring, so that the effective control range of the servers is dynamically determined; the selection interval of the connection ID is dynamically determined based on a mapping relation between the effective control range and the connectionID; and the QUIC requests are mapped onto the ring, so that the requests are efficiently scheduled. With the method of the invention adopted, the problem of the failure of QUIC connection smooth migration in a cluster environment which is caused by a condition that a traditional software load balancing request scheduling method fails to adapt to the characteristics of the QUIC protocol requests can be solved, and negative impact on the performance and scalability of a server cluster can be avoided.

The invention relates to the technical field of data filter, in particular to a Hash ring-based distributed data filter method. The method comprises the following steps: S101, receiving information of a distributed de-weighted cluster by a client; S102, client interface data condition request; S103, node reception request; S104, data filter block positioning: positioning data filter blocks of data after hash complementation according to a partition request sent by an RPC; and S105, data returning: carrying out data existence judgement on corresponding data blocks according to filter keys, returning corresponding states and returning the data. After adopting the structure, the method is capable of realizing a multi-tenant function so that the clients can add filters of established types according to the business demands; the lasting backup restoration of the filters can be realized so that the data loss can be avoided; the whole cluster is constructed on the basis of a consistent hash ring, so that the cluster is filtered to realize linear extension; and for the same filter, a plurality of sub-filters can be constructed, so that the false judgement rate can be reduced.

The invention discloses a partitioning method for processing virtual asset data based on perception of node computing power. The partitioning method includes the steps of processing between machines and processing inside machines. In the step of processing between machines, the weight is added on the basis of the distributed consistent hashing and is determined according to the relational expression Qi=Q*ai/A. In the step of processing inside machines, related attributes of data are set and data with the related attributes are stored in the same machine node, and virtual asset data platforms with same identifier attributes are stored adjacently. Compared with the prior art, the partitioning method with consideration in machine performance and processing related to identifiers and platforms has the advantages of shortening inquiry time greatly and improving inquiry efficiency when applied to storage of mass information of virtual asset.

The invention discloses a method for automatic spreading of a hotspot file. The method is characterized in that hotspot discovery is determined according to two dimensions: the number of requests, and the bandwidth; and as long as a judgment result is positive according to one of the dimensions, discovery of the hotspot file is determined, and load balancing equipment carries out a hotspot spreading action, that is, the load balancing equipment does not execute the consistent hashing strategy when accessing backend cache equipment but randomly selects the cache equipment to be accessed and enables the file to uniformly access each piece of the cache equipment. The method has the advantages of reasonable structure and high convenience in use; and uniform loads on the equipment in a cluster are achieved, so that the loads of the equipment corresponding to the files are enlarged, and the equipment can conveniently provide services at any time.

The invention discloses a server load balancing method based on improved consistent hashing, and the method can achieve the dynamic adjustment of the node load in a server cluster according to the performance of a server. According to the method, the virtual nodes are constructed according to the performance of the nodes in the server cluster and the connection number in one period, so the problemof non-uniform load of a load balancing system based on consistent hash at present is solved. The method comprises the following specific steps: enabling a routing server to periodically maintain thenumber of nodes and performance data of the server cluster, and calculating the number of virtual nodes to be constructed by each node according to the performance ratio of the nodes and the number of the nodes; constructing a hash ring by the cluster nodes and the virtual nodes according to a hash algorithm; enabling the routing server to periodically maintain the connection number of each cluster node; calculating a virtual node number redundancy value according to the difference of connection numbers under the same performance value after each period; and periodically repeating the five steps, updating the number of the virtual nodes, and reconstructing the hash ring.

ECMP routing is carried out in fabric of network entities by representing valid destinations and invalid destinations in a group of the entities by a member vector. The order of the elements in the member vector is permuted and fanned out. A portion of the elements in the fanned out vector is pseudo-randomly masked. A flow of packets is transmitted to the first valid destination in the masked member vector.