Data processing system, method and device

Abstract

The invention provides a data processing system, method and device, and the system comprises a sending server which is configured to monitor a network state; under the condition that the network state is abnormal, a first connection queue is set to be in a sending stopping state; a queue reset request is sent to the receiving server through an out-of-band channel; the receiving server side is configured to receive the queue reset request sent by the sending server side; in response to the queue reset request, a queue serial number of a second connection queue is reset; under the condition that the queue serial number of the second connection queue is reset, reset completion information is sent to the sending server through the out-of-band channel; the sending server is further configured to receive reset completion information; the queue serial number of the first connection queue is reset according to the reset completion information; and under the condition that the queue serial number of the first connection queue is reset, the first connection queue is set to be in a sendable state.

Description

technical field [0001] This description relates to the field of computer technology, in particular to data processing systems, methods and devices. Background technique [0002] Data center networks are increasingly using RDMA (Remote Direct Memory Access) to achieve high-performance network transmission services for distributed computing and distributed storage. Data center distributed computing and storage are usually in clusters Using the Internet-wide communication model, an RDMA network connection will be established between each service node. [0003] In a large-scale cluster, the full Internet model will generate a large number of RDMA network connections on each server node. When a network failure or network jitter occurs, problems such as data retransmission timeout or heartbeat timeout will occur, resulting in a large number of RDMA network connections in a short period of time. When the connection is disconnected, a large number of connection disconnection events...

AI Technical Summary

Problems solved by technology

[0003] In a large-scale cluster, the full Internet model will generate a large number of RDMA network connections on each server node. When a network failure or network jitter occurs, problems such as data retransmission timeout or heartbeat timeout will occur, resulting in a large number of RDMA network connections in a short period of time. When the connection is disconnected, a large number of connection disconnection events are generated on each node core, and then explosive memory resource recovery is initiated. After the network failure is restored, the application will re-initiate a connection establishment request for the disconnected connection in a short period of time. In the kernel, a large number of connection establishment events and explosive memory resource allocation are generated here. Because the process is very sudden, it will cause a surge in CPU utilization, machine load jitter, and a large number of memory and cache refreshes in a short period of time. The entire restoration of connection establishment The process takes a long time to complete, which greatly affects the availability and stability of the entire cluster service, so an effective solution is urgently needed to solve the above problems

Images