Eureka AIR delivers breakthrough ideas for toughest innovation challenges, trusted by R&D personnel around the world.

Zero copy data flow based on RDMA

A data flow, zero-copy technology, applied in the field of data flow, can solve the problem that the copy overhead time is not significant

Active Publication Date: 2017-12-15
CLUSTAR TECH LO LTD
View PDF8 Cites 9 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In a low-speed network (the network protocol stack is generally TCP / IP / Ethernet), such replication overhead is not significant relative to the transmission time of network hardware; however, in modern 40 / 100Gpbs networks, due to the transmission time of the network hardware itself The performance is greatly improved, and the overhead caused by copying these large data blocks to the memory buffer becomes an important bottleneck in the entire data transmission pipeline

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Zero copy data flow based on RDMA
  • Zero copy data flow based on RDMA
  • Zero copy data flow based on RDMA

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0023] The preferred embodiments of the present invention will be described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described here are only used to illustrate and explain the present invention, and are not intended to limit the present invention.

[0024] like figure 1 As shown, an RDMA-based zero-copy data flow mainly includes a memory allocator and an information collector;

[0025] The memory allocator is used to implement different memory allocation rules;

[0026] The memory type information collector is used to analyze the data flow calculation graph, and determine the buffer management rules of each step according to the data source and data receiving node of each edge in the data flow calculation graph, and register them in the address conversion table The buffer will be shared between the computing and communication subsystems, and the buffer will be freed when it is no longer used by both subsystems...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses zero copy data flow based on RDMA. The zero copy data flow mainly comprises a memory allocator and an information collector; the memory allocator is used for achieving allocation rules of different memories; the memory classification information collector is used for analyzing a data flow calculation graph and calculating a data source and a data receiving node according to every side in the data flow calculation graph to determine a buffering area management rule of every step. The zero copy data flow based on the RDMA has the advantages that the high tensor transmission speed and high-speed extension of a GPU can be achieved and unnecessary memory copy is eliminated.

Description

technical field [0001] The present invention relates to the technical field of data streams, in particular to an RDMA-based zero-copy data stream. Background technique [0002] A high-speed network architecture built with commercial switches and network cards is a key component of a modern data center. Remote Direct Memory Access (RDMA) provides applications with zero-copy and CPU-bypass communication. Recently, the RDMA over ConvergedEthernet (RoCE) communication protocol has received extensive attention and is considered to be the core communication protocol layer of the 40 / 100Gbps Ethernet network for high-throughput, low-latency data centers. However, existing applications are often designed around Berkeley socket-based interfaces. In order to transplant the existing program to the RoCE communication protocol without modification, the existing application program usually uses the socket interface simulated in the user space. This design requires the application progra...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F13/28G06F12/0871
CPCG06F12/0871G06F13/28
Inventor 易百忍陈力陈凯
Owner CLUSTAR TECH LO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com