Remote direct memory access

<h2 id="overview">Overview</h2>
RDMA supports <a href="/facts/Zero-copy/cc13RBxU">zero-copy</a> networking by enabling the <a href="/facts/Network_adapter/wyf5FR29">network adapter</a> to transfer data from the wire directly to application memory or from application memory directly to the wire, eliminating the need to copy data between application memory and the data buffers in the operating system. Such transfers require no work to be done by <a href="/facts/Central_processing_unit/Jxp1bqMV">CPUs</a>, <a href="/facts/CPU_cache/x8BYFcdv">caches</a>, or <a href="/facts/Context_switch/N0oJQ6CC">context switches</a>, and transfers continue in parallel with other system operations. This reduces latency in message transfer.
However, this strategy presents several problems related to the fact that the target node is not notified of the completion of the request (single-sided communications).

<h2 id="acceptance">Acceptance</h2>
As of 2018 RDMA had achieved broader acceptance as a result of implementation enhancements that enable good performance over ordinary networking infrastructure.<a class="footnote-ref" id="fnref:1" href="#fn:1">1</a> For example <a href="/facts/RDMA_over_Converged_Ethernet/SxDcW1aH">RDMA over Converged Ethernet</a> (RoCE) now is able to run over either lossy or lossless infrastructure. In addition <a href="/facts/IWARP/b7rVFH83">iWARP</a> enables an <a href="/facts/Ethernet/nMfhU8tU">Ethernet</a> RDMA implementation at the physical layer using <a href="/facts/Transmission_Control_Protocol/WW4sPrt9">TCP</a>/<a href="/facts/Internet_Protocol/Qq6VeEal">IP</a> as the transport, combining the performance and latency advantages of RDMA with a low-cost, standards-based solution.<a class="footnote-ref" id="fnref:2" href="#fn:2">2</a> The RDMA Consortium and the DAT Collaborative<a class="footnote-ref" id="fnref:3" href="#fn:3">3</a> have played key roles in the development of RDMA protocols and <a href="/facts/Application_programming_interface/GMlN4vUr">APIs</a> for consideration by standards groups such as the <a href="/facts/Internet_Engineering_Task_Force/tIcCsFhX">Internet Engineering Task Force</a> and the Interconnect Software Consortium.<a class="footnote-ref" id="fnref:4" href="#fn:4">4</a>
Hardware vendors have started working on higher-capacity RDMA-based network adapters, with rates of 100 Gbit/s reported.<a class="footnote-ref" id="fnref:5" href="#fn:5">5</a><a class="footnote-ref" id="fnref:6" href="#fn:6">6</a> Software vendors, such as <a href="/facts/IBM/QszA7nwd">IBM</a>,<a class="footnote-ref" id="fnref:7" href="#fn:7">7</a> <a href="/facts/Red_Hat/5rhmBLBa">Red Hat</a> and <a href="/facts/Oracle_Corporation/0z14FygH">Oracle Corporation</a>, support these APIs in their latest products,<a class="footnote-ref" id="fnref:8" href="#fn:8">8</a> and as of 2013<a href="https://en.wikipedia.org/w/index.php?title=Remote_direct_memory_access&action=edit">[update]</a> engineers have started developing network adapters that implement RDMA over Ethernet.<a class="footnote-ref" id="fnref:9" href="#fn:9">9</a> 
Both <a href="/facts/Red_Hat_Enterprise_Linux/6Cekx2ed">Red Hat Enterprise Linux</a> and <a href="/facts/Red_Hat_Enterprise_MRG/5rhmBLBa">Red Hat Enterprise MRG</a><a class="footnote-ref" id="fnref:10" href="#fn:10">10</a> have support for RDMA. Microsoft supports RDMA in <a href="/facts/Windows_Server_2012/W1vuUlSB">Windows Server 2012</a> via <a href="/facts/Server_Message_Block/pWM35pl8">SMB Direct</a>. <a href="/facts/VMware_ESXi/VQrtsQQg">VMware ESXi</a> also supports RDMA as of 2015.
Common RDMA implementations include the <a href="/facts/Virtual_Interface_Architecture/RkTWQNMs">Virtual Interface Architecture</a>, <a href="/facts/RDMA_over_Converged_Ethernet/SxDcW1aH">RDMA over Converged Ethernet</a> (RoCE), <a href="/facts/InfiniBand/6CUz47wQ">InfiniBand</a>, <a href="/facts/Omni-Path/uqL5Ct3r">Omni-Path</a> and <a href="/facts/IWARP/b7rVFH83">iWARP</a>.

<h2 id="using-rdma">Using RDMA</h2>
Applications access control structures using well-defined APIs originally designed for the InfiniBand Protocol (although the APIs can be used for any of the underlying RDMA implementations). Using send and completion queues, applications perform RDMA operations by submitting work queue entries (WQEs) into the submission queue (SQ) and getting notified of responses from the completion queue (CQ). <a class="footnote-ref" id="fnref:11" href="#fn:11">11</a>

<h2 id="transport-types">Transport types</h2>
RDMA can transport data reliably or unreliably over the Reliably Connected (RC) and Unreliable Datagram (UD) transport protocols, respectively. The former has the benefit of preserving requests (no requests are lost), while the latter requires fewer queue pairs when handling multiple connections. This is due to the fact that UD is connection-less, allowing a single host to communicate with any other using a single queue.<a class="footnote-ref" id="fnref:12" href="#fn:12">12</a>

<h2 id="external-links">External links</h2>
<ul><li><a href="http://www.rdmaconsortium.org/">RDMA Consortium</a></li>
<li><a href="/facts/RFC_(identifier)/esftTvE7">RFC</a> <a href="https://www.rfc-editor.org/rfc/rfc5040">5040</a>: A Remote Direct Memory Access Protocol Specification</li>
<li><a href="http://www.hpcwire.com/2006/09/15/a_tutorial_of_the_rdma_model-1/">A Tutorial of the RDMA Model</a></li>
<li><a href="https://www.hpcwire.com/2006/10/06/why_compromise-1/">"Why Compromise?"</a> // HPCwire, Gilad Shainer (Mellanox Technologies), 2006</li>
<li><a href="http://www.hpcwire.com/hpcwire/2006-08-18/a_critique_of_rdma-1.html">A Critique of RDMA</a> for high-performance computing</li>
<li><a href="https://www.cs.utah.edu/~stutsman/cs6450/public/papers/rdma.pdf">RDMA Reads: To Use or Not to Use?</a></li>
<li><a href="https://www.openfabrics.org/wp-content/uploads/2022-workshop/2022-workshop-presentations/201_RPolig.pdf">[1]</a></li></ul>

<h2 id="references">References</h2>

<ol>
<li id="fn:1">RoCE Rocks over Lossy Network: https://dl.acm.org/citation.cfm?id=3098588&dl=ACM&coll=DL <a href="https://dl.acm.org/citation.cfm?id=3098588&dl=ACM&coll=DL" target="_blank">https://dl.acm.org/citation.cfm?id=3098588&dl=ACM&coll=DL</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></li>
<li id="fn:2">"Understanding iWARP" (PDF). Intel Corporation. Retrieved 16 May 2018. <a href="https://www.intel.com/content/dam/support/us/en/documents/network/sb/understanding_iwarp_final.pdf" target="_blank">https://www.intel.com/content/dam/support/us/en/documents/network/sb/understanding_iwarp_final.pdf</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></li>
<li id="fn:3">"DAT Collaborative website". Archived from the original on 17 January 2015. Retrieved 14 October 2014. <a href="https://web.archive.org/web/20150117180600/http://www.datcollaborative.org/" target="_blank">https://web.archive.org/web/20150117180600/http://www.datcollaborative.org/</a> <a href="#fnref:3" class="footnote-back-ref">↩</a></li>
<li id="fn:4">The Interconnect Software Consortium website Archived 2005-08-30 at the Wayback Machine <a href="http://www.opengroup.org/icsc/" target="_blank">http://www.opengroup.org/icsc/</a> <a href="#fnref:4" class="footnote-back-ref">↩</a></li>
<li id="fn:5">"Microsoft Based Solutions - Mellanox Technologies". Retrieved 14 October 2014. <a href="http://www.mellanox.com/page/file_storage/" target="_blank">http://www.mellanox.com/page/file_storage/</a> <a href="#fnref:5" class="footnote-back-ref">↩</a></li>
<li id="fn:6">"40Gbe SMB Direct RDMA Over Ethernet For Windows Server 2012 - Chelsio Communications". 2 April 2013. Retrieved 14 October 2014. <a href="http://www.chelsio.com/chelsio-to-demonstrate-40g-smb-direct-rdma-over-ethernet-for-windows-server-2012/" target="_blank">http://www.chelsio.com/chelsio-to-demonstrate-40g-smb-direct-rdma-over-ethernet-for-windows-server-2012/</a> <a href="#fnref:6" class="footnote-back-ref">↩</a></li>
<li id="fn:7">"SOFA-STORAGE: CREATING A VENDOR AGNOSTIC FRAMEWORK TO ENABLE SEAMLESS STORAGE OFFLOAD USING SMARTNICS" (PDF). <a href="https://www.openfabrics.org/wp-content/uploads/2022-workshop/2022-workshop-presentations/201_RPolig.pdf" target="_blank">https://www.openfabrics.org/wp-content/uploads/2022-workshop/2022-workshop-presentations/201_RPolig.pdf</a> <a href="#fnref:7" class="footnote-back-ref">↩</a></li>
<li id="fn:8">"What RDMA hardware is supported in Red Hat Enterprise Linux?". 2 June 2016. <a href="https://access.redhat.com/solutions/22188" target="_blank">https://access.redhat.com/solutions/22188</a> <a href="#fnref:8" class="footnote-back-ref">↩</a></li>
<li id="fn:9">"40Gbe SMB Direct RDMA Over Ethernet For Windows Server 2012 - Chelsio Communications". Chelsio Communications. 2013-04-02. Retrieved 2016-07-15. The demonstration will show Microsoft's Windows Server 2012 SMB Direct running at line-rate 40Gb using RDMA over Ethernet (iWARP). <a href="http://www.chelsio.com/chelsio-to-demonstrate-40g-smb-direct-rdma-over-ethernet-for-windows-server-2012/" target="_blank">http://www.chelsio.com/chelsio-to-demonstrate-40g-smb-direct-rdma-over-ethernet-for-windows-server-2012/</a> <a href="#fnref:9" class="footnote-back-ref">↩</a></li>
<li id="fn:10">"Red Hat Enterprise MRG 2.0 Now Available". Archived from the original on 25 August 2016. Retrieved 23 June 2011. <a href="https://web.archive.org/web/20160825215016/https://investors.redhat.com/news-and-events/press-releases/2011/06-23-2011" target="_blank">https://web.archive.org/web/20160825215016/https://investors.redhat.com/news-and-events/press-releases/2011/06-23-2011</a> <a href="#fnref:10" class="footnote-back-ref">↩</a></li>
<li id="fn:11">Storm: a fast transactional dataplane for remote data structures: https://dl.acm.org/doi/abs/10.1145/3319647.3325827 <a href="https://dl.acm.org/doi/abs/10.1145/3319647.3325827" target="_blank">https://dl.acm.org/doi/abs/10.1145/3319647.3325827</a> <a href="#fnref:11" class="footnote-back-ref">↩</a></li>
<li id="fn:12">Storm: a fast transactional dataplane for remote data structures: https://dl.acm.org/doi/pdf/10.1145/3319647.3325827 <a href="https://dl.acm.org/doi/pdf/10.1145/3319647.3325827" target="_blank">https://dl.acm.org/doi/pdf/10.1145/3319647.3325827</a> <a href="#fnref:12" class="footnote-back-ref">↩</a></li>
</ol>

Remote direct memory access open-in-new

Remote direct memory access