SCSI RDMA Protocol

<h2 id="history">History</h2>
SRP was published as an ANSI standard (ANSI INCITS 365-2002) in 2002 and renewed in 2007 and 2019.<a class="footnote-ref" id="fnref:3" href="#fn:3">3</a><a class="footnote-ref" id="fnref:4" href="#fn:4">4</a>

<h2 id="related-protocols">Related Protocols</h2>
As with the <a href="/facts/ISCSI_Extensions_for_RDMA/fCDQnKBm">ISCSI Extensions for RDMA</a> (iSER) communication protocol, there is the notion of a target (a system that stores the data) and an initiator (a client accessing the target) with the target initiating data transfers. In other words, when an initiator writes data to a target, the target executes an RDMA read to fetch the data from the initiator and when a user issues a SCSI read command, the target sends an RDMA write to the initiator.
While the SRP protocol is easier to implement than the iSER protocol, iSER offers more management functionality, e.g. the target discovery infrastructure enabled by the iSCSI protocol.

<h2 id="performance">Performance</h2>
Bandwidth and latency of storage targets supporting the SRP or the iSER protocol should be similar. On Linux, there are two SRP and two iSER storage target implementations available that run inside the kernel (<a href="/facts/SCST/B40fo8Gf">SCST</a><a class="footnote-ref" id="fnref:5" href="#fn:5">5</a> and <a href="/facts/LIO_Target/XsEBu3lU">LIO</a>) and an iSER storage target implementation that runs in user space (STGT). Measurements have shown that the <a href="/facts/SCST/B40fo8Gf">SCST</a> SRP target has a lower latency and a higher bandwidth than the STGT iSER target. This is probably because the RDMA communication overhead is lower for a component implemented in the Linux kernel than for a user space Linux process, and not because of protocol differences.<a class="footnote-ref" id="fnref:6" href="#fn:6">6</a>

<h2 id="implementations">Implementations</h2>
In order to use the SRP protocol, an SRP initiator implementation, an SRP target implementation and networking hardware supported by the initiator and target are needed. The following software SRP initiator implementations exist:

<ul><li>Linux SRP initiator, available since November 2005 (kernel version 2.6.15).</li>
<li>Windows SRP initiator, available through the winOFED InfiniBand stack.<a class="footnote-ref" id="fnref:7" href="#fn:7">7</a></li>
<li>VMWare SRP initiator, available since January 2008 through <a href="/facts/Mellanox/FcpB6kPy">Mellanox</a>' OFED Drivers for VMware Infrastructure 3 and vSphere 4.<a class="footnote-ref" id="fnref:8" href="#fn:8">8</a></li>
<li>Solaris 10 SRP initiator, available through Sun's download page.<a class="footnote-ref" id="fnref:9" href="#fn:9">9</a></li>
<li>Solaris 11 and OpenSolaris SRP initiator, integrated as a component of project COMSTAR.<a class="footnote-ref" id="fnref:10" href="#fn:10">10</a></li>
<li>The IBM POWER virtual SCSI client driver for Linux (ibmvscsi), available since January 2008 (kernel version 2.6.24<a class="footnote-ref" id="fnref:11" href="#fn:11">11</a>). Virtual SCSI allows client logical partitions to access I/O devices (disk, CD, and tape) that are owned by another logical partition.<a class="footnote-ref" id="fnref:12" href="#fn:12">12</a><a class="footnote-ref" id="fnref:13" href="#fn:13">13</a></li></ul>
The following SRP target implementations exist:

<ul><li>The <a href="/facts/SCST/B40fo8Gf">SCST</a> SRP target implementation.<a class="footnote-ref" id="fnref:14" href="#fn:14">14</a> This is a mature SRP target implementation available since 2008 via both <a href="/facts/SCST/B40fo8Gf">SCST</a> and OFED.<a class="footnote-ref" id="fnref:15" href="#fn:15">15</a></li>
<li>Linux <a href="/facts/LIO_Target/XsEBu3lU">LIO</a> SRP target,<a class="footnote-ref" id="fnref:16" href="#fn:16">16</a> available since January 2012 (kernel version 3.3<a class="footnote-ref" id="fnref:17" href="#fn:17">17</a>), based on the <a href="/facts/SCST/B40fo8Gf">SCST</a> SRP target.</li>
<li>The IBM POWER virtual SCSI target driver (ibmvstgt), available since January 2008 (kernel version 2.6.24<a class="footnote-ref" id="fnref:18" href="#fn:18">18</a>).<a class="footnote-ref" id="fnref:19" href="#fn:19">19</a><a class="footnote-ref" id="fnref:20" href="#fn:20">20</a></li>
<li>DataDirect Network's (DDN) disk subsystems such as the S2A9900 and SFA10000, which use the SRP target implementation in the disk subsystem's controllers to present LUNs to servers (the servers act as SRP initiators).<a class="footnote-ref" id="fnref:21" href="#fn:21">21</a><a class="footnote-ref" id="fnref:22" href="#fn:22">22</a></li>
<li>IBM's FlashSystem.<a class="footnote-ref" id="fnref:23" href="#fn:23">23</a></li>
<li>The <a href="http://docs.oracle.com/cd/E23824_01/html/821-1459/gldpr.html">Solaris COMSTAR target</a>, available since early 2009 in OpenSolaris and Solaris 11.<a class="footnote-ref" id="fnref:24" href="#fn:24">24</a></li></ul>
<h2 id="see-also">See also</h2>
<ul><li><a href="/facts/ISCSI_Extensions_for_RDMA/fCDQnKBm">iSCSI Extensions for RDMA (iSER)</a></li>
<li><a href="https://www.youtube.com/watch?v=4UVRuq-lgKo">"SRP Update and Directions"</a>. openfabrics.org. 27 Mar 2012.</li></ul>

<h2 id="references">References</h2>

<ol>
<li id="fn:1">ANSI T10 SRPr16a, www.t10.org. <a href="http://www.t10.org/drafts.htm#SCSI3_SRP" target="_blank">http://www.t10.org/drafts.htm#SCSI3_SRP</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></li>
<li id="fn:2">ANSI T10 SRPr16a, web.archive.org <a href="https://web.archive.org/web/20080422013122/http://www.t10.org/ftp/t10/drafts/srp/srp-r16a.pdf" target="_blank">https://web.archive.org/web/20080422013122/http://www.t10.org/ftp/t10/drafts/srp/srp-r16a.pdf</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></li>
<li id="fn:3">ANSI webstore for purchasing standards - ANSI INCITS 365-2002 <a href="http://webstore.ansi.org/RecordDetail.aspx?sku=ANSI+INCITS+365-2002+%28R2007%29" target="_blank">http://webstore.ansi.org/RecordDetail.aspx?sku=ANSI+INCITS+365-2002+%28R2007%29</a> <a href="#fnref:3" class="footnote-back-ref">↩</a></li>
<li id="fn:4">"SCSI RDMA Protocol - 2 (SRP-2)" (PDF). ANSI T10. 7 May 2019. <a href="http://www.t10.org/cgi-bin/ac.pl?t=f&f=srp2r06.pdf" target="_blank">http://www.t10.org/cgi-bin/ac.pl?t=f&f=srp2r06.pdf</a> <a href="#fnref:4" class="footnote-back-ref">↩</a></li>
<li id="fn:5">The SCST Project, an open source SCSI target implementation for Linux that includes an SRP target implementation. <a href="http://sourceforge.net/projects/scst/" target="_blank">http://sourceforge.net/projects/scst/</a> <a href="#fnref:5" class="footnote-back-ref">↩</a></li>
<li id="fn:6">Performance of SCST versus STGT. <a href="https://lkml.org/lkml/2008/1/29/387" target="_blank">https://lkml.org/lkml/2008/1/29/387</a> <a href="#fnref:6" class="footnote-back-ref">↩</a></li>
<li id="fn:7">OpenFabrics Enterprise Distribution for Windows. <a href="https://www.openfabrics.org/index.php/-download-windows-installer-.html" target="_blank">https://www.openfabrics.org/index.php/-download-windows-installer-.html</a> <a href="#fnref:7" class="footnote-back-ref">↩</a></li>
<li id="fn:8">Mellanox OFED Drivers for VMware Infrastructure 3 and vSphere 4. <a href="http://www.mellanox.com/content/pages.php?pg=products_dyn&product_family=36&menu_section=34#tab-two" target="_blank">http://www.mellanox.com/content/pages.php?pg=products_dyn&product_family=36&menu_section=34#tab-two</a> <a href="#fnref:8" class="footnote-back-ref">↩</a></li>
<li id="fn:9">Sun's download page. <a href="http://www.sun.com/download/" target="_blank">http://www.sun.com/download/</a> <a href="#fnref:9" class="footnote-back-ref">↩</a></li>
<li id="fn:10">"Configuring SRP Devices With COMSTAR". Retrieved 4 February 2013. <a href="http://docs.oracle.com/cd/E23824_01/html/821-1459/gldpr.html" target="_blank">http://docs.oracle.com/cd/E23824_01/html/821-1459/gldpr.html</a> <a href="#fnref:10" class="footnote-back-ref">↩</a></li>
<li id="fn:11">Linux kernel version 2.6.24 change log. <a href="https://www.kernel.org/pub/linux/kernel/v2.6/ChangeLog-2.6.24" target="_blank">https://www.kernel.org/pub/linux/kernel/v2.6/ChangeLog-2.6.24</a> <a href="#fnref:11" class="footnote-back-ref">↩</a></li>
<li id="fn:12">D. Boutcher and D. Engebretsen, Linux Virtualization on IBM POWER5 Systems, Proceedings of the Linux Symposium, Vol. 1, July 2004, pp. 113-120. <a href="https://www.kernel.org/doc/mirror/ols2004v1.pdf#page=113" target="_blank">https://www.kernel.org/doc/mirror/ols2004v1.pdf#page=113</a> <a href="#fnref:12" class="footnote-back-ref">↩</a></li>
<li id="fn:13">IBM Systems Hardware Information Center, Virtual SCSI. <a href="http://publib.boulder.ibm.com/infocenter/systems/scope/hw/index.jsp?topic=/iphb1/iphb1_vios_concepts_stor.htm" target="_blank">http://publib.boulder.ibm.com/infocenter/systems/scope/hw/index.jsp?topic=/iphb1/iphb1_vios_concepts_stor.htm</a> <a href="#fnref:13" class="footnote-back-ref">↩</a></li>
<li id="fn:14">The SCST Project, an open source SCSI target implementation for Linux that includes an SRP target implementation. <a href="http://sourceforge.net/projects/scst/" target="_blank">http://sourceforge.net/projects/scst/</a> <a href="#fnref:14" class="footnote-back-ref">↩</a></li>
<li id="fn:15">OFED 1.5.4.1 Release Notes, OpenFabrics website, January 2012. <a href="http://www.openfabrics.org/downloads/OFED/release_notes/OFED_1.5.4.1_release_notes" target="_blank">http://www.openfabrics.org/downloads/OFED/release_notes/OFED_1.5.4.1_release_notes</a> <a href="#fnref:15" class="footnote-back-ref">↩</a></li>
<li id="fn:16">"SCSI RDMA Protocol". linux-iscsi.org. <a href="http://linux-iscsi.org/wiki/SCSI_RDMA_Protocol" target="_blank">http://linux-iscsi.org/wiki/SCSI_RDMA_Protocol</a> <a href="#fnref:16" class="footnote-back-ref">↩</a></li>
<li id="fn:17">Linus Torvalds (2012-01-18). "InfiniBand/SRP merge". lkml.org. <a href="https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=f59e842fc0871cd5baa213dc32e0ce8e5aaf4758" target="_blank">https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=f59e842fc0871cd5baa213dc32e0ce8e5aaf4758</a> <a href="#fnref:17" class="footnote-back-ref">↩</a></li>
<li id="fn:18">Linux kernel version 2.6.24 change log. <a href="https://www.kernel.org/pub/linux/kernel/v2.6/ChangeLog-2.6.24" target="_blank">https://www.kernel.org/pub/linux/kernel/v2.6/ChangeLog-2.6.24</a> <a href="#fnref:18" class="footnote-back-ref">↩</a></li>
<li id="fn:19">D. Boutcher and D. Engebretsen, Linux Virtualization on IBM POWER5 Systems, Proceedings of the Linux Symposium, Vol. 1, July 2004, pp. 113-120. <a href="https://www.kernel.org/doc/mirror/ols2004v1.pdf#page=113" target="_blank">https://www.kernel.org/doc/mirror/ols2004v1.pdf#page=113</a> <a href="#fnref:19" class="footnote-back-ref">↩</a></li>
<li id="fn:20">IBM Systems Hardware Information Center, Virtual SCSI. <a href="http://publib.boulder.ibm.com/infocenter/systems/scope/hw/index.jsp?topic=/iphb1/iphb1_vios_concepts_stor.htm" target="_blank">http://publib.boulder.ibm.com/infocenter/systems/scope/hw/index.jsp?topic=/iphb1/iphb1_vios_concepts_stor.htm</a> <a href="#fnref:20" class="footnote-back-ref">↩</a></li>
<li id="fn:21">"DDN SFA10000 User Guide" (PDF). ddn.com. 2012-01-18. <a href="http://www.ddn.com/pdfs/SFA10000_1.4.0_User_Guide_C.pdf" target="_blank">http://www.ddn.com/pdfs/SFA10000_1.4.0_User_Guide_C.pdf</a> <a href="#fnref:21" class="footnote-back-ref">↩</a></li>
<li id="fn:22">"DDN Corporate Overview, IB Storage 101 section" (PDF). ddn.com. 2012-01-18. <a href="http://www.hpcadvisorycouncil.com/events/switzerland_workshop/pdf/Presentations/Day%201/11_DDN.pdf" target="_blank">http://www.hpcadvisorycouncil.com/events/switzerland_workshop/pdf/Presentations/Day%201/11_DDN.pdf</a> <a href="#fnref:22" class="footnote-back-ref">↩</a></li>
<li id="fn:23">IBM (10 March 2014). "IBM FlashSystem Integration Guide". <a href="http://www-01.ibm.com/support/docview.wss?uid=ssg1S7004373&aid=1" target="_blank">http://www-01.ibm.com/support/docview.wss?uid=ssg1S7004373&aid=1</a> <a href="#fnref:23" class="footnote-back-ref">↩</a></li>
<li id="fn:24">Moellenkamp, Joerg. "PSARC/2009/111: SRP Target in Comstar". Retrieved 4 February 2013. <a href="http://www.c0t0d0s0.org/archives/5583-PSARC2009111-SRP-Target-in-Comstar.html" target="_blank">http://www.c0t0d0s0.org/archives/5583-PSARC2009111-SRP-Target-in-Comstar.html</a> <a href="#fnref:24" class="footnote-back-ref">↩</a></li>
</ol>

SCSI RDMA Protocol open-in-new

SCSI RDMA Protocol