Menu
Home
People
Places
Arts
History
Plants & Animals
Science
Life & Culture
Technology
Reference.org
Linux DM Multipath
open-in-new
<h2 id="considerations">Considerations</h2> <p>When utilizing Linux DM-MPIO in a <a href="/facts/Datacenter/DS5wJY41">datacenter</a> that has other <a href="/facts/Operating_system/8XTuvMh2">operating systems</a> and multipath solutions, key components of path management must be considered. </p> <ul><li>Load balancing — The workload is distributed across the available hardware components. Goal: Reduce I/O completion time, maximize throughput, and optimize resources</li> <li>Path failover and recovery — Utilizes redundant I/O channels to redirect application reads and writes when one or more paths are no longer available.</li></ul> <h2 id="history">History</h2> <p>DM-MPIO started as a patch set created by Joe Thornber, and was later maintained by Alasdair G Kergon at Red Hat. It was included in mainline Linux with kernel version 2.6.12, which was released on June 17, 2005.<a class="footnote-ref" id="fnref:7" href="#fn:7"><sup>7</sup></a> </p> <h2 id="components">Components</h2> <p>DM-MPIO in Linux consists of kernel components and user-space components. </p> <ul><li>Kernel – <a href="/facts/Device_mapper/MuSuYo3B"><i>device-mapper</i></a> – block subsystem that provides layering mechanism for block devices. <ul><li>dm-multipath – kernel module implementing the <i>multipath</i> device-mapper target.</li></ul></li> <li>User-space – <i>multipath-tools</i> – provides the tools to manage multipathed devices by instructing the device-mapper multipath module what to do. The tools consist of: <ul><li>Multipath: scans the system for multipathed devices, assembles them, updates the device-mapper's map.<a class="footnote-ref" id="fnref:8" href="#fn:8"><sup>8</sup></a></li> <li>Multipathd: daemon that waits for maps events, and then executes multipath and monitors the paths. Marks a path as failed when the path becomes faulty. Depending on the failback policy, it can reactivate the path.<a class="footnote-ref" id="fnref:9" href="#fn:9"><sup>9</sup></a></li> <li>Devmap-name: provides a meaningful device-name to <a href="/facts/Udev/89smRMUF">udev</a> for devmaps.<a class="footnote-ref" id="fnref:10" href="#fn:10"><sup>10</sup></a></li> <li>Kpartx: maps linear devmaps to device partitions to make multipath maps partitionable.<a class="footnote-ref" id="fnref:11" href="#fn:11"><sup>11</sup></a></li> <li>Multipath.conf: <a href="/facts/Configuration_file/DDzU0FLJ">configuration file</a> for the multipath daemon. Used to overwrite the built-in configuration table of multipathd.</li></ul></li></ul> <h3>Configuration file</h3> <p>The configuration file <i>/etc/multipath.conf</i> makes many of the DM-MPIO features user-configurable. The <i>multipath</i> command and the kernel daemon <i>multipathd</i> use information found in this file. The file is only consulted during the configuration of the multipath devices. Changes must be made prior to running the <i>multipath</i> command. Changes to the file afterward will require <i>multipath</i> to be executed again. </p><p>The multipath.conf has five sections:<a class="footnote-ref" id="fnref:12" href="#fn:12"><sup>12</sup></a> </p> <ol><li>System level defaults (<i>defaults</i>): User can override system level defaults.</li> <li>Blacklisted devices (<i>blacklist</i>): User specifies the list of devices that are not to be under the control of DM-MPIO.</li> <li>Blacklist exceptions (<i>blacklist_exceptions</i>): Specific devices to be treated as multipath devices even if listed in the blacklist.</li> <li>Storage controller specific settings (<i>devices</i>): User specified configuration settings will be applied to devices with specified "Vendor" and "Product" information.</li> <li>Device specific settings (<i>multipaths</i>): Fine tune the configuration settings for individual LUNs.</li></ol> <h2 id="terminology">Terminology</h2> <ul><li>HBA: <a href="/facts/Host_bus_adapter/nQbbDHF2">Host bus adapters</a> provide the physical interface between the input/output (I/O) host bus of <a href="/facts/Fibre_Channel/C7C0yreb">Fibre Channel</a> devices and the underlying Fibre Channel network.<a class="footnote-ref" id="fnref:13" href="#fn:13"><sup>13</sup></a></li> <li>Path: Connection from the server through the HBA to a specific LUN.</li> <li>DM Path States: The device mapper's view of the path condition. Only two conditions are possible: <ul><li>Active: The last I/O operation sent through this path successfully completed. Analogous to ready path state.</li> <li>Failed: The last I/O operation sent through this path did not successfully complete. Analogous to faulty path state.</li></ul></li> <li><a href="/facts/Failover/Lw0rXfv9">Failover</a>: When a path is determined to be in a failed state, a path that is in a ready state will be made active.<a class="footnote-ref" id="fnref:14" href="#fn:14"><sup>14</sup></a></li> <li>Failback: When a failed path is determined to be active again, multipathd may choose to failback to the path as determined by the failback policy.<a class="footnote-ref" id="fnref:15" href="#fn:15"><sup>15</sup></a></li> <li>Failback Policy: Four options as set in the multipath.conf configuration file. <ul><li>Immediate: Immediately failback to the highest priority path.</li> <li>Manual: The failed path is not monitored, requires user intervention to failback.</li> <li>Followover(for clusters): Only perform automatic failback when the first path of a pathgroup becomes active. This keeps a node from automatically failing back when another node requested the failover.</li> <li>Number of seconds: Wait for a specified number of seconds to allow the I/O to stabilize, then failback to the highest priority path.</li></ul></li> <li>Active/Active: In a system that has two storage controllers, each controller can process I/O.<a class="footnote-ref" id="fnref:16" href="#fn:16"><sup>16</sup></a></li> <li>Active/Passive: In a system that has two storage controllers, only one controller at a time is able to process I/O, the other (passive) is in a standby mode.<a class="footnote-ref" id="fnref:17" href="#fn:17"><sup>17</sup></a></li> <li>LUN: <a href="/facts/SCSI/pxXulrBr">SCSI</a> <a href="/facts/Logical_Unit_Number/6FkUxbw6">Logical Unit Number</a></li> <li><a href="/facts/WWID/ffdG9eWj">WWID</a>: Worldwide Identifier is an identifier for the multipath device that is guaranteed to be globally unique and unchanging.</li></ul> <h2 id="further-reading">Further reading</h2> <ul><li>Michael, T., Kabir, R., Giles, J. & Hull, J. (2006.) Configuring Linux to Enable Multipath I/O. Retrieved from <a href="http://www.dell.com/downloads/global/power/ps3q06-20060189-Michael.pdf">http://www.dell.com/downloads/global/power/ps3q06-20060189-Michael.pdf</a> <a href="https://web.archive.org/web/20120227111213/http://www.dell.com/downloads/global/power/ps3q06-20060189-Michael.pdf">Archived</a> 2012-02-27 at the <a href="/facts/Wayback_Machine/nmQ3a6JC">Wayback Machine</a></li> <li>Goggin, E., Kergon, A., Varoqui, C., & Olien, D. (2005) Proceedings of the Linux Symposium – Linux Multipathing. Retrieved from <a href="https://web.archive.org/web/20101227213252/http://www.linuxinsight.com/files/ols2005/goggin-reprint.pdf">https://web.archive.org/web/20101227213252/http://www.linuxinsight.com/files/ols2005/goggin-reprint.pdf</a></li> <li>Red Hat Documentation. (n.d.) Red Hat Enterprise Linux 6, DM Multipath. Retrieved from <a href="https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html-single/DM_Multipath/">https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html-single/DM_Multipath/</a></li> <li>Varoqui, C. (2010.) The Linux multipath implementation. Retrieved from <a href="http://christophe.varoqui.free.fr/refbook.html">http://christophe.varoqui.free.fr/refbook.html</a></li></ul> <h2 id="references">References</h2> <ol> <li id="fn:1"><p>Goggin, Edward; Kergon, Alasdair; Varoqui, Christophe; Olien, David. "Linux Multipathing" (PDF). <a href="https://www.kernel.org/doc/ols/2005/ols2005v1-pages-155-176.pdf" target="_blank">https://www.kernel.org/doc/ols/2005/ols2005v1-pages-155-176.pdf</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></p></li> <li id="fn:2"><p>Varoqui, Christophe. "The Linux multipath implementation". <a href="http://christophe.varoqui.free.fr/refbook.html" target="_blank">http://christophe.varoqui.free.fr/refbook.html</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></p></li> <li id="fn:3"><p>Oberg, Michael. "Exploration of Parallel Storage Architectures for a Blue Gene/L on the TeraGrid" (PDF). Archived from the original (PDF) on 2013-12-02. Retrieved 2012-02-13. <a href="https://web.archive.org/web/20131202232044/http://www.csc.cs.colorado.edu/~matthew/papers/200804-lci2008-maelstrom-paper.pdf" target="_blank">https://web.archive.org/web/20131202232044/http://www.csc.cs.colorado.edu/~matthew/papers/200804-lci2008-maelstrom-paper.pdf</a> <a href="#fnref:3" class="footnote-back-ref">↩</a></p></li> <li id="fn:4"><p>van Vugt, Sander (8 March 2010). A Practical Guide to XEN High Availability. Books4Brains. ISBN 9789072389084. <a href="9789072389084" target="_blank">9789072389084</a> <a href="#fnref:4" class="footnote-back-ref">↩</a></p></li> <li id="fn:5"><p>SUSE. "Storage Administration Guide, SUSE Linux Enterprise Server 11 SP1" (PDF). SLES11 Documentation, pg. 49. <a href="http://www.suse.com/documentation/sles11/pdfdoc/stor_admin/stor_admin.pdf" target="_blank">http://www.suse.com/documentation/sles11/pdfdoc/stor_admin/stor_admin.pdf</a> <a href="#fnref:5" class="footnote-back-ref">↩</a></p></li> <li id="fn:6"><p>"Multipath support in the device mapper [LWN.net]". <a href="https://lwn.net/Articles/124703/" target="_blank">https://lwn.net/Articles/124703/</a> <a href="#fnref:6" class="footnote-back-ref">↩</a></p></li> <li id="fn:7"><p>Linux 2.6.12 kernelnewbies.org <a href="http://kernelnewbies.org/Linux_2_6_12" target="_blank">http://kernelnewbies.org/Linux_2_6_12</a> <a href="#fnref:7" class="footnote-back-ref">↩</a></p></li> <li id="fn:8"><p>SUSE. "Storage Administration Guide, SUSE Linux Enterprise Server 11 SP1" (PDF). SLES11 Documentation, pg. 49. <a href="http://www.suse.com/documentation/sles11/pdfdoc/stor_admin/stor_admin.pdf" target="_blank">http://www.suse.com/documentation/sles11/pdfdoc/stor_admin/stor_admin.pdf</a> <a href="#fnref:8" class="footnote-back-ref">↩</a></p></li> <li id="fn:9"><p>SUSE. "Storage Administration Guide, SUSE Linux Enterprise Server 11 SP1" (PDF). SLES11 Documentation, pg. 49. <a href="http://www.suse.com/documentation/sles11/pdfdoc/stor_admin/stor_admin.pdf" target="_blank">http://www.suse.com/documentation/sles11/pdfdoc/stor_admin/stor_admin.pdf</a> <a href="#fnref:9" class="footnote-back-ref">↩</a></p></li> <li id="fn:10"><p>SUSE. "Storage Administration Guide, SUSE Linux Enterprise Server 11 SP1" (PDF). SLES11 Documentation, pg. 49. <a href="http://www.suse.com/documentation/sles11/pdfdoc/stor_admin/stor_admin.pdf" target="_blank">http://www.suse.com/documentation/sles11/pdfdoc/stor_admin/stor_admin.pdf</a> <a href="#fnref:10" class="footnote-back-ref">↩</a></p></li> <li id="fn:11"><p>SUSE. "Storage Administration Guide, SUSE Linux Enterprise Server 11 SP1" (PDF). SLES11 Documentation, pg. 49. <a href="http://www.suse.com/documentation/sles11/pdfdoc/stor_admin/stor_admin.pdf" target="_blank">http://www.suse.com/documentation/sles11/pdfdoc/stor_admin/stor_admin.pdf</a> <a href="#fnref:11" class="footnote-back-ref">↩</a></p></li> <li id="fn:12"><p>RedHat. "Using Device-Mapper Multipath". Archived from the original on 2011-12-29. Retrieved 2012-01-30. <a href="https://web.archive.org/web/20111229064407/http://www.centos.org/docs/5/html/5.1/DM_Multipath/config_file_overview.html" target="_blank">https://web.archive.org/web/20111229064407/http://www.centos.org/docs/5/html/5.1/DM_Multipath/config_file_overview.html</a> <a href="#fnref:12" class="footnote-back-ref">↩</a></p></li> <li id="fn:13"><p>Gupta, Meeta (2002). Storage Area Network Fundamentals. Indianapolis, IN: Cisco Press. p. 81. ISBN 1-58705-065-X. <a href="1-58705-065-X" target="_blank">1-58705-065-X</a> <a href="#fnref:13" class="footnote-back-ref">↩</a></p></li> <li id="fn:14"><p>Anderson, Michael. "SCSI Mid-Level Multipath" (PDF). <a href="http://landley.net/kdocs/ols/2003/ols2003-pages-23-33.pdf" target="_blank">http://landley.net/kdocs/ols/2003/ols2003-pages-23-33.pdf</a> <a href="#fnref:14" class="footnote-back-ref">↩</a></p></li> <li id="fn:15"><p>"Storage Administration Guide, SLES11 Documentation" (PDF). SUSE. p. 73. <a href="http://www.suse.com/documentation/sles11/pdfdoc/stor_admin/stor_admin.pdf" target="_blank">http://www.suse.com/documentation/sles11/pdfdoc/stor_admin/stor_admin.pdf</a> <a href="#fnref:15" class="footnote-back-ref">↩</a></p></li> <li id="fn:16"><p>Centos. "Overview of DM-Multipath". Using Device-Mapper Multipath. Archived from the original on 2012-03-05. Retrieved 2012-01-30. <a href="https://web.archive.org/web/20120305235712/http://www.centos.org/docs/5/html/5.2/DM_Multipath/MPIO_description.html" target="_blank">https://web.archive.org/web/20120305235712/http://www.centos.org/docs/5/html/5.2/DM_Multipath/MPIO_description.html</a> <a href="#fnref:16" class="footnote-back-ref">↩</a></p></li> <li id="fn:17"><p>Centos. "Overview of DM-Multipath". Using Device-Mapper Multipath. Archived from the original on 2012-03-05. Retrieved 2012-01-30. <a href="https://web.archive.org/web/20120305235712/http://www.centos.org/docs/5/html/5.2/DM_Multipath/MPIO_description.html" target="_blank">https://web.archive.org/web/20120305235712/http://www.centos.org/docs/5/html/5.2/DM_Multipath/MPIO_description.html</a> <a href="#fnref:17" class="footnote-back-ref">↩</a></p></li> </ol>