Menu
Home
People
Places
Arts
History
Plants & Animals
Science
Life & Culture
Technology
Reference.org
Moose File System
open-in-new
<h2 id="design">Design</h2> <p>The MooseFS follows similar design principles as <a href="/facts/Fossil_(file_system)/g1rVbjKX">Fossil</a>, <a href="/facts/Google_File_System/mDeBYrAb">Google File System</a>, <a href="/facts/Lustre_(file_system)/byz4hsWW">Lustre</a> or <a href="/facts/Ceph_(file_system)/EU6zMFbx">Ceph</a>. The file system comprises three components: </p> <ul><li>Metadata server (MDS) — manages the location (layout) of files, file access and namespace hierarchy. The current version of MooseFS does support multiple metadata servers and automatic <a href="/facts/Failover/Lw0rXfv9">failover</a>. Clients only talk to the MDS to retrieve/update a file's layout and attributes; the data itself is transferred directly between clients and chunk servers. The Metadata server is a user-space <a href="/facts/Daemon_(computer_software)/bzuEVGGX">daemon</a>; the metadata is kept in memory and lazily stored on local disk.</li> <li>Metalogger server — periodically pulls the metadata from the MDS to store it for backup. Since version 1.6.5, this is an optional feature.</li> <li>Chunk servers (CSS) — store the data and optionally replicate it among themselves. There can be many of them, though the scalability limit has not been published. The biggest cluster reported so far consists of 160 servers.<a class="footnote-ref" id="fnref:1" href="#fn:1"><sup>1</sup></a> The Chunk server is also a user-space <a href="/facts/Daemon_(computer_software)/bzuEVGGX">daemon</a> that relies on the underlying local file system to manage the actual storage.</li> <li>Clients — talk to both the MDS and CSS. MooseFS clients mount the file system into user-space via <a href="/facts/Filesystem_in_Userspace/RnSLIf1H">FUSE</a>.</li></ul> <h2 id="features">Features</h2> <p>To achieve high reliability and performance MooseFS offers the following features: </p> <ul><li><a href="/facts/Fault-tolerance/rasrYlJk">Fault-tolerance</a> — MooseFS uses <a href="/facts/Replication_(computer_science)/QIaooMjL">replication</a>, data can be replicated across chunkservers, the replication ratio (<i>N</i>) is set per file/directory. If (<i>N</i>−1) replicas fail the data will still be available. At the moment MooseFS does not offer any other technique for <a href="/facts/Fault-tolerance/rasrYlJk">fault-tolerance</a>. <a href="/facts/Fault-tolerance/rasrYlJk">Fault-tolerance</a> for very big files thus requires vast amount of space - <i>N</i> × filesize instead of filesize + (<i>N</i> × stripesize) as would be the case for <a href="/facts/RAID_4/StvCqldY">RAID 4</a>, <a href="/facts/RAID_5/StvCqldY">RAID 5</a> or <a href="/facts/RAID_6/StvCqldY">RAID 6</a>. Version 4.x PRO of MooseFS implements 8+<i>n</i> <a href="/facts/Erasure_code/ea1C1z2X">Erasure Coding</a>.</li> <li><a href="/facts/Data_striping/bIIyMsyp">Striping</a> — Large files are divided into chunks (up to 64 <a href="/facts/Megabyte/RBF4XpWb">megabytes</a>) that might be stored on different chunk servers in order to achieve higher aggregate bandwidth.</li> <li><a href="/facts/Load_balancing_(computing)/jddk6ang">Load balancing</a> — MooseFS attempts to use storage resources equally, the current algorithm seems to take into account only the consumed space.</li> <li><a href="/facts/Security/B4uxkyxh">Security</a> — Apart from classical <a href="/facts/POSIX/mQkSt234">POSIX</a> file permissions, since the 1.6 release MooseFS offers a simple, NFS-like, <a href="/facts/Authentication/64y4S69b">authentication</a>/<a href="/facts/Authorization/FYFAkCH2">authorization</a>.</li> <li><a href="/facts/Snapshot_(computer_storage)/LKm3NX2A">Coherent snapshots</a> — Quick, low-overhead snapshots.</li> <li>Transparent "trash bin" — Deleted files are retained for a configurable period of time.</li> <li>Data tiering / storage classes — Possibility to "label" servers, create label definitions called "Storage Classes" and decide, on which types of servers the data is stored<a class="footnote-ref" id="fnref:2" href="#fn:2"><sup>2</sup></a></li> <li><a href="/facts/Disk_quota/ld30QhUB">"Project" quotas</a> support</li> <li>POSIX locks, flock locks support</li></ul> <h2 id="hardware-software-and-networking">Hardware, software and networking</h2> <p>Similarly to other cluster-based file systems MooseFS uses <a href="/facts/Commodity_hardware/fbCrXREb">commodity hardware</a> running a <a href="/facts/POSIX/mQkSt234">POSIX</a> compliant operating system. <a href="/facts/TCP/IP/KNkxtSpw">TCP/IP</a> is used as the interconnect. </p> <h2 id="moosefs-in-figures">MooseFS in figures</h2> <p>Source:<a class="footnote-ref" id="fnref:3" href="#fn:3"><sup>3</sup></a> </p> <ul><li>Storage size is up to: 264 <a href="/facts/Byte/BNHUp9Qq">Bytes</a> = 16 EiB = 16 384 PiB</li> <li>Single file size is up to: 257 Bytes = 128 PiB</li> <li>Number of files is up to: 231 = 2.1 × 109</li> <li>Number of active clients is unlimited it depends on number of file descriptors in the system</li></ul> <h2 id="see-also">See also</h2> <ul> <li>Free and open-source software portal</li></ul> <ul><li><a href="/facts/BeeGFS/4NnHwQCr">BeeGFS</a></li> <li><a href="/facts/Ceph_(file_system)/EU6zMFbx">Ceph</a></li> <li><a href="/facts/Distributed_file_system/Bz61f7Ce">Distributed file system</a></li> <li><a href="/facts/GlusterFS/sVzz6ERY">GlusterFS</a></li> <li><a href="/facts/Google_File_System/mDeBYrAb">Google File System</a></li> <li><a href="/facts/List_of_file_systems/GMUaz18t">List of file systems § Distributed fault-tolerant file systems</a></li> <li><a href="/facts/LizardFS/rCxwpGpd">LizardFS</a> – a fork of MooseFS v. 1.6.x</li> <li><a href="/facts/Lustre_(file_system)/byz4hsWW">Lustre</a></li></ul> <h2 id="external-links">External links</h2> <ul><li><a href="https://moosefs.com">Official website</a> </li> <li><a href="https://github.com/moosefs/moosefs">moosefs</a> on <a href="/facts/GitHub/n1hNCsc3">GitHub</a></li> <li><a href="https://sourceforge.net/projects/moosefs/">MooseFS</a> on <a href="/facts/SourceForge/lxYAgUAX">SourceForge</a></li></ul> <h2 id="references">References</h2> <ol> <li id="fn:1"><p>Mariusz Gądarowski (2010-04-01). "MooseFS: Bezpieczny i rozproszony system plików" (PDF) (in Polish). Linux Magazine Poland. <a href="http://moosefs.org/tl_files/moosefs-rozproszony-system-plikow-linux-magazine201004.pdf" target="_blank">http://moosefs.org/tl_files/moosefs-rozproszony-system-plikow-linux-magazine201004.pdf</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></p></li> <li id="fn:2"><p>MooseFS 3.0 Storage Classes Manual https://moosefs.com/Content/Downloads/moosefs-storage-classes-manual.pdf <a href="https://moosefs.com/Content/Downloads/moosefs-storage-classes-manual.pdf" target="_blank">https://moosefs.com/Content/Downloads/moosefs-storage-classes-manual.pdf</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></p></li> <li id="fn:3"><p>MooseFS Factsheet <a href="https://moosefs.com/factsheet" target="_blank">https://moosefs.com/factsheet</a> <a href="#fnref:3" class="footnote-back-ref">↩</a></p></li> </ol>