Menu
Home
People
Places
Arts
History
Plants & Animals
Science
Life & Culture
Technology
Reference.org
Semantic file system
open-in-new
<h2 id="history">History</h2> <p>The notion of semantic file system was proposed in 1991 by researchers of the <a href="/facts/Massachusetts_Institute_of_Technology/Xk49vNA8">MIT</a> and <a href="/facts/Mines_Paris_%E2%80%93_PSL/xLpmkXIf">École des Mines de Paris</a>.<a class="footnote-ref" id="fnref:1" href="#fn:1"><sup>1</sup></a> They proposed an integrated system whose main query interface looked like a traditional file system interface via a virtual directory system that interpreted a path as a <a href="/facts/Conjunctive_query/slsqwIb6">conjunctive query</a>. Their implementation had automatic extraction of the relevant <a href="/facts/Metadata/qt35z79C">metadata</a> via what they called <a href="/facts/File_format/qrXbwJaH">file type</a> specific transducers. </p><p>Starting in around 2004, a new wave of implementations centered on manual tagging of files and folders. </p><p>In 2008, researchers proposed to integrate semantic file systems with <a href="/facts/Semantic_Web/NFHo9dB7">Semantic Web</a> technologies.<a class="footnote-ref" id="fnref:2" href="#fn:2"><sup>2</sup></a> </p> <h2 id="types-of-metadata">Types of metadata</h2> <h3>Tags</h3> <p>Tags can be used instead of folders to circumvent the limits of a hierarchical model. </p> <h3>File type-specific</h3> <p>Gifford et al.<a class="footnote-ref" id="fnref:3" href="#fn:3"><sup>3</sup></a> suggested the idea of <a href="/facts/File_format/qrXbwJaH">file type</a>-specific metadata automatically extracted by a file-type specific transducer. </p><p>For instance, for a source code text file, metadata could include the names of the procedures that the program exports or imports, procedure types, and the files included by the program. For a document, its date, author, title and structure (sections and subsections). For an e-mail, its sender, recipient and subject. </p> <h3>Lineage</h3> <p>In scientific workflows, provenance of a data file is important. A scientist might want to select a results file by filtering by the input dataset. </p> <h2 id="architecture">Architecture</h2> <p>Vasudevan and Pazandak<a class="footnote-ref" id="fnref:4" href="#fn:4"><sup>4</sup></a> introduce the distinction between integrated and augmented approaches: </p> <ul><li>In integrated approaches, semantics are a feature of the file system. <ul><li>Tightly coupled systems are implemented within a file system</li> <li>Loosely coupled systems are implemented on top of a classical file system, but hide its interface.</li></ul></li> <li>In augmented approaches, semantics are an abstraction on top of a classical file system. Access to the classical file system interface is maintained, the user can choose.</li></ul> <p>They suggest <a href="/facts/Open_systems_architecture/renWEjxP">Open systems architecture</a> as being well adapted to semantic file system implementations. </p> <h3>Compatibility with hierarchical file systems</h3> <p>Even integrated semantic file systems may choose to expose an interface for compatibility with existing local or <a href="/facts/Distributed_file_system/Bz61f7Ce">distributed file system</a> protocols. For instance, Gifford et al.’s 1991 implementation was fully compatible with <a href="/facts/Network_File_System/jWRRALVo">NFS</a>.<a class="footnote-ref" id="fnref:5" href="#fn:5"><sup>5</sup></a> </p> <h3>Metadata storage</h3> <p><a href="/facts/Extended_file_attributes/1wmerAyK">Extended file attributes</a> provided by the file system can be a way to store the metadata. </p><p>A <a href="/facts/Relational_database/pEKGmJLy">relational database</a> is another very frequent way to store the metadata. </p> <h2 id="research-implementations">Research implementations</h2> <table><tbody><tr><th>Name</th><th>Type</th><th>Metadata</th><th>OS</th><th>Date</th><th>Comment</th></tr><tr><td>Lineage File System<a class="footnote-ref" id="fnref:6" href="#fn:6"><sup>6</sup></a></td><td>File system extension</td><td>Lineage</td><td><a href="/facts/Linux/9FvCKSTq">Linux</a></td><td>2005</td><td>Modifies the Linux kernel to log all process creation and file-related system calls. Uses a MySQL database.</td></tr><tr><td><a href="https://web.archive.org/web/20091124184129/http://www.uni-koblenz.de/FB4/Institutes/IFI/AGStaab/Research/SemFS/">SemFS (formerly TagFS)</a><a class="footnote-ref" id="fnref:7" href="#fn:7"><sup>7</sup></a></td><td>File system</td><td>Tags</td><td><a href="/facts/Linux/9FvCKSTq">Linux</a>, Windows</td><td>2006</td><td>On Windows, can be mounted as a WebDAV drive. On Linux, based on <a href="/facts/Filesystem_in_Userspace/RnSLIf1H">FUSE</a>. Tags are stored as RDF. Uses an internal file system, not exposed.</td></tr><tr><td>SFS<a class="footnote-ref" id="fnref:8" href="#fn:8"><sup>8</sup></a></td><td>File system extension</td><td>File type-specific</td><td>Linux</td><td>1991</td><td></td></tr></tbody></table> <h2 id="implementations">Implementations</h2> <table><tbody><tr><th>Name</th><th>Type</th><th>Metadata</th><th><a href="/facts/Operating_system/8XTuvMh2">OS</a></th><th><a href="/facts/Software_license/7enfIqhL">License</a></th><th><a href="/facts/Programming_language/cWTYbgWa">Programming language</a>(s)</th><th>Last update</th><th>Comment</th></tr><tr><td><a href="/facts/Be_File_System/fQ5uUJj9">Be File System</a> (BFS)</td><td>File system</td><td></td><td><a href="/facts/BeOS/tGYfe835">BeOS</a></td><td>Proprietary; last version is <a href="/facts/Freeware/c8oGA9ju">freeware</a></td><td></td><td></td><td>Metadata is stored in <a href="/facts/Extended_file_attributes/1wmerAyK">extended file attributes</a>. Works with file manager <a href="/facts/Tracker_(file_manager)/RuujGB32">Tracker</a></td></tr><tr><td><a href="https://darkfeline.github.io/dantalian/">dantalian</a></td><td>File system extension</td><td>Tags</td><td><a href="/facts/Linux/9FvCKSTq">Linux</a> and contiguous <a href="/facts/POSIX/mQkSt234">POSIX</a>-compatible file systems</td><td><a href="/facts/Apache_License/5AeIjas8">Apache</a> 2</td><td><a href="/facts/Python_(programming_language)/YbuGqofa">Python</a></td><td>2016</td><td>Uses <a href="/facts/Symbolic_link/laB0S68t">symlinks</a></td></tr><tr><td><a href="https://code.google.com/p/dhtfs/">dhtfs</a></td><td>User-level file system extension</td><td>Tags</td><td>Linux</td><td><a href="/facts/BSD_licenses/XLtmLp5x">BSD 3-clause</a></td><td><a href="/facts/Python_(programming_language)/YbuGqofa">Python</a></td><td>2009</td><td>Based on <a href="/facts/Filesystem_in_Userspace/RnSLIf1H">FUSE</a></td></tr><tr><td><a href="http://silkwoodsoftware.com/">Elyse</a></td><td>Graphical <a href="/facts/File_manager/t7CSQ40a">file manager</a></td><td>Tags</td><td>Windows and MacOS</td><td>Proprietary, no cost</td><td></td><td>2021</td><td></td></tr><tr><td><a href="https://github.com/clipland/fuse-taglayer">Fuse::TagLayer</a></td><td>File system extension</td><td>Tags</td><td>Linux</td><td><a href="/facts/GNU_General_Public_License/S75KhLSX">GPL</a> v3 / <a href="/facts/Artistic_License/9vHBm47P">AL</a> v2</td><td><a href="/facts/Perl/fx31kjlT">Perl</a></td><td>2013</td><td>Based on <a href="/facts/Filesystem_in_Userspace/RnSLIf1H">FUSE</a></td></tr><tr><td><a href="https://tabbles.net/">Tabbles</a></td><td>Graphical <a href="/facts/File_manager/t7CSQ40a">file manager</a></td><td>Tags</td><td>Windows <a href="/facts/Windows_Vista/IASEUCTB">Vista</a> to <a href="/facts/Windows_11/J2ywV8r4">11</a></td><td>Proprietary, freemium</td><td><a href="/facts/.NET_Framework/J6QDAUhP">.NET Framework</a></td><td></td><td>Uses a <a href="/facts/Microsoft_SQL_Server/eCdt2JBL">SQL Server</a> relational database.</td></tr><tr><td><a href="http://www.tag2find.com/">Tag2Find</a></td><td></td><td>Tags</td><td>Windows <a href="/facts/Windows_XP/IozrkAYS">XP</a> and <a href="/facts/Windows_Vista/IASEUCTB">Vista</a> 32-bit</td><td></td><td></td><td>2007</td><td></td></tr><tr><td><a href="http://akslab.com/">TagsForAll</a></td><td>Graphical <a href="/facts/File_manager/t7CSQ40a">file manager</a></td><td>Tags</td><td>Windows x64</td><td>Freemium</td><td></td><td>2014</td><td>70 tag limit in free version. Metadata is stored in two places: in files as ADS (Alternate Data Stream for <a href="/facts/NTFS/dryNM3BK">NTFS</a>), and in local database.</td></tr><tr><td><a href="/facts/Tagsistant/j4H3hTYQ">Tagsistant</a></td><td>File system</td><td>Tags</td><td>Linux</td><td><a href="/facts/GNU_General_Public_License/S75KhLSX">GPL</a></td><td><a href="/facts/C_(programming_language)/Ky2No763">C</a></td><td>2017</td><td>Tag-based, based on <a href="/facts/Filesystem_in_Userspace/RnSLIf1H">FUSE</a></td></tr><tr><td><a href="https://www.tagspaces.org/">TagSpaces</a></td><td>Graphical <a href="/facts/File_manager/t7CSQ40a">file manager</a>, web or desktop (uses <a href="/facts/Electron_(software_framework)/15eHlAWQ">Electron</a>)</td><td>Tags</td><td>Windows, macOS, Linux, and <a href="/facts/Android_(operating_system)/nNyHb0xz">Android</a>.</td><td><a href="/facts/GNU_Affero_General_Public_License/n1h9A893">AGPL</a> (Freemium)</td><td><a href="/facts/TypeScript/W9J27V1N">TypeScript</a>, <a href="/facts/JavaScript/XhhIIEsl">JavaScript</a>, <a href="/facts/Java_(programming_language)/9ScgFyAL">Java</a>, <a href="/facts/Objective-C/5mM557BW">Objective-C</a></td><td>Continues</td><td></td></tr><tr><td><a href="http://tagxfs.sourceforge.net/">tagxfs</a></td><td>File system extension</td><td>Tags</td><td>Linux</td><td><a href="/facts/Boost_(C%2B%2B_libraries)/vGLGG51A">Boost</a> Software License 1.0</td><td><a href="/facts/C%2B%2B/Et0F2qn9">C++</a></td><td>2013</td><td>Extends the user space file system to a tag based hierarchy.</td></tr><tr><td><a href="http://tmsu.org/">TMSU</a></td><td>Virtual file system</td><td>Tags</td><td></td><td></td><td></td><td>2022</td><td>Uses a <a href="/facts/SQLite/1PwYzIft">SQLite</a> <a href="/facts/Relational_database/pEKGmJLy">relational database</a>.</td></tr><tr><td><a href="http://transparentag.sourceforge.net/">TransparenTag</a></td><td>File system</td><td>Tags</td><td>Linux, <a href="/facts/Berkeley_Software_Distribution/XQKcURrh">BSD</a></td><td><a href="/facts/GNU_General_Public_License/S75KhLSX">GPL</a> v2</td><td><a href="/facts/OCaml/ThBGLND8">OCaml</a></td><td>2013</td><td>Data and tags are stored as regular files</td></tr><tr><td><a href="/facts/WinFS/ytcFY3ne">WinFS</a></td><td>File system and manager</td><td>Any type</td><td><a href="/facts/Windows_XP/IozrkAYS">Windows XP</a></td><td>Proprietary</td><td><a href="/facts/.NET_Framework/J6QDAUhP">.NET Framework</a></td><td>2006</td><td>Uses a relational database</td></tr><tr><td><a href="https://code.google.com/p/xtagfs/">xtagfs</a></td><td>File system extension</td><td>Tags</td><td><a href="/facts/MacOS/Y93vVpTm">MacOS X</a></td><td><a href="/facts/GNU_General_Public_License/S75KhLSX">GPL</a> v2</td><td><a href="/facts/Python_(programming_language)/YbuGqofa">Python</a></td><td>2009</td><td>Based on <a href="/facts/Filesystem_in_Userspace/RnSLIf1H">FUSE</a></td></tr></tbody></table> <h2 id="see-also">See also</h2> <ul><li><a href="/facts/Content-addressable_storage/cIw1MKwK">Content-addressable storage</a></li> <li><a href="/facts/Logic_File_System/K0A8v1al">Logic File System</a></li> <li><a href="/facts/Semantic_desktop/Ga9xSFye">Semantic desktop</a></li> <li><a href="/facts/Semantic_Web/NFHo9dB7">Semantic Web</a></li></ul> <h2 id="external-links">External links</h2> <p>Research & Specifications </p> <ul><li><a href="http://eprints.cs.univie.ac.at/158/1/schandl.pdf">The Sile Model. A Semantic File System Infrastructure for the Desktop</a></li> <li><a href="https://web.archive.org/web/20060905031903/http://www.psrg.lcs.mit.edu/publications/Papers/sfs.pdf">Semantic FS @ MIT Programming Systems Research Group</a></li> <li><a href="https://blueprints.launchpad.net/ubuntu/+spec/tag-based-filesystem">Launchpad Blueprints: A tag-based filesystem for Ubuntu</a></li> <li><a href="https://web.archive.org/web/20070927003401/http://www.namesys.com/whitepaper.html">ReiserFS future vision</a></li> <li><a href="http://semanticweb.org/wiki/SemFS#Related_Work_and_Publications">external list of related work on semantic file systems @ semanticweb.org</a></li> <li><a href="https://www.nayuki.io/page/designing-better-file-organization-around-tags-not-hierarchies#hierarchical-organization-is-clumsy">"Designing better file organization around tags, not hierarchies" detailed writeup by Nayuki</a></li> <li><a href="https://sites.google.com/view/non-directory/home">Non-Directory Filesystem</a></li></ul> <h2 id="references">References</h2> <ol> <li id="fn:1"><p>Gifford, David; Jouvelot, Pierre; Sheldon, Mark A.; O’Toole, James W. Jr. (1991). "Semantic file systems" (PDF). ACM Operating Systems Review. 25 (5): 16–25. doi:10.1145/121133.121138. <a href="https://web.mit.edu/6.826/www/notes/HO13.pdf" target="_blank">https://web.mit.edu/6.826/www/notes/HO13.pdf</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></p></li> <li id="fn:2"><p>Faubel, Sebastian; Kuschel, Christian (2008). "Towards Semantic File System Interfaces" (PDF). ISWC (Posters & Demos). <a href="http://ftp.informatik.rwth-aachen.de/Publications/CEUR-WS/Vol-401/iswc2008pd_submission_8.pdf" target="_blank">http://ftp.informatik.rwth-aachen.de/Publications/CEUR-WS/Vol-401/iswc2008pd_submission_8.pdf</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></p></li> <li id="fn:3"><p>Gifford, David; Jouvelot, Pierre; Sheldon, Mark A.; O’Toole, James W. Jr. (1991). "Semantic file systems" (PDF). ACM Operating Systems Review. 25 (5): 16–25. doi:10.1145/121133.121138. <a href="https://web.mit.edu/6.826/www/notes/HO13.pdf" target="_blank">https://web.mit.edu/6.826/www/notes/HO13.pdf</a> <a href="#fnref:3" class="footnote-back-ref">↩</a></p></li> <li id="fn:4"><p>Vasudevan, Venu; Pazandak, Paul (1997). "Semantic File Systems". Object Services and Consulting, Inc. Retrieved 2024-03-05. <a href="http://www.objs.com/survey/OFSExt.htm" target="_blank">http://www.objs.com/survey/OFSExt.htm</a> <a href="#fnref:4" class="footnote-back-ref">↩</a></p></li> <li id="fn:5"><p>Gifford, David; Jouvelot, Pierre; Sheldon, Mark A.; O’Toole, James W. Jr. (1991). "Semantic file systems" (PDF). ACM Operating Systems Review. 25 (5): 16–25. doi:10.1145/121133.121138. <a href="https://web.mit.edu/6.826/www/notes/HO13.pdf" target="_blank">https://web.mit.edu/6.826/www/notes/HO13.pdf</a> <a href="#fnref:5" class="footnote-back-ref">↩</a></p></li> <li id="fn:6"><p>Sar, Can; Cao, Pei (2005). "Lineage File System". Stanford University. Retrieved 2024-03-14. <a href="https://crypto.stanford.edu/~cao/lineage" target="_blank">https://crypto.stanford.edu/~cao/lineage</a> <a href="#fnref:6" class="footnote-back-ref">↩</a></p></li> <li id="fn:7"><p>Bloehdorn, Stephan; Völkel, Max (2006). "TagFS — Tag Semantics for Hierarchical File Systems". WWW Conference Proceedings – via CiteSeerX. <a href="https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=8389bf4a6b9ed2f67e7afc75619933d0bdcb9ad8" target="_blank">https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=8389bf4a6b9ed2f67e7afc75619933d0bdcb9ad8</a> <a href="#fnref:7" class="footnote-back-ref">↩</a></p></li> <li id="fn:8"><p>Gifford, David; Jouvelot, Pierre; Sheldon, Mark A.; O’Toole, James W. Jr. (1991). "Semantic file systems" (PDF). ACM Operating Systems Review. 25 (5): 16–25. doi:10.1145/121133.121138. <a href="https://web.mit.edu/6.826/www/notes/HO13.pdf" target="_blank">https://web.mit.edu/6.826/www/notes/HO13.pdf</a> <a href="#fnref:8" class="footnote-back-ref">↩</a></p></li> </ol>