Menu
Home
People
Places
Arts
History
Plants & Animals
Science
Life & Culture
Technology
Reference.org
TeraGrid
open-in-new
<h2 id="history">History</h2> <p>The US <a href="/facts/National_Science_Foundation/FlDT8TbW">National Science Foundation</a> (NSF) issued a solicitation asking for a "distributed terascale facility" from program director Richard L. Hilderbrandt.<a class="footnote-ref" id="fnref:1" href="#fn:1"><sup>1</sup></a> The TeraGrid project was launched in August 2001 with $53 million in funding to four sites: the <a href="/facts/National_Center_for_Supercomputing_Applications/vNNK19se">National Center for Supercomputing Applications</a> (NCSA) at the <a href="/facts/University_of_Illinois_at_Urbana-Champaign/gusw7HoS">University of Illinois at Urbana-Champaign</a>, the <a href="/facts/San_Diego_Supercomputer_Center/LtSAk3Wh">San Diego Supercomputer Center</a> (SDSC) at the <a href="/facts/University_of_California%2c_San_Diego/srMClpNv">University of California, San Diego</a>, the <a href="/facts/University_of_Chicago/z6zRMUmV">University of Chicago</a> <a href="/facts/Argonne_National_Laboratory/EAoR1Rnm">Argonne National Laboratory</a>, and the Center for Advanced Computing Research (CACR) at the <a href="/facts/California_Institute_of_Technology/3FPcp7BL">California Institute of Technology</a> in <a href="/facts/Pasadena%2c_California/Y6AEEI03">Pasadena, California</a>. </p><p>The design was meant to be an extensible distributed <a href="/facts/Open_system_(computing)/tVwr9W6r">open system</a> from the start.<a class="footnote-ref" id="fnref:2" href="#fn:2"><sup>2</sup></a> In October 2002, the <a href="/facts/Pittsburgh_Supercomputing_Center/AqVMSaa6">Pittsburgh Supercomputing Center</a> (PSC) at <a href="/facts/Carnegie_Mellon_University/XKd7L2KA">Carnegie Mellon University</a> and the <a href="/facts/University_of_Pittsburgh/U3lxG2L1">University of Pittsburgh</a> joined the TeraGrid as major new partners when NSF announced $35 million in supplementary funding. The TeraGrid network was transformed through the ETF project from a 4-site <a href="/facts/Mesh_networking/XnVSy5rK">mesh</a> to a dual-hub <a href="/facts/Backbone_network/EondGP5q">backbone network</a> with connection points in <a href="/facts/Los_Angeles/pyXpYEX9">Los Angeles</a> and at the Starlight facilities in <a href="/facts/Chicago/yPMokrXH">Chicago</a>. </p><p>In October 2003, NSF awarded $10 million to add four sites to TeraGrid as well as to establish a third network hub, in <a href="/facts/Atlanta/8hPVGb78">Atlanta</a>. These new sites were <a href="/facts/Oak_Ridge_National_Laboratory/Elq84HKF">Oak Ridge National Laboratory</a> (ORNL), <a href="/facts/Purdue_University/dRuTdltL">Purdue University</a>, <a href="/facts/Indiana_University_Bloomington/44gyhspU">Indiana University</a>, and the <a href="/facts/Texas_Advanced_Computing_Center/vm3Vj3Gt">Texas Advanced Computing Center</a> (TACC) at The <a href="/facts/University_of_Texas_at_Austin/6NApstxT">University of Texas at Austin</a>. </p><p>TeraGrid construction was also made possible through corporate partnerships with <a href="/facts/Sun_Microsystems/VJ9EzEFd">Sun Microsystems</a>, <a href="/facts/IBM/QszA7nwd">IBM</a>, <a href="/facts/Intel_Corporation/SMF0gJJX">Intel Corporation</a>, <a href="/facts/Qwest_Communications/evrR7oDl">Qwest Communications</a>, <a href="/facts/Juniper_Networks/Vw0DBzXP">Juniper Networks</a>, <a href="/facts/Myrinet/V93lvD9t">Myricom</a>, <a href="/facts/Hewlett-Packard_Company/abuNSYNu">Hewlett-Packard Company</a>, and <a href="/facts/Oracle_Corporation/0z14FygH">Oracle Corporation</a>. </p><p>TeraGrid construction was completed in October 2004, at which time the TeraGrid facility began full production. </p> <h3>Operation</h3> <p>In August 2005, NSF's newly created office of <a href="/facts/Cyberinfrastructure/QwhPPWq0">cyberinfrastructure</a> extended support for another five years with a $150 million set of awards. It included $48 million for coordination and user support to the Grid Infrastructure Group at the <a href="/facts/University_of_Chicago/z6zRMUmV">University of Chicago</a> led by <a href="/facts/Charlie_Catlett/CyYuyK2f">Charlie Catlett</a>.<a class="footnote-ref" id="fnref:3" href="#fn:3"><sup>3</sup></a> Using high-performance network connections, the TeraGrid featured high-performance computers, data resources and tools, and high-end experimental facilities around the USA. The work supported by the project is sometimes called <a href="/facts/E-Science/bVr55aUe">e-Science</a>. In 2006, the <a href="/facts/University_of_Michigan/6e1Ea5xh">University of Michigan</a>'s School of Information began a study of TeraGrid.<a class="footnote-ref" id="fnref:4" href="#fn:4"><sup>4</sup></a> </p><p>In May 2007, TeraGrid integrated resources included more than 250 teraflops of computing capability and more than 30 petabytes (quadrillions of bytes) of online and archival data storage with rapid access and retrieval over high-performance networks. Researchers could access more than 100 discipline-specific databases. In late 2009, The TeraGrid resources had grown to 2 petaflops of computing capability and more than 60 petabytes storage. In mid 2009, NSF extended the operation of TeraGrid to 2011. </p> <h3>Transition to XSEDE</h3> <p>A follow-on project was approved in May 2011.<a class="footnote-ref" id="fnref:5" href="#fn:5"><sup>5</sup></a> In July 2011, a partnership of 17 institutions announced the <a href="/facts/Extreme_Science_and_Engineering_Discovery_Environment/KUkq0zL1">Extreme Science and Engineering Discovery Environment</a> (XSEDE). NSF announced funding the XSEDE project for five years, at $121 million.<a class="footnote-ref" id="fnref:6" href="#fn:6"><sup>6</sup></a> XSEDE is led by John Towns at the <a href="/facts/University_of_Illinois_at_Urbana%25E2%2580%2593Champaign/gusw7HoS">University of Illinois</a>'s <a href="/facts/National_Center_for_Supercomputing_Applications/vNNK19se">National Center for Supercomputing Applications</a>.<a class="footnote-ref" id="fnref:7" href="#fn:7"><sup>7</sup></a> </p> <h2 id="architecture">Architecture</h2> <p>TeraGrid resources are integrated through a <a href="/facts/Service-oriented_architecture/g8krR0dk">service-oriented architecture</a> in that each resource provides a "service" that is defined in terms of interface and operation. Computational resources run a set of software packages called "Coordinated TeraGrid Software and Services" (CTSS). CTSS provides a familiar user environment on all TeraGrid systems, allowing scientists to more easily port code from one system to another. CTSS also provides integrative functions such as single-signon, remote job submission, workflow support, data movement tools, etc. CTSS includes the Globus Toolkit, Condor, distributed accounting and account management software, verification and validation software, and a set of compilers, programming tools, and <a href="/facts/Environment_variable/WxkrJQpP">environment variables</a>. </p><p>TeraGrid uses a 10 Gigabits per second dedicated fiber-optical backbone network, with hubs in Chicago, Denver, and Los Angeles. All resource provider sites connect to a backbone node at 10 Gigabits per second. Users accessed the facility through national research networks such as the <a href="/facts/Internet2/APYDGudp">Internet2</a> <a href="/facts/Abilene_Network/njAvcCxy">Abilene backbone</a> and <a href="/facts/National_LambdaRail/CEYEVmcT">National LambdaRail</a>. </p> <h2 id="usage">Usage</h2> <p>TeraGrid users primarily came from U.S. universities. There are roughly 4,000 users at over 200 universities. Academic researchers in the United States can obtain exploratory, or <i>development</i> allocations (roughly, in "CPU hours") based on an abstract describing the work to be done. More extensive allocations involve a proposal that is reviewed during a quarterly peer-review process. All allocation proposals are handled through the TeraGrid website. Proposers select a scientific discipline that most closely describes their work, and this enables reporting on the allocation of, and use of, TeraGrid by scientific discipline. As of July 2006 the scientific profile of TeraGrid allocations and usage was: </p> <table><tbody><tr><th>Allocated (%)</th><th>Used (%)</th><th>Scientific Discipline</th></tr><tr><td>19</td><td>23</td><td>Molecular Biosciences</td></tr><tr><td>17</td><td>23</td><td>Physics</td></tr><tr><td>14</td><td>10</td><td>Astronomical Sciences</td></tr><tr><td>12</td><td>21</td><td>Chemistry</td></tr><tr><td>10</td><td>4</td><td>Materials Research</td></tr><tr><td>8</td><td>6</td><td>Chemical, Thermal Systems</td></tr><tr><td>7</td><td>7</td><td>Atmospheric Sciences</td></tr><tr><td>3</td><td>2</td><td>Advanced Scientific Computing</td></tr><tr><td>2</td><td>0.5</td><td>Earth Sciences</td></tr><tr><td>2</td><td>0.5</td><td>Biological and Critical Systems</td></tr><tr><td>1</td><td>0.5</td><td>Ocean Sciences</td></tr><tr><td>1</td><td>0.5</td><td>Cross-Disciplinary Activities</td></tr><tr><td>1</td><td>0.5</td><td>Computer and Computation Research</td></tr><tr><td>0.5</td><td>0.25</td><td>Integrative Biology and Neuroscience</td></tr><tr><td>0.5</td><td>0.25</td><td>Mechanical and Structural Systems</td></tr><tr><td>0.5</td><td>0.25</td><td>Mathematical Sciences</td></tr><tr><td>0.5</td><td>0.25</td><td>Electrical and Communication Systems</td></tr><tr><td>0.5</td><td>0.25</td><td>Design and Manufacturing Systems</td></tr><tr><td>0.5</td><td>0.25</td><td>Environmental Biology</td></tr></tbody></table> <p>Each of these discipline categories correspond to a specific program area of the <a href="/facts/National_Science_Foundation/FlDT8TbW">National Science Foundation</a>. </p><p>Starting in 2006, TeraGrid provided application-specific services to <i>Science Gateway</i> partners, who serve (generally via a web portal) discipline-specific scientific and education communities. Through the Science Gateways program TeraGrid aims to broaden access by at least an order of magnitude in terms of the number of scientists, students, and educators who are able to use TeraGrid. </p> <h2 id="resource-providers">Resource providers</h2> <ul><li><a href="/facts/Argonne_National_Laboratory/EAoR1Rnm">Argonne National Laboratory</a> (ANL) operated by the <a href="/facts/University_of_Chicago/z6zRMUmV">University of Chicago</a> and the <a href="/facts/United_States_Department_of_Energy/PRp6Cz8g">Department of Energy</a></li> <li><a href="/facts/Indiana_University/TELCwWYK">Indiana University</a> - Big Red - <a href="/facts/IBM_BladeCenter/XfHbTe6E">IBM BladeCenter</a> JS21 Cluster<a class="footnote-ref" id="fnref:8" href="#fn:8"><sup>8</sup></a></li> <li>Louisiana Optical Network Initiative (LONI)<a class="footnote-ref" id="fnref:9" href="#fn:9"><sup>9</sup></a></li> <li><a href="/facts/National_Center_for_Atmospheric_Research/7um3PUh3">National Center for Atmospheric Research</a> (NCAR)</li> <li><a href="/facts/National_Center_for_Supercomputing_Applications/vNNK19se">National Center for Supercomputing Applications</a> (NCSA)</li> <li><a href="/facts/National_Institute_for_Computational_Sciences/bxUoPKv6">National Institute for Computational Sciences</a> (NICS) operated by <a href="/facts/University_of_Tennessee/V28AVeaU">University of Tennessee</a> at <a href="/facts/Oak_Ridge_National_Laboratory/Elq84HKF">Oak Ridge National Laboratory</a>.</li> <li><a href="/facts/Oak_Ridge_National_Laboratory/Elq84HKF">Oak Ridge National Laboratory</a> (ORNL)</li> <li><a href="/facts/Pittsburgh_Supercomputing_Center/AqVMSaa6">Pittsburgh Supercomputing Center</a> (PSC) operated by <a href="/facts/University_of_Pittsburgh/U3lxG2L1">University of Pittsburgh</a> and <a href="/facts/Carnegie_Mellon_University/XKd7L2KA">Carnegie Mellon University</a>.</li> <li><a href="/facts/Purdue_University/dRuTdltL">Purdue University</a></li> <li><a href="/facts/San_Diego_Supercomputer_Center/LtSAk3Wh">San Diego Supercomputer Center</a> (SDSC)</li> <li><a href="/facts/Texas_Advanced_Computing_Center/vm3Vj3Gt">Texas Advanced Computing Center</a> (TACC)</li></ul> <h2 id="similar-projects">Similar projects</h2> <ul><li><a href="/facts/Distributed_European_Infrastructure_for_Supercomputing_Applications/Ejv7wgm8">Distributed European Infrastructure for Supercomputing Applications</a> (DEISA), integrating eleven European supercomputing centers</li> <li><a href="/facts/Enabling_Grids_for_E-sciencE/y1NMAbMc">Enabling Grids for E-sciencE</a> (EGEE)</li> <li>National Research Grid Initiative (NAREGEGI) involving several <a href="/facts/Supercomputing_in_Japan/eKRHJVBb">supercomputer centers in Japan</a> from 2003<a class="footnote-ref" id="fnref:10" href="#fn:10"><sup>10</sup></a></li> <li><a href="/facts/Open_Science_Grid/9bY716Wt">Open Science Grid</a> - a distributed computing infrastructure for scientific research</li> <li><a href="/facts/Extreme_Science_and_Engineering_Discovery_Environment/KUkq0zL1">Extreme Science and Engineering Discovery Environment</a> (XSEDE) - the TeraGrid successor</li></ul> <h2 id="external-links">External links</h2> <ul><li><a href="https://web.archive.org/web/20070630140544/http://www.teragrid.org/">TeraGrid website</a></li> <li><a href="https://web.archive.org/web/20090124004142/http://www.teragridforum.org/mediawiki/index.php?title=TeraGrid_Architecture">"TeraGrid Architecture"</a>. <i>TeraGrid Forum</i>. December 20, 2007. Archived from <a href="http://www.teragridforum.org/mediawiki/index.php?title=TeraGrid_Architecture">the original</a> on January 24, 2009. Retrieved September 23, 2011.</li></ul> <h2 id="references">References</h2> <ol> <li id="fn:1"><p>Distributed Terascale Facility (DTF). National Science Foundation. January 2001. Retrieved September 23, 2011. {{cite book}}: |work= ignored (help) <a href="https://www.nsf.gov/pubs/2001/nsf0151/nsf0151.htm" target="_blank">https://www.nsf.gov/pubs/2001/nsf0151/nsf0151.htm</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></p></li> <li id="fn:2"><p>Charlie Catlett (May 21, 2002). The Philosophy of TeraGrid: Building an Open, Extensible, Distributed TeraScale Facility. 2nd IEEE/ACM International Symposium on Cluster Computing and the Grid. p. 8. doi:10.1109/CCGRID.2002.1017101. ISBN 0-7695-1582-7. <a href="0-7695-1582-7" target="_blank">0-7695-1582-7</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></p></li> <li id="fn:3"><p>"$150 Million TeraGrid Award Heralds New Era for Scientific Computing". News release. National Science Foundation. August 17, 2005. Retrieved September 23, 2011. <a href="https://www.nsf.gov/news/news_summ.jsp?cntn_id=104248" target="_blank">https://www.nsf.gov/news/news_summ.jsp?cntn_id=104248</a> <a href="#fnref:3" class="footnote-back-ref">↩</a></p></li> <li id="fn:4"><p>Ann Zimmerman; Thomas A. Finholt (August 2008). Report from the TeraGrid Evaluation Study, Part 1: Project Findings (PDF). National Science Foundation. Retrieved September 23, 2011. <a href="https://www.nsf.gov/od/oci/XD_Program/TeraGrid_evaluation_report_09_02_08.pdf" target="_blank">https://www.nsf.gov/od/oci/XD_Program/TeraGrid_evaluation_report_09_02_08.pdf</a> <a href="#fnref:4" class="footnote-back-ref">↩</a></p></li> <li id="fn:5"><p>National Science Board (May 26, 2011). "Summary Report of the May 10-11, 2011 Meeting" (PDF). Retrieved September 23, 2011. <a href="https://www.nsf.gov/nsb/meetings/2011/0510/summary_report.pdf" target="_blank">https://www.nsf.gov/nsb/meetings/2011/0510/summary_report.pdf</a> <a href="#fnref:5" class="footnote-back-ref">↩</a></p></li> <li id="fn:6"><p>"XSEDE Project Brings Advanced Cyberinfrastructure, Digital Services, and Expertise to Nation's Scientists and Engineers". News release. National Science Foundation. July 25, 2011. Retrieved September 23, 2011. <a href="https://www.nsf.gov/news/news_summ.jsp?cntn_id=121181" target="_blank">https://www.nsf.gov/news/news_summ.jsp?cntn_id=121181</a> <a href="#fnref:6" class="footnote-back-ref">↩</a></p></li> <li id="fn:7"><p>"XSEDE Project Brings Advanced Cyberinfrastructure, Digital Services, and Expertise to Nation's Scientists and Engineers". News release. National Science Foundation. July 25, 2011. Retrieved September 23, 2011. <a href="https://www.nsf.gov/news/news_summ.jsp?cntn_id=121181" target="_blank">https://www.nsf.gov/news/news_summ.jsp?cntn_id=121181</a> <a href="#fnref:7" class="footnote-back-ref">↩</a></p></li> <li id="fn:8"><p>"Big Red at IU". rt.uits.iu.edu. Retrieved 9 Feb 2015. <a href="http://rt.uits.iu.edu/systems/hps/bigred.php" target="_blank">http://rt.uits.iu.edu/systems/hps/bigred.php</a> <a href="#fnref:8" class="footnote-back-ref">↩</a></p></li> <li id="fn:9"><p>"LONI Gets Funding for TeraGrid Research" (PDF). News release. Louisiana State University. September 9, 2009. Archived from the original (PDF) on July 26, 2011. Retrieved September 23, 2011. <a href="https://web.archive.org/web/20110726183908/http://www.loni.org/press/LONI_GetsTGFunding.pdf" target="_blank">https://web.archive.org/web/20110726183908/http://www.loni.org/press/LONI_GetsTGFunding.pdf</a> <a href="#fnref:9" class="footnote-back-ref">↩</a></p></li> <li id="fn:10"><p>S. Matsuokaet; et al. (March 2005). "Japanese Computational Grid Research Project: NAREGI". Proceedings of the IEEE. 93 (3): 522–533. doi:10.1109/JPROC.2004.842748. S2CID 22562197. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:10" class="footnote-back-ref">↩</a></p></li> </ol>