Kepler scientific workflow system

<h2 id="scientific-workflow">Scientific workflow</h2>
<p>A scientific workflow is the process of combining data and processes into a configurable, structured set of steps that implement semi-automated computational solutions to a scientific problem. <a href="/facts/Scientific_workflow_system/HkvT6FTF">Scientific workflow systems</a> often provide graphical user interfaces to combine different technologies along with efficient methods for using them, and thus increase the efficiency of the scientists.
</p>
<h2 id="access-to-scientific-data">Access to scientific data</h2>
<p>Kepler provides direct access to scientific data that has been archived in many of the commonly used data archives.  For example, Kepler provides access to data stored in the Knowledge Network for Biocomplexity (KNB) Metacat server<a class="footnote-ref" id="fnref:5" href="#fn:5"><sup>5</sup></a> and described using <a href="/facts/Ecological_Metadata_Language/CJoAyxYq">Ecological Metadata Language</a>.  Additional data sources that are supported include data accessible using the DiGIR protocol, the <a href="/facts/OPeNDAP/rtrBQh8m">OPeNDAP</a> protocol, GridFTP, <a href="/facts/JDBC/uUVYg2sB">JDBC</a>, <a href="/facts/Storage_Resource_Broker/z4BvPtPj">SRB</a>, and others.
</p>
<h2 id="models-of-computation">Models of Computation</h2>
<p>Kepler differs from many of the other <a href="/facts/Bioinformatics_workflow_management_systems/vbqXe4Hh">bioinformatics workflow management systems</a> in that it separates the structure of the workflow model from its model of computation, such that different models for the computation of the workflow can be bound to a given workflow graph. Kepler inherits several common models of computation from the <a href="/facts/Ptolemy_Project_(computing)/jmwrFBP3">Ptolemy system</a>, including Synchronous Data Flow (SDF), Continuous Time (CT), Process Network (PN), and Dynamic Data Flow (DDF), among others.
</p>
<h2 id="hierarchical-workflows">Hierarchical workflows</h2>
<p>Kepler supports hierarchy in workflows, which allows complex tasks to be composed of simpler components.  This feature allows workflow authors to build re-usable, modular components that can be saved for use across many different workflows.
</p>
<h2 id="workflow-semantics">Workflow semantics</h2>
<p>Kepler provides a model for the semantic annotation of workflow components using terms drawn from an <a href="/facts/Ontology_(information_science)/SQlhl6lp">ontology</a>.  These annotations support many advanced features, including improved search capabilities, automated workflow validation, and improved workflow editing.<a class="footnote-ref" id="fnref:6" href="#fn:6"><sup>6</sup></a>
</p>
<h2 id="sharing-workflows">Sharing workflows</h2>
<p>Kepler components can be shared by exporting the workflow or component into a Kepler Archive (KAR) file, which is an extension of the <a href="/facts/JAR_file/LcIBCWXT">JAR file</a> format from Java.  Once a KAR file is created, it can be emailed to colleagues, shared on web sites, or uploaded to the Kepler Component Repository.  The Component Repository is centralized system for sharing Kepler workflows that is accessible via both a web portal and a <a href="/facts/Web_service/qxrwqB3D">web service</a> interface.  Users can directly search for and utilize components from the repository from within the Kepler workflow composition GUI.
</p>
<h2 id="provenance">Provenance</h2>
<p>Provenance is a critical concept in scientific workflows, since it allows scientists to understand the origin of their results, to repeat their experiments, and to validate the processes that were used to derive data products.<a class="footnote-ref" id="fnref:7" href="#fn:7"><sup>7</sup></a>  In order for a workflow to be reproduced, provenance information must be recorded that indicates where the data originated, how it was altered, and which components and what parameter settings were used. This will allow other scientists to re-conduct the experiment, confirming the results.<a class="footnote-ref" id="fnref:8" href="#fn:8"><sup>8</sup></a>
</p><p>Little support exists in current systems to allow end-users to query provenance information in scientifically meaningful ways, in particular when advanced workflow execution models go beyond simple DAGs (as in process networks).<a class="footnote-ref" id="fnref:9" href="#fn:9"><sup>9</sup></a>
</p>
<h2 id="kepler-history">Kepler history</h2>
<p>The Kepler Project was created in 2002 by members of the Science Environment for Ecological Knowledge (SEEK) project <a class="footnote-ref" id="fnref:10" href="#fn:10"><sup>10</sup></a> and the Scientific Data Management (SDM) project.  The project was founded by researchers at the <a href="/facts/National_Center_for_Ecological_Analysis_and_Synthesis/bCY116fn">National Center for Ecological Analysis and Synthesis</a> (NCEAS) at the <a href="/facts/University_of_California,_Santa_Barbara/G7UvBWlK">University of California, Santa Barbara</a> and the <a href="/facts/San_Diego_Supercomputer_Center/LtSAk3Wh">San Diego Supercomputer Center</a> at the <a href="/facts/University_of_California,_San_Diego/srMClpNv">University of California, San Diego</a>.  Kepler extends Ptolemy II, which is a software system for  modeling, simulation, and design of concurrent, real-time, embedded systems developed at UC Berkeley.  Collaboration on Kepler quickly grew as members of various scientific disciplines realized the benefits of scientific workflows for analysis and modeling and began contributing to the system.  As of 2008, Kepler collaborators come from many science disciplines, including ecology, molecular biology, genetics, physics, chemistry, conservation science, oceanography, hydrology, library science, computer science, and others.
Kepler is a workflow orchestration engine which is used to make workflows for making work much easier, in the form of actor.
</p>
<h2 id="see-also">See also</h2>
<ul>
<li>Free and open-source software portal</li><li>Science portal</li></ul>
<ul><li><a href="/facts/Apache_Taverna/Y2v63Cax">Apache Taverna</a></li>
<li><a href="/facts/Discovery_Net/Yrm0KI2U">Discovery Net</a></li>
<li><a href="/facts/VisTrails/5rLvNxhj">VisTrails</a></li>
<li><a href="/facts/LONI_Pipeline/PSMJDGk5">LONI Pipeline</a></li>
<li><a href="/facts/Bioinformatics_workflow_management_systems/vbqXe4Hh">Bioinformatics workflow management systems</a></li>
<li><a href="/facts/DataONE/UUlHo1ly">DataONE</a> Investigator Toolkit</li></ul>

<h2 id="external-links">External links</h2>
<ul><li><a href="http://kepler-project.org/">Kepler Project website</a></li>
<li><a href="http://library.kepler-project.org/">Kepler Component Repository</a></li>
<li><a href="http://ptolemy.eecs.berkeley.edu/ptolemyII/">Ptolemy II project website</a></li>
<li><a href="http://knb.ecoinformatics.org">Knowledge Network for Biocomplexity (KNB) Data archive</a></li>
<li><a href="http://www.dataone.org/cattools/Scientific%20Workflows">List of software tools</a>[<i>permanent dead link</i>‍] related to workflows on the <a href="/facts/DataONE/UUlHo1ly">DataONE</a> website</li></ul>

<h2 id="references">References</h2>

<ol>
<li id="fn:1"><p>Ludäscher B., Altintas I., Berkley C., Higgins D., Jaeger-Frank E., Jones M., Lee E., Tao J., Zhao Y. 2006. Scientific Workflow Management and the Kepler System. Special Issue: Workflow in Grid Systems.  Concurrency and Computation: Practice & Experience 18(10): 1039-1065. <a href="#fnref:1" class="footnote-back-ref">↩</a></p></li>
<li id="fn:2"><p>Altintas I, Berkley C, Jaeger E, Jones M, Ludäscher B, Mock S. 2004. Kepler: An Extensible System for Design and Execution of Scientific Workflows. Proceedings of the Future of Grid Data Environments, Global Grid Forum 10. <a href="#fnref:2" class="footnote-back-ref">↩</a></p></li>
<li id="fn:3"><p>Michener, William K., James H. Beach, Matthew B. Jones, Bertram Ludaescher, Deana D. Pennington, Ricardo S. Pereira, Arcot Rajasekar, and Mark Schildhauer. 2007. "A Knowledge Environment for the Biodiversity and Ecological Sciences", Journal of Intelligent Information Systems, 29(1): 111-126. doi:10.1007/s10844-006-0034-8 <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:3" class="footnote-back-ref">↩</a></p></li>
<li id="fn:4"><p>Taylor, I.J.; Deelman, E.; Gannon, D.B.; Shields, M. (Eds.), “Workflows for e-Science: Scientific Workflows for Grids”,  530 p., Springer. ISBN 978-1-84628-519-6. <a href="/wiki/Ewa_Deelman" target="_blank">/wiki/Ewa_Deelman</a> <a href="#fnref:4" class="footnote-back-ref">↩</a></p></li>
<li id="fn:5"><p>Jones, Matthew B., C. Berkley, J. Bojilova, M. Schildhauer. 2001.  Managing Scientific Metadata.  IEEE Internet Computing 5 (5): 59-68. <a href="#fnref:5" class="footnote-back-ref">↩</a></p></li>
<li id="fn:6"><p>Berkley, Chad, Shawn Bowers, Matthew B. Jones, Bertram Ludaescher, Mark Schildhauer, Jing Tao. 2005. Incorporating Semantics in Scientific Workflow Authoring. 17th International Conference on Scientific and Statistical Database Management.  IEEE Computer Society. <a href="#fnref:6" class="footnote-back-ref">↩</a></p></li>
<li id="fn:7"><p>"WebHome < Challenge < TWiki". Archived from the original on 2008-07-06. Retrieved 2009-04-06. <a href="https://web.archive.org/web/20080706052044/http://twiki.ipaw.info/bin/view/Challenge/WebHome" target="_blank">https://web.archive.org/web/20080706052044/http://twiki.ipaw.info/bin/view/Challenge/WebHome</a> <a href="#fnref:7" class="footnote-back-ref">↩</a></p></li>
<li id="fn:8"><p>Barker, A.; van Hemert, J (2008). "Scientific Workflow: A Survey and Research Directions". Parallel Processing and Applied Mathematics. PPAM 2007. Lecture Notes in Computer Science. 4967. doi:10.1007/978-3-540-68111-3_78. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:8" class="footnote-back-ref">↩</a></p></li>
<li id="fn:9"><p>Shawn Bowers, Timothy McPhillips, Bertram Ludascher, Shirley Cohen, Susan B. Davidson 2006. A Model for User-Oriented Data Provenance in Pipelined Scientific Workflows. <a href="http://repository.upenn.edu/cgi/viewcontent.cgi?article=1307&context=cis_papers" target="_blank">http://repository.upenn.edu/cgi/viewcontent.cgi?article=1307&context=cis_papers</a> <a href="#fnref:9" class="footnote-back-ref">↩</a></p></li>
<li id="fn:10"><p>Michener, William K., James H. Beach, Matthew B. Jones, Bertram Ludaescher, Deana D. Pennington, Ricardo S. Pereira, Arcot Rajasekar, and Mark Schildhauer. 2007. "A Knowledge Environment for the Biodiversity and Ecological Sciences", Journal of Intelligent Information Systems, 29(1): 111-126. doi:10.1007/s10844-006-0034-8 <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:10" class="footnote-back-ref">↩</a></p></li>
</ol>

Kepler scientific workflow system open-in-new

Kepler scientific workflow system