Menu
Home
People
Places
Arts
History
Plants & Animals
Science
Life & Culture
Technology
Reference.org
Data farming
open-in-new
<h2 id="history">History</h2> <p>The science of <a href="/facts/Design_of_Experiments/VMrNftbr">Design of Experiments</a> (DOE) has been around for over a century, pioneered by <a href="/facts/R.A._Fisher/I2FVIpy9">R.A. Fisher</a> for agricultural studies. Many of the classic experiment designs can be used in simulation studies. However, computational experiments have far fewer restrictions than do real-world experiments, in terms of costs, number of factors, time required, ability to replicate, ability to automate, etc. Consequently, a framework specifically oriented toward large-scale simulation experiments is warranted. </p><p>People have been conducting computational experiments for as long as computers have been around. The term “data farming” is more recent, coined in 1998<a class="footnote-ref" id="fnref:4" href="#fn:4"><sup>4</sup></a> in conjunction with the Marine Corp's Project Albert,<a class="footnote-ref" id="fnref:5" href="#fn:5"><sup>5</sup></a> in which small agent-based distillation models (a type of stochastic simulation) were created to capture specific military challenges. These models were run thousands or millions of times at the Maui High Performance Computer Center<a class="footnote-ref" id="fnref:6" href="#fn:6"><sup>6</sup></a> and other facilities. Project Albert analysts would work with the military subject matter experts to refine the models and interpret the results. </p><p>Initially, the use of brute-force <a href="/facts/Factorial_experiment/lPlCA847">full factorial</a> (gridded) designs meant that the simulations needed to run very quickly and the studies required <a href="/facts/High-performance_computing/HT9NwGAN">high-performance computing</a>. Even so, only a small number of factors (at a limited number of levels) could be investigated, due to the <a href="/facts/Curse_of_dimensionality/dk7SnY6L">curse of dimensionality</a>. </p><p>The SEED Center for Data Farming<a class="footnote-ref" id="fnref:7" href="#fn:7"><sup>7</sup></a> at the <a href="/facts/Naval_Postgraduate_School/K7CDD8XD">Naval Postgraduate School</a><a class="footnote-ref" id="fnref:8" href="#fn:8"><sup>8</sup></a> also worked closely with Project Albert in model generation, output analysis, and the creation of new <a href="/facts/Design_of_experiments/VMrNftbr">experimental designs</a> to better leverage the computing capabilities at Maui and other facilities. Recent breakthroughs in designs specifically developed for data farming can be found in<a class="footnote-ref" id="fnref:9" href="#fn:9"><sup>9</sup></a><a class="footnote-ref" id="fnref:10" href="#fn:10"><sup>10</sup></a> among others. </p> <h2 id="workshops">Workshops</h2> <p>A series of international data farming workshops have been held since 1998 by the SEED Center for Data Farming.<a class="footnote-ref" id="fnref:11" href="#fn:11"><sup>11</sup></a> International Data Farming Workshop 1 occurred in 1991, and since then 16 more workshops have taken place. The workshops have seen a diverse array of representation from participating countries, such as Canada, Singapore, Mexico, Turkey, and the United States.<a class="footnote-ref" id="fnref:12" href="#fn:12"><sup>12</sup></a> </p><p>The International Data Farming Workshops operate through collaboration between various teams of experts. The most recent workshop held in 2008 saw over 100 teams participating. The teams of data farmers are assigned a specific area of study, such as <a href="/facts/Robotics/wkmvodJz">robotics</a>, <a href="/facts/Homeland_security/Uz6ITRXM">homeland security</a>, and <a href="/facts/Disaster_relief/YjGIyW91">disaster relief</a>. Different forms of data farming are experimented with and utilized by each group, such as the <a href="/facts/Pythagoras_ABM/nrDX0LSl">Pythagoras ABM</a>, the Logistics Battle Command model, and the agent-based sensor effector model (ABSEM).<a class="footnote-ref" id="fnref:13" href="#fn:13"><sup>13</sup></a> </p> <h2 id="external-links">External links</h2> <ul><li><a href="http://harvest.nps.edu/">SEED Center for Data Farming</a> website, with links to numerous papers, applications, designs, and software.</li> <li>An article on the 27th Data Farming Workshop in Finland in <a href="http://www.defensemedianetwork.com/stories/international-data-farming-workshop-under-way-in-finland/">Defense Media Network from January 2014</a></li> <li>An article on data farming in <a href="http://webarchive.loc.gov/all/20130109111949/http://www.defensenews.com/article/20130103/TSJ01/301030005/Technical%2DBriefing%2DData%2DFarming">Defense News from January 2013</a></li> <li>An article summarizing data farming in the <a href="http://www.afcea.org/signal/articles/templates/SIGNAL_Article_Template.asp?articleid=975&zoneid=158">June 2005 issue of SIGNAL</a></li> <li><a href="http://www.informs-sim.org/wsc04papers/100.pdf">MITRE Corporation research paper on data farming</a></li></ul> <h2 id="references">References</h2> <ol> <li id="fn:1"><p>Lucas, T. W.; Kelton, W. D.; Sanchez, P. J.; Sanchez, S. M.; Anderson, B. L. (2015). "Changing the Paradigm: Simulation, Now a Method of First Resort". Naval Research Logistics. 62 (4): 293–305. doi:10.1002/nav.21628. hdl:10945/57859. S2CID 60846350. <a href="/wiki/Susan_M._Sanchez" target="_blank">/wiki/Susan_M._Sanchez</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></p></li> <li id="fn:2"><p>https://www.cso.nato.int/Pubs/rdp.asp?RDP=STO-TR-MSG-088 [bare URL] <a href="https://www.cso.nato.int/Pubs/rdp.asp?RDP=STO-TR-MSG-088" target="_blank">https://www.cso.nato.int/Pubs/rdp.asp?RDP=STO-TR-MSG-088</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></p></li> <li id="fn:3"><p>"Data Farming". Archived from the original on 2015-08-29. Retrieved 2014-04-22. <a href="https://web.archive.org/web/20150829212237/http://www.datafarming.org/Data_Farming/Welcome.html" target="_blank">https://web.archive.org/web/20150829212237/http://www.datafarming.org/Data_Farming/Welcome.html</a> <a href="#fnref:3" class="footnote-back-ref">↩</a></p></li> <li id="fn:4"><p>Brandstein, A.; Horne, G. (1998). "Data Farming: A Meta-Technique for Research in the 21st Century". Maneuver Warfare Science. Quantico, VA: Marine Corps Combat Development Command. <a href="#fnref:4" class="footnote-back-ref">↩</a></p></li> <li id="fn:5"><p>http://projectalbert.org [bare URL] <a href="http://projectalbert.org" target="_blank">http://projectalbert.org</a> <a href="#fnref:5" class="footnote-back-ref">↩</a></p></li> <li id="fn:6"><p>https://www.mhpcc.hpc.mil/ [bare URL] <a href="https://www.mhpcc.hpc.mil/" target="_blank">https://www.mhpcc.hpc.mil/</a> <a href="#fnref:6" class="footnote-back-ref">↩</a></p></li> <li id="fn:7"><p>http://harvest.nps.edu [bare URL] <a href="http://harvest.nps.edu" target="_blank">http://harvest.nps.edu</a> <a href="#fnref:7" class="footnote-back-ref">↩</a></p></li> <li id="fn:8"><p>http://www.nps.edu/ [bare URL] <a href="http://www.nps.edu/" target="_blank">http://www.nps.edu/</a> <a href="#fnref:8" class="footnote-back-ref">↩</a></p></li> <li id="fn:9"><p>Kleijnen, J. P. C.; Sanchez, S. M.; Lucas, T. W.; Cioppa, T. M. (2005). "A User's Guide to the Brave New World of Designing Simulation Experiments". INFORMS Journal on Computing. 17 (3): 263–289. doi:10.1287/ijoc.1050.0136. <a href="/wiki/Susan_M._Sanchez" target="_blank">/wiki/Susan_M._Sanchez</a> <a href="#fnref:9" class="footnote-back-ref">↩</a></p></li> <li id="fn:10"><p>Sanchez, S. M.; Sanchez, P.; Wan, H. (2021). "Work Smarter, Not Harder: A Tutorial on Designing and ConductingSimulation Experiments" (PDF). 2021 Winter Simulation Conference (WSC). Piscataway, NJ: Institute of Electrical and Electronics Engineers, Inc. pp. 1–15. doi:10.1109/WSC52266.2021.9715422. hdl:10945/44883. ISBN 9780903440660. S2CID 247059747. <a href="9780903440660" target="_blank">9780903440660</a> <a href="#fnref:10" class="footnote-back-ref">↩</a></p></li> <li id="fn:11"><p>http://harvest.nps.edu [bare URL] <a href="http://harvest.nps.edu" target="_blank">http://harvest.nps.edu</a> <a href="#fnref:11" class="footnote-back-ref">↩</a></p></li> <li id="fn:12"><p>Horne, G., & Schwierz, K. (2008). Data farming around the world overview. Paper presented at the 1442-1447. doi:10.1109/WSC.2008.4736222 <a href="#fnref:12" class="footnote-back-ref">↩</a></p></li> <li id="fn:13"><p>Horne, G., & Schwierz, K. (2008). Data farming around the world overview. Paper presented at the 1442-1447. doi:10.1109/WSC.2008.4736222 <a href="#fnref:13" class="footnote-back-ref">↩</a></p></li> </ol>