Menu
Home
People
Places
Arts
History
Plants & Animals
Science
Life & Culture
Technology
Reference.org
Aggregative Contingent Estimation Program
open-in-new
<h2 id="history">History</h2> <p>The broad program announcement for ACE was published on June 30, 2010.<a class="footnote-ref" id="fnref:4" href="#fn:4"><sup>4</sup></a> ACE funded the Aggregative Contingent Estimation System (ACES) website and interface on July 15, 2011.<a class="footnote-ref" id="fnref:5" href="#fn:5"><sup>5</sup></a> They funded <a href="/facts/The_Good_Judgment_Project/xSPDsmqx">The Good Judgment Project</a> some time around July 2011.<a class="footnote-ref" id="fnref:6" href="#fn:6"><sup>6</sup></a> ACE has been covered in <i><a href="/facts/The_Washington_Post/dRvzvqJc">The Washington Post</a>''</i><a class="footnote-ref" id="fnref:7" href="#fn:7"><sup>7</sup></a> and <i><a href="/facts/Wired_Magazine/RQb6gcyf">Wired Magazine</a></i>.<a class="footnote-ref" id="fnref:8" href="#fn:8"><sup>8</sup></a> The program was concluded by late 2015.<a class="footnote-ref" id="fnref:9" href="#fn:9"><sup>9</sup></a> The program manager was future IARPA director <a href="/facts/Jason_Gaverick_Matheny/wnJ4qg4R">Jason Gaverick Matheny</a>.<a class="footnote-ref" id="fnref:10" href="#fn:10"><sup>10</sup></a> </p> <h2 id="goals-and-methods">Goals and methods</h2> <p>The official website says that the goals of ACE are "to dramatically enhance the accuracy, precision, and timeliness of intelligence forecasts for a broad range of event types, through the development of advanced techniques that elicit, weight, and combine the judgments of many intelligence analysts."<a class="footnote-ref" id="fnref:11" href="#fn:11"><sup>11</sup></a> The website claims that ACE seeks technical innovations in the following areas:<a class="footnote-ref" id="fnref:12" href="#fn:12"><sup>12</sup></a> </p> <ul><li>efficient elicitation of probabilistic judgments, including conditional probabilities for contingent events</li> <li>mathematical aggregation of judgments by many individuals, based on factors that may include: past performance, expertise, cognitive style, metaknowledge, and other attributes predictive of accuracy</li> <li>effective representation of aggregated probabilistic forecasts and their distributions.</li></ul> <p>There is a fair amount of research funded by grants made by the IARPA ACE program.<a class="footnote-ref" id="fnref:13" href="#fn:13"><sup>13</sup></a> </p> <h2 id="partners">Partners</h2> <p>The ACE has collaborated with partners who compete in its forecasting tournaments. Their most notable partner is <a href="/facts/The_Good_Judgment_Project/xSPDsmqx">The Good Judgment Project</a> from <a href="/facts/Philip_E._Tetlock/EXKMF8Pw">Philip E. Tetlock</a> et al.<a class="footnote-ref" id="fnref:14" href="#fn:14"><sup>14</sup></a> (winner of a 2013 ACE tournament)<a class="footnote-ref" id="fnref:15" href="#fn:15"><sup>15</sup></a> ACE also partnered with the ARA to create the Aggregative Contingent Estimation System (ACES).<a class="footnote-ref" id="fnref:16" href="#fn:16"><sup>16</sup></a> </p><p>Data from ACE is fed into another program, called Forecasting Science and Technology (ForeST), which partners with <a href="/facts/SciCast/SA8xT0lF">SciCast</a> from <a href="/facts/George_Mason_University/rWKeeMrh">George Mason University</a>.<a class="footnote-ref" id="fnref:17" href="#fn:17"><sup>17</sup></a> </p> <h2 id="external-links">External links</h2> <ul><li><a href="http://www.iarpa.gov/index.php/research-programs/ace">Official website</a></li></ul> <h2 id="references">References</h2> <ol> <li id="fn:1"><p>Matheny, Jason; Rieber, Steve. "Aggregative Contingent Estimation (ACE)". Intelligence Advanced Research Projects Activity. Retrieved May 6, 2014. <a href="http://www.iarpa.gov/index.php/research-programs/ace" target="_blank">http://www.iarpa.gov/index.php/research-programs/ace</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></p></li> <li id="fn:2"><p>"Aggregative Contingent Estimation" (PDF). Office of the Director of National Intelligence, United States. Retrieved May 6, 2014. <a href="https://obamawhitehouse.archives.gov/sites/default/files/microsites/ostp/iarpa_big_data.pdf" target="_blank">https://obamawhitehouse.archives.gov/sites/default/files/microsites/ostp/iarpa_big_data.pdf</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></p></li> <li id="fn:3"><p>Harbert, Tam (2015-10-19). "IARPA's New Director Wants You to Surprise Him". IEEE Spectrum. Retrieved 2016-03-31. <a href="https://spectrum.ieee.org/iarpas-new-director-wants-you-to-surprise-him" target="_blank">https://spectrum.ieee.org/iarpas-new-director-wants-you-to-surprise-him</a> <a href="#fnref:3" class="footnote-back-ref">↩</a></p></li> <li id="fn:4"><p>"Aggregative Contingent Estimation System". Federal Business Opportunities. June 30, 2010. Retrieved May 6, 2014. <a href="https://www.fbo.gov/index?s=opportunity&mode=form&tab=core&id=54f9f4b696ffeedf52cffc433a2fc878" target="_blank">https://www.fbo.gov/index?s=opportunity&mode=form&tab=core&id=54f9f4b696ffeedf52cffc433a2fc878</a> <a href="#fnref:4" class="footnote-back-ref">↩</a></p></li> <li id="fn:5"><p>Hickey, Kathleen (July 15, 2011). "Intell site tests crowdsourcing's ability to predict future". GCN. Retrieved May 6, 2014. <a href="http://gcn.com/articles/2011/07/15/intell-crowdsourcing-forecasting-ace.aspx" target="_blank">http://gcn.com/articles/2011/07/15/intell-crowdsourcing-forecasting-ace.aspx</a> <a href="#fnref:5" class="footnote-back-ref">↩</a></p></li> <li id="fn:6"><p>"The idea behind the Good Judgment Project". The Good Judgment Project. July 27, 2011. Archived from the original on May 6, 2014. Retrieved May 5, 2014. <a href="https://web.archive.org/web/20140506202243/http://goodjudgmentproject.com/blog/?p=4" target="_blank">https://web.archive.org/web/20140506202243/http://goodjudgmentproject.com/blog/?p=4</a> <a href="#fnref:6" class="footnote-back-ref">↩</a></p></li> <li id="fn:7"><p>Horowitz, Michael (November 26, 2013). "Good judgment in forecasting international affairs (and an invitation for season 3)". The Washington Post. Retrieved May 5, 2014. <a href="https://www.washingtonpost.com/blogs/monkey-cage/wp/2013/11/26/good-judgment-in-forecasting-international-affairs-and-an-invitation-for-season-3/" target="_blank">https://www.washingtonpost.com/blogs/monkey-cage/wp/2013/11/26/good-judgment-in-forecasting-international-affairs-and-an-invitation-for-season-3/</a> <a href="#fnref:7" class="footnote-back-ref">↩</a></p></li> <li id="fn:8"><p>Drummond, Katie (April 22, 2010). "Can Algorithms Find the Best Intelligence Analysts?". Wired Magazine. Retrieved May 6, 2014. <a href="https://www.wired.com/2010/04/can-algorithms-find-the-best-intelligence-analysts/" target="_blank">https://www.wired.com/2010/04/can-algorithms-find-the-best-intelligence-analysts/</a> <a href="#fnref:8" class="footnote-back-ref">↩</a></p></li> <li id="fn:9"><p>Corrin, Amber (2015-09-23). "IARPA's high-stakes intelligence experiment". C4ISR & Networks. Archived from the original on 2017-06-21. Retrieved 2016-03-31. <a href="https://web.archive.org/web/20170621170049/http://www.c4isrnet.com/story/military-tech/it/2015/09/23/iarpa-anticipating-surprise/72632204/" target="_blank">https://web.archive.org/web/20170621170049/http://www.c4isrnet.com/story/military-tech/it/2015/09/23/iarpa-anticipating-surprise/72632204/</a> <a href="#fnref:9" class="footnote-back-ref">↩</a></p></li> <li id="fn:10"><p>Marc Prensky (7 August 2012). Brain Gain: Technology and the Quest for Digital Wisdom. St. Martin's Press. p. 260. ISBN 978-1-137-09317-2. The ACE program manager is Jason Matheny <a href="978-1-137-09317-2" target="_blank">978-1-137-09317-2</a> <a href="#fnref:10" class="footnote-back-ref">↩</a></p></li> <li id="fn:11"><p>Matheny, Jason; Rieber, Steve. "Aggregative Contingent Estimation (ACE)". Intelligence Advanced Research Projects Activity. Retrieved May 6, 2014. <a href="http://www.iarpa.gov/index.php/research-programs/ace" target="_blank">http://www.iarpa.gov/index.php/research-programs/ace</a> <a href="#fnref:11" class="footnote-back-ref">↩</a></p></li> <li id="fn:12"><p>Matheny, Jason; Rieber, Steve. "Aggregative Contingent Estimation (ACE)". Intelligence Advanced Research Projects Activity. Retrieved May 6, 2014. <a href="http://www.iarpa.gov/index.php/research-programs/ace" target="_blank">http://www.iarpa.gov/index.php/research-programs/ace</a> <a href="#fnref:12" class="footnote-back-ref">↩</a></p></li> <li id="fn:13"><p>"Google Scholar listing of research funded by IARPA ACE". Retrieved May 6, 2014. <a href="https://scholar.google.com/scholar?hl=en&q=%22D11PC20058+OR+D11PC20059+OR+D11PC20060+OR+D11PC20061+OR+D11PC20062%22&btnG=&as_sdt=1%2C47&as_sdtp=" target="_blank">https://scholar.google.com/scholar?hl=en&q=%22D11PC20058+OR+D11PC20059+OR+D11PC20060+OR+D11PC20061+OR+D11PC20062%22&btnG=&as_sdt=1%2C47&as_sdtp=</a> <a href="#fnref:13" class="footnote-back-ref">↩</a></p></li> <li id="fn:14"><p>"The Project". The Good Judgment Project. Archived from the original on May 6, 2014. Retrieved May 5, 2014. <a href="https://web.archive.org/web/20140506201121/http://www.goodjudgmentproject.com/about_project.html" target="_blank">https://web.archive.org/web/20140506201121/http://www.goodjudgmentproject.com/about_project.html</a> <a href="#fnref:14" class="footnote-back-ref">↩</a></p></li> <li id="fn:15"><p>Horowitz, Michael (November 26, 2013). "Good judgment in forecasting international affairs (and an invitation for season 3)". The Washington Post. Retrieved May 5, 2014. <a href="https://www.washingtonpost.com/blogs/monkey-cage/wp/2013/11/26/good-judgment-in-forecasting-international-affairs-and-an-invitation-for-season-3/" target="_blank">https://www.washingtonpost.com/blogs/monkey-cage/wp/2013/11/26/good-judgment-in-forecasting-international-affairs-and-an-invitation-for-season-3/</a> <a href="#fnref:15" class="footnote-back-ref">↩</a></p></li> <li id="fn:16"><p>Hickey, Kathleen (July 15, 2011). "Intell site tests crowdsourcing's ability to predict future". GCN. Retrieved May 6, 2014. <a href="http://gcn.com/articles/2011/07/15/intell-crowdsourcing-forecasting-ace.aspx" target="_blank">http://gcn.com/articles/2011/07/15/intell-crowdsourcing-forecasting-ace.aspx</a> <a href="#fnref:16" class="footnote-back-ref">↩</a></p></li> <li id="fn:17"><p>Matheny, Jason. "Forecasting Science & Technology (ForeST)". Intelligence Advanced Research Projects Activity. Retrieved May 6, 2014. <a href="http://www.iarpa.gov/index.php/research-programs/forest" target="_blank">http://www.iarpa.gov/index.php/research-programs/forest</a> <a href="#fnref:17" class="footnote-back-ref">↩</a></p></li> </ol>