Menu
Home
People
Places
Arts
History
Plants & Animals
Science
Life & Culture
Technology
Reference.org
Exploratory causal analysis
open-in-new
<h2 id="motivation">Motivation</h2> <p>Data analysis is primarily concerned with causal questions.<a class="footnote-ref" id="fnref:7" href="#fn:7"><sup>7</sup></a><a class="footnote-ref" id="fnref:8" href="#fn:8"><sup>8</sup></a><a class="footnote-ref" id="fnref:9" href="#fn:9"><sup>9</sup></a><a class="footnote-ref" id="fnref:10" href="#fn:10"><sup>10</sup></a><a class="footnote-ref" id="fnref:11" href="#fn:11"><sup>11</sup></a> For example, did the fertilizer cause the crops to grow?<a class="footnote-ref" id="fnref:12" href="#fn:12"><sup>12</sup></a> Or, can a given sickness be prevented?<a class="footnote-ref" id="fnref:13" href="#fn:13"><sup>13</sup></a> Or, why is my friend depressed?<a class="footnote-ref" id="fnref:14" href="#fn:14"><sup>14</sup></a> The <a href="/facts/Rubin_causal_model/m6h998gw">potential outcomes</a> and <a href="/facts/Regression_analysis/n6z5Tf7K">regression analysis</a> techniques handle such queries when data is collected using designed experiments. Data collected in <a href="/facts/Observational_study/zEhxI4eh">observational</a> studies require different techniques for causal inference (because, for example, of issues such as <a href="/facts/Confounding/G4TVx1dt">confounding</a>).<a class="footnote-ref" id="fnref:15" href="#fn:15"><sup>15</sup></a> Causal inference techniques used with experimental data require additional assumptions to produce reasonable inferences with observation data.<a class="footnote-ref" id="fnref:16" href="#fn:16"><sup>16</sup></a> The difficulty of causal inference under such circumstances is often summed up as "<a href="/facts/Correlation_does_not_imply_causation/IeWeoMB9">correlation does not imply causation</a>". </p> <h2 id="overview">Overview</h2> <p>ECA postulates that there exist <a href="/facts/Data_analysis/CNtARxLW">data analysis procedures</a> performed on specific subsets of <a href="/facts/Dependent_and_independent_variables/dINfzIF2">variables</a> within a larger set whose outputs might be indicative of causality between those variables.<a class="footnote-ref" id="fnref:17" href="#fn:17"><sup>17</sup></a> For example, if we assume every relevant <a href="/facts/Covariate/dINfzIF2">covariate</a> in the data is observed, then <a href="/facts/Propensity_score_matching/ajVY6GqP">propensity score matching</a> can be used to find the causal effect between two observational variables.<a class="footnote-ref" id="fnref:18" href="#fn:18"><sup>18</sup></a> <a href="/facts/Granger_causality/LAEvgA1A">Granger causality</a> can also be used to find the causality between two observational variables under different, but similarly strict, assumptions.<a class="footnote-ref" id="fnref:19" href="#fn:19"><sup>19</sup></a> </p><p>The two broad approaches to developing such procedures are using <i>operational definitions of causality</i><a class="footnote-ref" id="fnref:20" href="#fn:20"><sup>20</sup></a> or <i>verification by "truth"</i> (i.e., explicitly ignoring the problem of defining <a href="/facts/Causality/Swh496X7">causality</a> and showing that a given algorithm implies a causal relationship in scenarios when causal relationships are known to exist, e.g., using <a href="/facts/Synthetic_data/MVSNrFPI">synthetic data</a><a class="footnote-ref" id="fnref:21" href="#fn:21"><sup>21</sup></a>). </p> <h3>Operational definitions of causality</h3> <p><a href="/facts/Clive_Granger/GtKQHHE9">Clive Granger</a> created the first operational definition of causality in 1969.<a class="footnote-ref" id="fnref:22" href="#fn:22"><sup>22</sup></a> Granger made the definition of <a href="/facts/Probabilistic_causation/JP8HFvyh">probabilistic causality</a> proposed by <a href="/facts/Norbert_Wiener/cVMRAHCU">Norbert Wiener</a> operational as a comparison of variances.<a class="footnote-ref" id="fnref:23" href="#fn:23"><sup>23</sup></a> </p><p>Some authors prefer using ECA techniques developed using operational definitions of causality because they believe it may help in the search for causal mechanisms.<a class="footnote-ref" id="fnref:24" href="#fn:24"><sup>24</sup></a><a class="footnote-ref" id="fnref:25" href="#fn:25"><sup>25</sup></a> </p> <h3>Verification by "truth"</h3> <p>Peter Spirtes, <a href="/facts/Clark_Glymour/hHwY8xY8">Clark Glymour</a>, and Richard Scheines introduced the idea of explicitly not providing a definition of causality.<a class="footnote-ref" id="fnref:26" href="#fn:26"><sup>26</sup></a> Spirtes and Glymour introduced the PC algorithm for causal discovery in 1990.<a class="footnote-ref" id="fnref:27" href="#fn:27"><sup>27</sup></a> Many recent causal discovery algorithms follow the Spirtes-Glymour approach to verification.<a class="footnote-ref" id="fnref:28" href="#fn:28"><sup>28</sup></a> </p> <h2 id="techniques">Techniques</h2> <p>There are many surveys of causal discovery techniques.<a class="footnote-ref" id="fnref:29" href="#fn:29"><sup>29</sup></a><a class="footnote-ref" id="fnref:30" href="#fn:30"><sup>30</sup></a><a class="footnote-ref" id="fnref:31" href="#fn:31"><sup>31</sup></a><a class="footnote-ref" id="fnref:32" href="#fn:32"><sup>32</sup></a><a class="footnote-ref" id="fnref:33" href="#fn:33"><sup>33</sup></a><a class="footnote-ref" id="fnref:34" href="#fn:34"><sup>34</sup></a> This section lists the well-known techniques. </p> <h3>Bivariate (or "pairwise")</h3> <ul><li><a href="/facts/Granger_causality/LAEvgA1A">Granger causality</a> (there is also the Scholarpedia entry <a href="http://www.scholarpedia.org/article/Granger_causality">[1]</a>)</li> <li><a href="/facts/Transfer_entropy/bn3jayn1">transfer entropy</a></li> <li><a href="/facts/Convergent_cross_mapping/usdIuYok">convergent cross mapping</a></li></ul> <h3>Multivariate</h3> <ul><li>causation entropy<a class="footnote-ref" id="fnref:35" href="#fn:35"><sup>35</sup></a></li> <li>PC algorithm<a class="footnote-ref" id="fnref:36" href="#fn:36"><sup>36</sup></a><a class="footnote-ref" id="fnref:37" href="#fn:37"><sup>37</sup></a></li> <li>FCI algorithm<a class="footnote-ref" id="fnref:38" href="#fn:38"><sup>38</sup></a><a class="footnote-ref" id="fnref:39" href="#fn:39"><sup>39</sup></a></li> <li>LiNGAM<a class="footnote-ref" id="fnref:40" href="#fn:40"><sup>40</sup></a> <a href="https://sites.google.com/site/sshimizu06/lingam/">[2]</a></li></ul> <p>Many of these techniques are discussed in the tutorials provided by the Center for Causal Discovery (CCD) <a href="https://www.ccd.pitt.edu/video-tutorials/">[3]</a>. </p> <h2 id="use-case-examples">Use-case examples</h2> <h3>Social science</h3> <p>The PC algorithm has been applied to several different social science data sets.<a class="footnote-ref" id="fnref:41" href="#fn:41"><sup>41</sup></a> </p> <h3>Medicine</h3> <p>The PC algorithm has been applied to medical data.<a class="footnote-ref" id="fnref:42" href="#fn:42"><sup>42</sup></a> Granger causality has been applied to <a href="/facts/Functional_magnetic_resonance_imaging/GIayRTAL">fMRI</a> data.<a class="footnote-ref" id="fnref:43" href="#fn:43"><sup>43</sup></a> CCD tested their tools using biomedical data <a href="https://www.ccd.pitt.edu/biomedical-science/">[4]</a>. </p> <h3>Physics</h3> <p>ECA is used in physics to understand the physical causal mechanisms of the system, e.g., in geophysics using the PC-stable algorithm (a variant of the original PC algorithm)<a class="footnote-ref" id="fnref:44" href="#fn:44"><sup>44</sup></a> and in dynamical systems using pairwise asymmetric inference (a variant of convergent cross mapping).<a class="footnote-ref" id="fnref:45" href="#fn:45"><sup>45</sup></a> </p> <h2 id="criticism">Criticism</h2> <p>There is debate over whether or not the relationships between data found using causal discovery are actually causal.<a class="footnote-ref" id="fnref:46" href="#fn:46"><sup>46</sup></a><a class="footnote-ref" id="fnref:47" href="#fn:47"><sup>47</sup></a> <a href="/facts/Judea_Pearl/J2JUSAfe">Judea Pearl</a> has emphasized that causal inference requires a causal model developed by "intelligence" through an iterative process of testing assumptions and fitting data.<a class="footnote-ref" id="fnref:48" href="#fn:48"><sup>48</sup></a> </p><p>Response to the criticism points out that assumptions used for developing ECA techniques may not hold for a given data set<a class="footnote-ref" id="fnref:49" href="#fn:49"><sup>49</sup></a><a class="footnote-ref" id="fnref:50" href="#fn:50"><sup>50</sup></a><a class="footnote-ref" id="fnref:51" href="#fn:51"><sup>51</sup></a><a class="footnote-ref" id="fnref:52" href="#fn:52"><sup>52</sup></a><a class="footnote-ref" id="fnref:53" href="#fn:53"><sup>53</sup></a> and that any causal relationships discovered during ECA are contingent on these assumptions holding true<a class="footnote-ref" id="fnref:54" href="#fn:54"><sup>54</sup></a><a class="footnote-ref" id="fnref:55" href="#fn:55"><sup>55</sup></a> </p> <h2 id="software-packages">Software Packages</h2> <h3>Comprehensive toolkits</h3> <ul><li><a href="https://cmu-phil.github.io/tetrad/manual/">Tetrad</a> is an open source GUI-based Java program that provides a collection of causal discovery algorithms.<a class="footnote-ref" id="fnref:56" href="#fn:56"><sup>56</sup></a> The algorithm library used by Tetrad is also available as a <a href="/facts/Command-line_interface/R9JpGmXQ">command-line</a> tool, <a href="/facts/Python_(programming_language)/YbuGqofa">Python</a> <a href="/facts/Application_programming_interface/GMlN4vUr">API</a>, and <a href="/facts/R_(programming_language)/LSrkr8K8">R</a> wrapper.<a class="footnote-ref" id="fnref:57" href="#fn:57"><sup>57</sup></a></li> <li><a href="https://jlizier.github.io/jidt/">Java Information Dynamics Toolkit (JIDT)</a> is an open source Java library for performing information-theoretic causal discovery (i.e., transfer entropy, conditional transfer entropy, etc.)<a href="https://github.com/jlizier/jidt/wiki/Documentation">[5]</a>. Examples of using the library in <a href="/facts/MATLAB/qPjLISCk">MATLAB</a>, <a href="/facts/GNU_Octave/vVVq5evM">GNU Octave</a>, <a href="/facts/Python_(programming_language)/YbuGqofa">Python</a>, <a href="/facts/R_(programming_language)/LSrkr8K8">R</a>, <a href="/facts/Julia_(programming_language)/AoB0PJ9C">Julia</a> and <a href="/facts/Clojure/3JRNxKXx">Clojure</a> are provided in the documentation <a href="https://github.com/jlizier/jidt">[6]</a>.</li> <li><a href="https://cran.r-project.org/web/packages/pcalg/index.html">pcalg</a> is an <a href="/facts/R_(programming_language)/LSrkr8K8">R</a> package that provides some of the same causal discovery algorithms provided in Tetrad <a href="https://cran.r-project.org/web/packages/pcalg/vignettes/pcalgDoc.pdf">[7]</a> <a href="https://web.archive.org/web/20170720172611/https://cran.r-project.org/web/packages/pcalg/vignettes/pcalgDoc.pdf">Archived</a> 2017-07-20 at the <a href="/facts/Wayback_Machine/nmQ3a6JC">Wayback Machine</a>.</li></ul> <h3>Specific Techniques</h3> <h4>Granger causality</h4> <ul><li><a href="/facts/R_(programming_language)/LSrkr8K8">R</a> package <a href="https://rdrr.io/cran/lmtest/man/grangertest.html">[8]</a></li> <li><a href="/facts/Python_(programming_language)/YbuGqofa">Python</a> package <a href="https://www.statsmodels.org/dev/generated/statsmodels.tsa.stattools.grangercausalitytests.html">[9]</a></li></ul> <h4>convergent cross mapping</h4> <ul><li><a href="/facts/R_(programming_language)/LSrkr8K8">R</a> package <a href="https://cran.r-project.org/web/packages/rEDM/vignettes/rEDM-tutorial.html">[10]</a> <a href="https://web.archive.org/web/20190925235524/https://cran.r-project.org/web/packages/rEDM/vignettes/rEDM-tutorial.html">Archived</a> 2019-09-25 at the <a href="/facts/Wayback_Machine/nmQ3a6JC">Wayback Machine</a></li></ul> <h4>LiNGAM</h4> <ul><li><a href="/facts/MATLAB/qPjLISCk">MATLAB</a>/<a href="/facts/GNU_Octave/vVVq5evM">GNU Octave</a> package <a href="https://sites.google.com/site/sshimizu06/lingam/">[11]</a></li></ul> <p>There is also a collection of tools and data maintained by the Causality Workbench team <a href="http://www.causality.inf.ethz.ch/resources.php">[12]</a> and the CCD team <a href="https://www.ccd.pitt.edu/tools/">[13]</a>. </p> <h2 id="references">References</h2> <ol> <li id="fn:1"><p>Rohlfing, Ingo; Schneider, Carsten Q. (2018). "A Unifying Framework for Causal Analysis in Set-Theoretic Multimethod Research" (PDF). Sociological Methods & Research. 47 (1): 37–63. doi:10.1177/0049124115626170. S2CID 124804330. Archived from the original (PDF) on 9 October 2022. Retrieved 29 February 2020. <a href="https://ghostarchive.org/archive/20221009/https://publications.ceu.edu/sites/default/files/publications/0049124115626170.pdf" target="_blank">https://ghostarchive.org/archive/20221009/https://publications.ceu.edu/sites/default/files/publications/0049124115626170.pdf</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></p></li> <li id="fn:2"><p>Brady, Henry E. (7 July 2011). "Causation and Explanation in Social Science". The Oxford Handbook of Political Science. doi:10.1093/oxfordhb/9780199604456.013.0049. Retrieved 29 February 2020. <a href="https://www.oxfordhandbooks.com/view/10.1093/oxfordhb/9780199604456.001.0001/oxfordhb-9780199604456-e-049" target="_blank">https://www.oxfordhandbooks.com/view/10.1093/oxfordhb/9780199604456.001.0001/oxfordhb-9780199604456-e-049</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></p></li> <li id="fn:3"><p>Spirtes, P.; Glymour, C.; Scheines, R. (2012). Causation, Prediction, and Search. Springer Science & Business Media. ISBN 978-1461227489. <a href="978-1461227489" target="_blank">978-1461227489</a> <a href="#fnref:3" class="footnote-back-ref">↩</a></p></li> <li id="fn:4"><p>Rosenbaum, Paul (2017). Observation and Experiment: An Introduction to Causal Inference. Harvard University Press. ISBN 9780674975576. <a href="9780674975576" target="_blank">9780674975576</a> <a href="#fnref:4" class="footnote-back-ref">↩</a></p></li> <li id="fn:5"><p>McCracken, James (2016). Exploratory Causal Analysis with Time Series Data (Synthesis Lectures on Data Mining and Knowledge Discovery). Morgan & Claypool Publishers. ISBN 978-1627059343. <a href="978-1627059343" target="_blank">978-1627059343</a> <a href="#fnref:5" class="footnote-back-ref">↩</a></p></li> <li id="fn:6"><p>Tukey, John W. (1977). Exploratory Data Analysis. Pearson. ISBN 978-0201076165. <a href="978-0201076165" target="_blank">978-0201076165</a> <a href="#fnref:6" class="footnote-back-ref">↩</a></p></li> <li id="fn:7"><p>Spirtes, P.; Glymour, C.; Scheines, R. (2012). Causation, Prediction, and Search. Springer Science & Business Media. ISBN 978-1461227489. <a href="978-1461227489" target="_blank">978-1461227489</a> <a href="#fnref:7" class="footnote-back-ref">↩</a></p></li> <li id="fn:8"><p>Rosenbaum, Paul (2017). Observation and Experiment: An Introduction to Causal Inference. Harvard University Press. ISBN 9780674975576. <a href="9780674975576" target="_blank">9780674975576</a> <a href="#fnref:8" class="footnote-back-ref">↩</a></p></li> <li id="fn:9"><p>Pearl, Judea (2018). The Book of Why: The New Science of Cause and Effect. Basic Books. ISBN 978-0465097616. <a href="978-0465097616" target="_blank">978-0465097616</a> <a href="#fnref:9" class="footnote-back-ref">↩</a></p></li> <li id="fn:10"><p>Kleinberg, Samantha (2015). Why: A Guide to Finding and Using Causes. O'Reilly Media, Inc. ISBN 978-1491952191. <a href="978-1491952191" target="_blank">978-1491952191</a> <a href="#fnref:10" class="footnote-back-ref">↩</a></p></li> <li id="fn:11"><p>Illari, P.; Russo, F. (2014). Causality: Philosophical Theory meets Scientific Practice. OUP Oxford. ISBN 978-0191639685. <a href="978-0191639685" target="_blank">978-0191639685</a> <a href="#fnref:11" class="footnote-back-ref">↩</a></p></li> <li id="fn:12"><p>Fisher, R. (1937). The design of experiments. Oliver And Boyd. <a href="#fnref:12" class="footnote-back-ref">↩</a></p></li> <li id="fn:13"><p>Hill, B. (1955). Principles of Medical Statistics. Lancet Limited. <a href="#fnref:13" class="footnote-back-ref">↩</a></p></li> <li id="fn:14"><p>Halpern, J. (2016). Actual Causality. MIT Press. ISBN 978-0262035026. <a href="978-0262035026" target="_blank">978-0262035026</a> <a href="#fnref:14" class="footnote-back-ref">↩</a></p></li> <li id="fn:15"><p>Pearl, J.; Glymour, M.; Jewell, N. P. (2016). Causal inference in statistics: a primer. John Wiley & Sons. ISBN 978-1119186847. <a href="978-1119186847" target="_blank">978-1119186847</a> <a href="#fnref:15" class="footnote-back-ref">↩</a></p></li> <li id="fn:16"><p>Stone, R. (1993). "The Assumptions on Which Causal Inferences Rest". Journal of the Royal Statistical Society. Series B (Methodological). 55 (2): 455–466. doi:10.1111/j.2517-6161.1993.tb01915.x. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:16" class="footnote-back-ref">↩</a></p></li> <li id="fn:17"><p>Spirtes, P.; Glymour, C.; Scheines, R. (2012). Causation, Prediction, and Search. Springer Science & Business Media. ISBN 978-1461227489. <a href="978-1461227489" target="_blank">978-1461227489</a> <a href="#fnref:17" class="footnote-back-ref">↩</a></p></li> <li id="fn:18"><p>Rosenbaum, Paul (2017). Observation and Experiment: An Introduction to Causal Inference. Harvard University Press. ISBN 9780674975576. <a href="9780674975576" target="_blank">9780674975576</a> <a href="#fnref:18" class="footnote-back-ref">↩</a></p></li> <li id="fn:19"><p>Granger, C (1980). "Testing for causality: a personal viewpoint". Journal of Economic Dynamics and Control. 2: 329–352. doi:10.1016/0165-1889(80)90069-X. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:19" class="footnote-back-ref">↩</a></p></li> <li id="fn:20"><p>McCracken, James (2016). Exploratory Causal Analysis with Time Series Data (Synthesis Lectures on Data Mining and Knowledge Discovery). Morgan & Claypool Publishers. ISBN 978-1627059343. <a href="978-1627059343" target="_blank">978-1627059343</a> <a href="#fnref:20" class="footnote-back-ref">↩</a></p></li> <li id="fn:21"><p>Spirtes, P.; Glymour, C.; Scheines, R. (2012). Causation, Prediction, and Search. Springer Science & Business Media. ISBN 978-1461227489. <a href="978-1461227489" target="_blank">978-1461227489</a> <a href="#fnref:21" class="footnote-back-ref">↩</a></p></li> <li id="fn:22"><p>Granger, C. W. J. (1969). "Investigating Causal Relations by Econometric Models and Cross-spectral Methods". Econometrica. 37 (3): 424–438. doi:10.2307/1912791. JSTOR 1912791. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:22" class="footnote-back-ref">↩</a></p></li> <li id="fn:23"><p>Granger, Clive. "Prize Lecture. NobelPrize.org. Nobel Media AB 2018". <a href="https://www.nobelprize.org/prizes/economic-sciences/2003/granger/lecture/" target="_blank">https://www.nobelprize.org/prizes/economic-sciences/2003/granger/lecture/</a> <a href="#fnref:23" class="footnote-back-ref">↩</a></p></li> <li id="fn:24"><p>McCracken, James (2016). Exploratory Causal Analysis with Time Series Data (Synthesis Lectures on Data Mining and Knowledge Discovery). Morgan & Claypool Publishers. ISBN 978-1627059343. <a href="978-1627059343" target="_blank">978-1627059343</a> <a href="#fnref:24" class="footnote-back-ref">↩</a></p></li> <li id="fn:25"><p>Woodward, James (2004). Making Things Happen: A Theory of Causal Explanation (Oxford Studies in the Philosophy of Science). Oxford University Press. ISBN 978-1435619999. <a href="978-1435619999" target="_blank">978-1435619999</a> <a href="#fnref:25" class="footnote-back-ref">↩</a></p></li> <li id="fn:26"><p>Spirtes, P.; Glymour, C.; Scheines, R. (2012). Causation, Prediction, and Search. Springer Science & Business Media. ISBN 978-1461227489. <a href="978-1461227489" target="_blank">978-1461227489</a> <a href="#fnref:26" class="footnote-back-ref">↩</a></p></li> <li id="fn:27"><p>Spirtes, P.; Glymour, C. (1991). "An algorithm for fast recovery of sparse causal graphs". Social Science Computer Review. 9 (1): 62–72. doi:10.1177/089443939100900106. S2CID 38398322. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:27" class="footnote-back-ref">↩</a></p></li> <li id="fn:28"><p>Guo, Ruocheng; Cheng, Lu; Li, Jundong; Hahn, P. Richard; Liu, Huan (2020). "A Survey of Learning Causality with Data". ACM Computing Surveys. 53 (4): 1–37. arXiv:1809.09337. doi:10.1145/3397269. S2CID 52822969. <a href="/wiki/ArXiv_(identifier)" target="_blank">/wiki/ArXiv_(identifier)</a> <a href="#fnref:28" class="footnote-back-ref">↩</a></p></li> <li id="fn:29"><p>Spirtes, P.; Glymour, C.; Scheines, R. (2012). Causation, Prediction, and Search. Springer Science & Business Media. ISBN 978-1461227489. <a href="978-1461227489" target="_blank">978-1461227489</a> <a href="#fnref:29" class="footnote-back-ref">↩</a></p></li> <li id="fn:30"><p>McCracken, James (2016). Exploratory Causal Analysis with Time Series Data (Synthesis Lectures on Data Mining and Knowledge Discovery). Morgan & Claypool Publishers. ISBN 978-1627059343. <a href="978-1627059343" target="_blank">978-1627059343</a> <a href="#fnref:30" class="footnote-back-ref">↩</a></p></li> <li id="fn:31"><p>Guo, Ruocheng; Cheng, Lu; Li, Jundong; Hahn, P. Richard; Liu, Huan (2020). "A Survey of Learning Causality with Data". ACM Computing Surveys. 53 (4): 1–37. arXiv:1809.09337. doi:10.1145/3397269. S2CID 52822969. <a href="/wiki/ArXiv_(identifier)" target="_blank">/wiki/ArXiv_(identifier)</a> <a href="#fnref:31" class="footnote-back-ref">↩</a></p></li> <li id="fn:32"><p>Malinsky, Daniel; Danks, David (2018). "Causal discovery algorithms: A practical guide". Philosophy Compass. 13 (1): e12470. doi:10.1111/phc3.12470. <a href="https://doi.org/10.1111%2Fphc3.12470" target="_blank">https://doi.org/10.1111%2Fphc3.12470</a> <a href="#fnref:32" class="footnote-back-ref">↩</a></p></li> <li id="fn:33"><p>Spirtes, P.; Zhang, K. (2016). "Causal discovery and inference: concepts and recent methodological advances". Appl Inform (Berl). 3: 3. doi:10.1186/s40535-016-0018-x. PMC 4841209. PMID 27195202. <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4841209" target="_blank">https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4841209</a> <a href="#fnref:33" class="footnote-back-ref">↩</a></p></li> <li id="fn:34"><p>Yu, Kui; Li, Jiuyong; Liu, Lin; Richard Hahn, P.; Liu, Huan (2016). "A review on algorithms for constraint-based causal discovery". arXiv:1611.03977 [cs.AI]. <a href="/wiki/ArXiv_(identifier)" target="_blank">/wiki/ArXiv_(identifier)</a> <a href="#fnref:34" class="footnote-back-ref">↩</a></p></li> <li id="fn:35"><p>Sun, Jie; Bollt, Erik M.; Li, Jundong; Richard Hahn, P.; Liu, Huan (2014). "Causation entropy identifies indirect influences, dominance of neighbors and anticipatory couplings". Physica D: Nonlinear Phenomena. 267: 49–57. arXiv:1504.03769. Bibcode:2014PhyD..267...49S. doi:10.1016/j.physd.2013.07.001. S2CID 14422483. <a href="/wiki/ArXiv_(identifier)" target="_blank">/wiki/ArXiv_(identifier)</a> <a href="#fnref:35" class="footnote-back-ref">↩</a></p></li> <li id="fn:36"><p>Spirtes, P.; Glymour, C.; Scheines, R. (2012). Causation, Prediction, and Search. Springer Science & Business Media. ISBN 978-1461227489. <a href="978-1461227489" target="_blank">978-1461227489</a> <a href="#fnref:36" class="footnote-back-ref">↩</a></p></li> <li id="fn:37"><p>Freedman, David; Humphreys, Paul (1999). "Are there algorithms that discover causal structure?". Synthese. 121 (1–2): 29–54. doi:10.1023/A:1005277613752. S2CID 6826436. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:37" class="footnote-back-ref">↩</a></p></li> <li id="fn:38"><p>Spirtes, P.; Glymour, C.; Scheines, R. (2012). Causation, Prediction, and Search. Springer Science & Business Media. ISBN 978-1461227489. <a href="978-1461227489" target="_blank">978-1461227489</a> <a href="#fnref:38" class="footnote-back-ref">↩</a></p></li> <li id="fn:39"><p>Raghu, V. K.; Ramsey, J. D.; Morris, A.; Manatakis, D. V.; Sprites, P.; Chrysanthis, P. K.; Glymour, C.; Benos, P. V. (2018). "Comparison of strategies for scalable causal discovery of latent variable models from mixed data". International Journal of Data Science and Analytics. 6 (33): 33–45. doi:10.1007/s41060-018-0104-3. PMC 6096780. PMID 30148202. <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6096780" target="_blank">https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6096780</a> <a href="#fnref:39" class="footnote-back-ref">↩</a></p></li> <li id="fn:40"><p>Shimizu, S (2014). "LiNGAM: non-Gaussian methods for estimating causal structures". Behaviormetrika. 41 (1): 65–98. doi:10.2333/bhmk.41.65. S2CID 49238101. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:40" class="footnote-back-ref">↩</a></p></li> <li id="fn:41"><p>Spirtes, P.; Glymour, C.; Scheines, R. (2012). Causation, Prediction, and Search. Springer Science & Business Media. ISBN 978-1461227489. <a href="978-1461227489" target="_blank">978-1461227489</a> <a href="#fnref:41" class="footnote-back-ref">↩</a></p></li> <li id="fn:42"><p>Cheek, C.; Zheng, H.; Hallstrom, B. R.; Hughes, R. E. (2018). "Application of a Causal Discovery Algorithm to the Analysis of Arthroplasty Registry Data". Biomedical Engineering and Computational Biology. 9: 117959721875689. doi:10.1177/1179597218756896. PMC 5826097. PMID 29511363. <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5826097" target="_blank">https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5826097</a> <a href="#fnref:42" class="footnote-back-ref">↩</a></p></li> <li id="fn:43"><p>Wen, X.; Rangarajan, G.; Ding, M. (2013). "Is Granger Causality a Viable Technique for Analyzing fMRI Data?". PLOS ONE. 8 (7): e67428. Bibcode:2013PLoSO...867428W. doi:10.1371/journal.pone.0067428. PMC 3701552. PMID 23861763. <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3701552" target="_blank">https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3701552</a> <a href="#fnref:43" class="footnote-back-ref">↩</a></p></li> <li id="fn:44"><p>Ebert-Uphoff, Imme; Deng, Yi (2017). "Causal discovery in the geosciences—Using synthetic data to learn how to interpret results". Computers & Geosciences. 99: 50–60. Bibcode:2017CG.....99...50E. doi:10.1016/j.cageo.2016.10.008. <a href="https://doi.org/10.1016%2Fj.cageo.2016.10.008" target="_blank">https://doi.org/10.1016%2Fj.cageo.2016.10.008</a> <a href="#fnref:44" class="footnote-back-ref">↩</a></p></li> <li id="fn:45"><p>McCracken, J. M.; Weigel, R. S.; Li, Jundong; Richard Hahn, P.; Liu, Huan (2014). "Convergent cross-mapping and pairwise asymmetric inference". Phys. Rev. E. 90 (6): 062903. arXiv:1407.5696. Bibcode:2014PhRvE..90f2903M. doi:10.1103/PhysRevE.90.062903. PMID 25615160. S2CID 7506718. <a href="/wiki/ArXiv_(identifier)" target="_blank">/wiki/ArXiv_(identifier)</a> <a href="#fnref:45" class="footnote-back-ref">↩</a></p></li> <li id="fn:46"><p>Spirtes, P.; Glymour, C.; Scheines, R. (2012). Causation, Prediction, and Search. Springer Science & Business Media. ISBN 978-1461227489. <a href="978-1461227489" target="_blank">978-1461227489</a> <a href="#fnref:46" class="footnote-back-ref">↩</a></p></li> <li id="fn:47"><p>Freedman, David; Humphreys, Paul (1999). "Are there algorithms that discover causal structure?". Synthese. 121 (1–2): 29–54. doi:10.1023/A:1005277613752. S2CID 6826436. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:47" class="footnote-back-ref">↩</a></p></li> <li id="fn:48"><p>Pearl, Judea (2018). The Book of Why: The New Science of Cause and Effect. Basic Books. ISBN 978-0465097616. <a href="978-0465097616" target="_blank">978-0465097616</a> <a href="#fnref:48" class="footnote-back-ref">↩</a></p></li> <li id="fn:49"><p>Spirtes, P.; Glymour, C.; Scheines, R. (2012). Causation, Prediction, and Search. Springer Science & Business Media. ISBN 978-1461227489. <a href="978-1461227489" target="_blank">978-1461227489</a> <a href="#fnref:49" class="footnote-back-ref">↩</a></p></li> <li id="fn:50"><p>Stone, R. (1993). "The Assumptions on Which Causal Inferences Rest". Journal of the Royal Statistical Society. Series B (Methodological). 55 (2): 455–466. doi:10.1111/j.2517-6161.1993.tb01915.x. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:50" class="footnote-back-ref">↩</a></p></li> <li id="fn:51"><p>Scheines, R. (1997). "An introduction to causal inference" (PDF). Causality in Crisis: 185–199. <a href="http://mlg.eng.cam.ac.uk/zoubin/SALD/Intro-Causal.pdf" target="_blank">http://mlg.eng.cam.ac.uk/zoubin/SALD/Intro-Causal.pdf</a> <a href="#fnref:51" class="footnote-back-ref">↩</a></p></li> <li id="fn:52"><p>Holland, P. W. (1986). "Statistics and causal inference". Journal of the American Statistical Association. 81 (396): 945–960. doi:10.1080/01621459.1986.10478354. S2CID 14377504. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:52" class="footnote-back-ref">↩</a></p></li> <li id="fn:53"><p>Imbens, G. W.; Rubin, D. B. (2015). Causal inference in statistics, social, and biomedical sciences. Cambridge University Press. ISBN 978-0521885881. <a href="978-0521885881" target="_blank">978-0521885881</a> <a href="#fnref:53" class="footnote-back-ref">↩</a></p></li> <li id="fn:54"><p>Freedman, David; Humphreys, Paul (1999). "Are there algorithms that discover causal structure?". Synthese. 121 (1–2): 29–54. doi:10.1023/A:1005277613752. S2CID 6826436. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:54" class="footnote-back-ref">↩</a></p></li> <li id="fn:55"><p>Morgan, S. L.; Winship, C. (2015). Counterfactuals and causal inference. Cambridge University Press. ISBN 978-1107065079. <a href="978-1107065079" target="_blank">978-1107065079</a> <a href="#fnref:55" class="footnote-back-ref">↩</a></p></li> <li id="fn:56"><p>"Causal Models and Statistical Data, The Tetrad Project". <a href="http://www.phil.cmu.edu/tetrad/publications.html" target="_blank">http://www.phil.cmu.edu/tetrad/publications.html</a> <a href="#fnref:56" class="footnote-back-ref">↩</a></p></li> <li id="fn:57"><p>"Tools, Center for Causal Discovery, University of Pittsburg". 10 August 2016. <a href="https://www.ccd.pitt.edu/tools/" target="_blank">https://www.ccd.pitt.edu/tools/</a> <a href="#fnref:57" class="footnote-back-ref">↩</a></p></li> </ol>