In business, data mining is the analysis of historical business activities, stored as static data in data warehouse databases. The goal is to reveal hidden patterns and trends. Data mining software uses advanced pattern recognition algorithms to sift through large amounts of data to assist in discovering previously unknown strategic business information. Examples of what businesses use data mining for include performing market analysis to identify new product bundles, finding the root cause of manufacturing problems, to prevent customer attrition and acquire new customers, cross-selling to existing customers, and profiling customers with more accuracy.
Data mining of government records – particularly records of the justice system (i.e., courts, prisons) – enables the discovery of systemic human rights violations in connection to generation and publication of invalid or fraudulent legal records by various government agencies.
Spatial data mining is the application of data mining methods to spatial data. The end objective of spatial data mining is to find patterns in data with respect to geography. So far, data mining and Geographic Information Systems (GIS) have existed as two separate technologies, each with its own methods, traditions, and approaches to visualization and data analysis. Particularly, most contemporary GIS have only very basic spatial analysis functionality. The immense explosion in geographically referenced data occasioned by developments in IT, digital mapping, remote sensing, and the global diffusion of GIS emphasizes the importance of developing data-driven inductive approaches to geographical analysis and modeling.
Data mining offers great potential benefits for GIS-based applied decision-making. Recently, the task of integrating these two technologies has become of critical importance, especially as various public and private sector organizations possessing huge databases with thematic and geographically referenced data begin to realize the huge potential of the information contained therein. Among those organizations are:
Challenges in Spatial mining:
Geospatial data repositories tend to be very large. Moreover, existing GIS datasets are often splintered into feature and attribute components that are conventionally archived in hybrid data management systems. Algorithmic requirements differ substantially for relational (attribute) data management and for topological (feature) data management. Related to this is the range and diversity of geographic data formats, which present unique challenges. The digital geographic data revolution is creating new types of data formats beyond the traditional "vector" and "raster" formats. Geographic data repositories increasingly include ill-structured data, such as imagery and geo-referenced multi-media.
There are several critical research challenges in geographic knowledge discovery and data mining. Miller and Han offer the following list of emerging research topics in the field:
Data may contain attributes generated and recorded at different times. In this case finding meaningful relationships in the data may require considering the temporal order of the attributes. A temporal relationship may indicate a causal relationship, or simply an association.
In the process of turning from analog into digital, large data sets have been generated, collected, and stored discovering statistical patterns, trends and information which is hidden in data, in order to build predictive patterns. Studies suggest visual data mining is faster and much more intuitive than is traditional data mining. See also Computer vision.
"Subject-based data mining" is a data mining method involving the search for associations between individuals in data. In the context of combating terrorism, the National Research Council provides the following definition: "Subject-based data mining uses an initiating individual or other datum that is considered, based on other information, to be of high interest, and the goal is to determine what other persons or financial transactions or movements, etc., are related to that initiating datum."
Knowledge discovery "On the Grid" generally refers to conducting knowledge discovery in an open environment using grid computing concepts, allowing users to integrate data from various online data sources, as well make use of remote resources, for executing their data mining tasks. The earliest example was the Discovery Net, developed at Imperial College London, which won the "Most Innovative Data-Intensive Application Award" at the ACM SC02 (Supercomputing 2002) conference and exhibition, based on a demonstration of a fully interactive distributed knowledge discovery application for a bioinformatics application. Other examples include work conducted by researchers at the University of Calabria, who developed a Knowledge Grid architecture for distributed knowledge discovery, based on grid computing.
O'Brien, J. A., & Marakas, G. M. (2011). Management Information Systems. New York, NY: McGraw-Hill/Irwin.
Alexander, Doug. "Data Mining". The University of Texas at Austin: College of Liberal Arts. http://www.laits.utexas.edu/~anorman/BUS.FOR/course.mat/Alex/
"Daniele Medri: Big Data & Business: An on-going revolution". Statistics Views. 21 Oct 2013. Archived from the original on 17 June 2015. Retrieved 21 September 2015. https://web.archive.org/web/20150617211645/http://www.statisticsviews.com/details/feature/5393251/Big-Data--Business-An-on-going-revolution.html
"Large Scale Item Categorization" (PDF). Archived from the original (PDF) on 2015-10-05. https://web.archive.org/web/20151005215055/http://labs.ebay.com/wp-content/uploads/2012/10/Jean-David-Ruvini-Large-scale_Item.pdf
Goss, S. (2013, April 10). Data-mining and our personal privacy. Retrieved from The Telegraph: "Data-mining and our personal privacy | the Sun News | Macon.com". Archived from the original on 2014-07-05. Retrieved 2015-09-21. https://web.archive.org/web/20140705055127/http://www.macon.com/2013/04/10/2429775/data-mining-and-our-personal-privacy.html
Monk, Ellen; Wagner, Bret (2006). Concepts in Enterprise Resource Planning, Second Edition. Boston, MA: Thomson Course Technology. ISBN 978-0-619-21663-4. OCLC 224465825. 978-0-619-21663-4
Chalutz-Ben Gal, H. (2023). Rokach, L.; Maimon, O.; Shmueli, E. (eds.). "Human Resources-Based Organizational Data Mining (HRODM): Themes, Trends, Focus, Future" (PDF). Springer. pp. 833–866. https://www.lambda.sites.tau.ac.il/_files/ugd/901879_491d6bb99b584942b6efaefaa0acb282.pdf
Elovici, Yuval; Braha, Dan (2003). "A Decision-Theoretic Approach to Data Mining" (PDF). IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans. 33 (1): 42–51. doi:10.1109/TSMCA.2003.812596. hdl:10150/105859. http://necsi.edu/affiliates/braha/IEEE_Decision_Theoretic.pdf
Battiti, Roberto; and Brunato, Mauro; Reactive Business Intelligence. From Data to Models to Insight, Reactive Search Srl, Italy, February 2011. ISBN 978-88-905795-0-9. http://www.reactivebusinessintelligence.com/
Battiti, Roberto; Passerini, Andrea (2010). "Brain-Computer Evolutionary Multi-Objective Optimization (BC-EMO): a genetic algorithm adapting to the decision maker" (PDF). IEEE Transactions on Evolutionary Computation. 14 (15): 671–687. doi:10.1109/TEVC.2010.2058118. S2CID 2182650. http://rtm.science.unitn.it/~battiti/archive/bcemo.pdf
Elovici, Yuval; Braha, Dan (2003). "A Decision-Theoretic Approach to Data Mining" (PDF). IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans. 33 (1): 42–51. doi:10.1109/TSMCA.2003.812596. hdl:10150/105859. http://necsi.edu/affiliates/braha/IEEE_Decision_Theoretic.pdf
Elovici, Yuval; Braha, Dan (2003). "A Decision-Theoretic Approach to Data Mining" (PDF). IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans. 33 (1): 42–51. doi:10.1109/TSMCA.2003.812596. hdl:10150/105859. http://necsi.edu/affiliates/braha/IEEE_Decision_Theoretic.pdf
Braha, Dan; Elovici, Yuval; Last, Mark (2007). "Theory of actionable data mining with application to semiconductor manufacturing control" (PDF). International Journal of Production Research. 45 (13): 3059–3084. CiteSeerX 10.1.1.127.1472. doi:10.1080/00207540600654475. S2CID 2299178. http://necsi.edu/affiliates/braha/TPRS_A_165421_O.pdf
Fountain, Tony; Dietterich, Thomas; and Sudyka, Bill (2000); Mining IC Test Data to Optimize VLSI Testing, in Proceedings of the Sixth ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, ACM Press, pp. 18–25 http://web.engr.oregonstate.edu/~tgd/publications/kdd2000-dlft.pdf
Braha, Dan; Shmilovici, Armin (2002). "Data Mining for Improving a Cleaning Process in the Semiconductor Industry" (PDF). IEEE Transactions on Semiconductor Manufacturing. 15 (1): 91–101. CiteSeerX 10.1.1.10.7921. doi:10.1109/66.983448. http://necsi.edu/affiliates/braha/IEEE-Cleaning_02.pdf
Braha, Dan; Shmilovici, Armin (2003). "On the Use of Decision Tree Induction for Discovery of Interactions in a Photolithographic Process" (PDF). IEEE Transactions on Semiconductor Manufacturing. 16 (4): 644–652. doi:10.1109/TSM.2003.818959. http://necsi.edu/affiliates/braha/IEEE_Decision_Trees.pdf
Zhu, Xingquan; Davidson, Ian (2007). Knowledge Discovery and Data Mining: Challenges and Realities. New York, NY: Hershey. p. 18. ISBN 978-1-59904-252-7. 978-1-59904-252-7
McGrail, Anthony J.; Gulski, Edward; Allan, David; Birtwhistle, David; Blackburn, Trevor R.; Groot, Edwin R. S. "Data Mining Techniques to Assess the Condition of High Voltage Electrical Plant". CIGRÉ WG 15.11 of Study Committee 15.
McGrail, Anthony J.; Gulski, Edward; Allan, David; Birtwhistle, David; Blackburn, Trevor R.; Groot, Edwin R. S. "Data Mining Techniques to Assess the Condition of High Voltage Electrical Plant". CIGRÉ WG 15.11 of Study Committee 15.
Baker, Ryan S. J. d. "Is Gaming the System State-or-Trait? Educational Data Mining Through the Multi-Contextual Application of a Validated Behavioral Model". Workshop on Data Mining for User Modeling 2007.
Superby Aguirre, Juan Francisco; Vandamme, Jean-Philippe; Meskens, Nadine. "Determination of factors influencing the achievement of the first-year university students using data mining methods". Workshop on Educational Data Mining 2006.
Zhu, Xingquan; Davidson, Ian (2007). Knowledge Discovery and Data Mining: Challenges and Realities. New York, NY: Hershey. pp. 163–189. ISBN 978-1-59904-252-7. 978-1-59904-252-7
Zhu, Xingquan; Davidson, Ian (2007). Knowledge Discovery and Data Mining: Challenges and Realities. New York, NY: Hershey. pp. 31–48. ISBN 978-1-59904-252-7. 978-1-59904-252-7
Chen, Yudong; Zhang, Yi; Hu, Jianming; Li, Xiang (2006). "Traffic Data Analysis Using Kernel PCA and Self-Organizing Map". 2006 IEEE Intelligent Vehicles Symposium. pp. 472–477. doi:10.1109/IVS.2006.1689673. ISBN 978-4-901122-86-3. S2CID 16645060. 978-4-901122-86-3
Bate, Andrew; Lindquist, Marie; Edwards, I. Ralph; Olsson, Sten; Orre, Roland; Lansner, Anders; de Freitas, Rogelio Melhado (Jun 1998). "A Bayesian neural network method for adverse drug reaction signal generation" (PDF). European Journal of Clinical Pharmacology. 54 (4): 315–21. doi:10.1007/s002280050466. PMID 9696956. S2CID 25966839.[permanent dead link] http://dml.cs.byu.edu/~cgc/docs/atdm/W11/BCPNN-ADR.pdf
Norén, G. Niklas; Bate, Andrew; Hopstadius, Johan; Star, Kristina; and Edwards, I. Ralph (2008); Temporal Pattern Discovery for Trends and Transient Effects: Its Application to Patient Records. Proceedings of the Fourteenth International Conference on Knowledge Discovery and Data Mining (SIGKDD 2008), Las Vegas, NV, pp. 963–971.
Hiura, Satoko; Koseki, Shige; Koyama, Kento (2021-05-19). "Prediction of population behavior of Listeria monocytogenes in food using machine learning and a microbial growth and survival database". Scientific Reports. 11 (1): 10613. Bibcode:2021NatSR..1110613H. doi:10.1038/s41598-021-90164-z. ISSN 2045-2322. PMC 8134468. PMID 34012066. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8134468
Zernik, Joseph; Data Mining as a Civic Duty – Online Public Prisoners' Registration Systems, International Journal on Social Media: Monitoring, Measurement, Mining, 1: 84–96 (2010) https://www.scribd.com/doc/38328591/
Zernik, Joseph; Data Mining of Online Judicial Records of the Networked US Federal Courts, International Journal on Social Media: Monitoring, Measurement, Mining, 1:69–83 (2010) https://www.scribd.com/doc/38328585/
Gagliardi, F (2011). "Instance-based classifiers applied to medical databases: Diagnosis and knowledge extraction". Artificial Intelligence in Medicine. 52 (3): 123–139. doi:10.1016/j.artmed.2011.04.002. PMID 21621400. /wiki/Doi_(identifier)
Martínez-Arranz, Ibon; Mayo, Rebeca; Pérez-Cormenzana, Miriam; Mincholé, Itziar; Salazar, Lorena; Alonso, Cristina; Mato, José M. (2015). "Enhancing metabolomics research through data mining". Journal of Proteomics. 127 (Pt B): 275–288. doi:10.1016/j.jprot.2015.01.019. PMID 25668325. /wiki/Doi_(identifier)
Martínez-Arranz, Ibon; Mayo, Rebeca; Pérez-Cormenzana, Miriam; Mincholé, Itziar; Salazar, Lorena; Alonso, Cristina; Mato, José M. (2015). "Enhancing metabolomics research through data mining". Journal of Proteomics. 127 (Pt B): 275–288. doi:10.1016/j.jprot.2015.01.019. PMID 25668325. /wiki/Doi_(identifier)
David G. Savage (2011-06-24). "Pharmaceutical industry: Supreme Court sides with pharmaceutical industry in two decisions". Los Angeles Times. Retrieved 2012-11-07. https://www.latimes.com/archives/la-xpm-2011-jun-24-la-na-court-drugs-20110624-story.html
Goth, Gregory (2012). "Analyzing medical data". Communications of the ACM. 55 (6): 13. doi:10.1145/2184319.2184324. /wiki/Doi_(identifier)
"What is HITECH (Health Information Technology for Economic and Clinical Health) Act of 2009? | Definition from TechTarget". http://searchhealthit.techtarget.com/definition/HITECH-Act
Goth, Gregory (2012). "Analyzing medical data". Communications of the ACM. 55 (6): 13. doi:10.1145/2184319.2184324. /wiki/Doi_(identifier)
Goth, Gregory (2012). "Analyzing medical data". Communications of the ACM. 55 (6): 13. doi:10.1145/2184319.2184324. /wiki/Doi_(identifier)
Healey, Richard G. (1991); Database Management Systems, in Maguire, David J.; Goodchild, Michael F.; and Rhind, David W., (eds.), Geographic Information Systems: Principles and Applications, London, GB: Longman
Camara, Antonio S.; and Raper, Jonathan (eds.) (1999); Spatial Multimedia and Virtual Reality, London, GB: Taylor and Francis
Miller, Harvey J.; and Han, Jiawei (eds.) (2001); Geographic Data Mining and Knowledge Discovery, London, GB: Taylor & Francis
Ma, Y.; Richards, M.; Ghanem, M.; Guo, Y.; Hassard, J. (2008). "Air Pollution Monitoring and Mining Based on Sensor Grid in London". Sensors. 8 (6): 3601–3623. Bibcode:2008Senso...8.3601M. doi:10.3390/s8063601. PMC 3714656. PMID 27879895. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3714656
Ma, Y.; Guo, Y.; Tian, X.; Ghanem, M. (2011). "Distributed Clustering-Based Aggregation Algorithm for Spatial Correlated Sensor Networks". IEEE Sensors Journal. 11 (3): 641. Bibcode:2011ISenJ..11..641M. CiteSeerX 10.1.1.724.1158. doi:10.1109/JSEN.2010.2056916. S2CID 1639100. /wiki/Bibcode_(identifier)
Zhao, Kaidi; and Liu, Bing; Tirpark, Thomas M.; and Weimin, Xiao; A Visual Data Mining Framework for Convenient Identification of Useful Knowledge http://dl.acm.org/citation.cfm?id=1106390
Keim, Daniel A.; Information Visualization and Visual Data Mining http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.135.7051
Burch, Michael; Diehl, Stephan; Weißgerber, Peter; Visual Data Mining in Software Archives http://dl.acm.org/citation.cfm?doid=1056018.1056024
Pachet, François; Westermann, Gert; and Laigre, Damien; Musical Data Mining for Electronic Music Distribution Archived 2014-03-27 at the Wayback Machine, Proceedings of the 1st WedelMusic Conference, Firenze, Italy, 2001, pp. 101–106. http://www.csl.sony.fr/downloads/papers/2001/pachet01c.pdf
Government Accountability Office, Data Mining: Early Attention to Privacy in Developing a Key DHS Program Could Reduce Risks, GAO-07-293 (February 2007), Washington, DC
Secure Flight Program report, NBC News https://www.nbcnews.com/id/wbna20604775
"Total/Terrorism Information Awareness (TIA): Is It Truly Dead?". Electronic Frontier Foundation (official website). 2003. Archived from the original on 2009-03-25. Retrieved 2009-03-15. https://web.archive.org/web/20090325113304/http://w2.eff.org/Privacy/TIA/20031003_comments.php
Agrawal, Rakesh; Mannila, Heikki; Srikant, Ramakrishnan; Toivonen, Hannu; and Verkamo, A. Inkeri; Fast discovery of association rules, in Advances in knowledge discovery and data mining, MIT Press, 1996, pp. 307–328
National Research Council, Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment, Washington, DC: National Academies Press, 2008
Haag, Stephen; Cummings, Maeve; Phillips, Amy (2006). Management Information Systems for the information age. Toronto: McGraw-Hill Ryerson. p. 28. ISBN 978-0-07-095569-1. OCLC 63194770. 978-0-07-095569-1
National Research Council, Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment, Washington, DC: National Academies Press, 2008
Ghanem, Moustafa; Guo, Yike; Rowe, Anthony; Wendel, Patrick (2002). "Grid-based knowledge discovery services for high throughput informatics". Proceedings 11th IEEE International Symposium on High Performance Distributed Computing. p. 416. doi:10.1109/HPDC.2002.1029946. ISBN 978-0-7695-1686-8. S2CID 28782519. 978-0-7695-1686-8
Ghanem, Moustafa; Curcin, Vasa; Wendel, Patrick; Guo, Yike (2009). "Building and Using Analytical Workflows in Discovery Net". Data Mining Techniques in Grid Computing Environments. p. 119. doi:10.1002/9780470699904.ch8. ISBN 9780470699904. 9780470699904
Cannataro, Mario; Talia, Domenico (January 2003). "The Knowledge Grid: An Architecture for Distributed Knowledge Discovery" (PDF). Communications of the ACM. 46 (1): 89–93. doi:10.1145/602421.602425. S2CID 8709194. Archived from the original (PDF) on 2011-11-10. Retrieved 17 October 2011. https://web.archive.org/web/20111110142404/http://grid.deis.unical.it/papers/pdf/CACM2003.pdf
Talia, Domenico; Trunfio, Paolo (July 2010). "How distributed data mining tasks can thrive as knowledge services" (PDF). Communications of the ACM. 53 (7): 132–137. CiteSeerX 10.1.1.378.2206. doi:10.1145/1785414.1785451. S2CID 14713292. Archived from the original (PDF) on 2011-10-27. Retrieved 17 October 2011. https://web.archive.org/web/20111027003824/http://grid.deis.unical.it/papers/pdf/CACM2010.pdf