Text mining

Text mining, text data mining (TDM) or text analytics is the process of deriving high-quality <a href="/facts/Information/3LdrzXtJ">information</a> from <a href="/facts/Plain_text/TSlSSAg2">text</a>. It involves "the discovery by computer of new, previously unknown information, by automatically extracting information from different written resources." Written resources may include <a href="/facts/Website/SAsjJnca">websites</a>, <a href="/facts/Book/Yylgt4AC">books</a>, <a href="/facts/Email/6pD9GLtI">emails</a>, <a href="/facts/Review/QlsuxDnn">reviews</a>, and articles. High-quality information is typically obtained by devising patterns and trends by means such as <a href="/facts/Pattern_recognition/o1s38opj">statistical pattern learning</a>. According to Hotho et al. (2005), there are three perspectives of text mining: <a href="/facts/Information_extraction/DL9nMD2X">information extraction</a>, <a href="/facts/Data_mining/AH53q5ac">data mining</a>, and <a href="/facts/Knowledge_discovery_in_databases/AH53q5ac">knowledge discovery in databases</a> (KDD). Text mining usually involves the process of structuring the input text (usually <a href="/facts/Parsing/nRKB01S3">parsing</a>, along with the addition of some derived linguistic features and the removal of others, and subsequent insertion into a <a href="/facts/Database/VYcpkevb">database</a>), deriving patterns within the <a href="/facts/Structured_data/j1YyzpJ3">structured data</a>, and finally evaluation and interpretation of the output. 'High quality' in text mining usually refers to some combination of <a href="/facts/Relevance_(information_retrieval)/0kkQ6B9A">relevance</a>, <a href="/facts/Novelty_(patent)/cYLSHC3w">novelty</a>, and interest. Typical text mining tasks include <a href="/facts/Text_categorization/B69lL6BH">text categorization</a>, <a href="/facts/Text_clustering/omXnhdeP">text clustering</a>, concept/entity extraction, production of granular taxonomies, <a href="/facts/Sentiment_analysis/QRdywf6g">sentiment analysis</a>, <a href="/facts/Document_summarization/yTQYaEIe">document summarization</a>, and <a href="/facts/Entity%25E2%2580%2593relationship_model/U8A0mwYe">entity relation modeling</a> (i.e., learning relations between <a href="/facts/Named_entity_recognition/ko6GRQQd">named entities</a>).
Text analysis involves <a href="/facts/Information_retrieval/MFSmpeNw">information retrieval</a>, <a href="/facts/Lexical_analysis/T1JYWpIf">lexical analysis</a> to study word frequency distributions, <a href="/facts/Pattern_recognition/o1s38opj">pattern recognition</a>, <a href="/facts/Tag_(metadata)/9CfRkCpI">tagging</a>/<a href="/facts/Annotation/dGfVSUKc">annotation</a>, <a href="/facts/Information_extraction/DL9nMD2X">information extraction</a>, <a href="/facts/Data_mining/AH53q5ac">data mining</a> techniques including link and association analysis, <a href="/facts/Information_visualization/DSHteV85">visualization</a>, and <a href="/facts/Predictive_analytics/tbqzk5qe">predictive analytics</a>. The overarching goal is, essentially, to turn text into data for analysis, via the application of <a href="/facts/Natural_language_processing/1hjMKsSN">natural language processing</a> (NLP), different types of <a href="/facts/Algorithm/fnl5NmRt">algorithms</a> and analytical methods. An important phase of this process is the interpretation of the gathered information.
A typical application is to scan a set of documents written in a <a href="/facts/Natural_language/B4gLS7Kd">natural language</a> and either model the <a href="/facts/Document/H4Hdxu91">document</a> set for <a href="/facts/Predictive_classification/tbqzk5qe">predictive classification</a> purposes or populate a database or search index with the information extracted. The <a href="/facts/Document/H4Hdxu91">document</a> is the basic element when starting with text mining. Here, we define a document as a unit of textual data, which normally exists in many types of collections.

Text mining open-in-new

Text mining