Data preprocessing

<p>Data preprocessing can refer to manipulation, filtration or augmentation of data before it is analyzed, and is often an important step in the <a href="/facts/Data_mining/AH53q5ac">data mining</a> process. <a href="/facts/Data_collection/rEGBdDzj">Data collection</a> methods are often loosely controlled, resulting in out-of-range values, impossible data combinations, and <a href="/facts/Missing_values/p2pKMXRV">missing values</a>, amongst other issues.
</p><p>The preprocessing pipeline used can often have large effects on the conclusions drawn from the downstream analysis. Thus, representation and <a href="/facts/Data_quality/Hr9APbfG">quality of data</a> is necessary before running any analysis. 
Often, data preprocessing is the most important phase of a <a href="/facts/Machine_learning/e0w0XJTu">machine learning</a> project, especially in <a href="/facts/Computational_biology/GxgNfwcX">computational biology</a>. If there is a high proportion of irrelevant and redundant information present or noisy and unreliable data, then <a href="/facts/Knowledge_discovery/eopEv228">knowledge discovery</a> during the training phase may be more difficult. <a href="/facts/Data_preparation/TSqKhkg8">Data preparation</a> and filtering steps can take a considerable amount of processing time. Examples of methods used in data preprocessing include <a href="/facts/Data_cleaning/FPVIcxV6">cleaning</a>, <a href="/facts/Instance_selection/PCXqL3DT">instance selection</a>, <a href="/facts/Data_normalization/K2jhB44k">normalization</a>, <a href="/facts/One-hot/d6YJwj3d">one-hot encoding</a>, <a href="/facts/Data_transformation_(statistics)/6wqoUa66">data transformation</a>, <a href="/facts/Feature_extraction/BSMX5e8I">feature extraction</a> and <a href="/facts/Feature_selection/gxbuF4VG">feature selection</a>.
</p>

Data preprocessing open-in-new

Data preprocessing