Data exploration

Data exploration is an approach similar to initial <a href="/facts/Data_analysis/CNtARxLW">data analysis</a>, whereby a <a href="/facts/Data_analyst/CNtARxLW">data analyst</a> uses visual exploration to understand what is in a dataset and the characteristics of the data, rather than through traditional data management systems. These characteristics can include size or amount of data, completeness of the data, correctness of the data, possible relationships amongst data elements or files/tables in the data.
Data exploration is typically conducted using a combination of automated and manual activities. Automated activities can include <a href="/facts/Data_profiling/K9zFvA8r">data profiling</a> or <a href="/facts/Data_visualization/ET1lSjFa">data visualization</a> or tabular reports to give the analyst an initial view into the data and an understanding of key characteristics.
This is often followed by manual <a href="/facts/Drill-down/F3oMzJDh">drill-down</a> or filtering of the data to identify anomalies or patterns identified through the automated actions. Data exploration can also require manual scripting and queries into the data (e.g. using languages such as <a href="/facts/SQL/UjQLJcYm">SQL</a> or <a href="/facts/R_(language)/LSrkr8K8">R</a>) or using <a href="/facts/Spreadsheet/0cvc1GeB">spreadsheets</a> or similar tools to view the <a href="/facts/Raw_data/fAVLcq7W">raw data</a>.
All of these activities are aimed at creating a mental model and understanding of the data in the mind of the analyst, and defining basic <a href="/facts/Metadata/qt35z79C">metadata</a> (statistics, structure, relationships) for the data set that can be used in further analysis.
Once this initial understanding of the data is had, the data can be pruned or refined by removing unusable parts of the data (<a href="/facts/Data_cleansing/FPVIcxV6">data cleansing</a>), correcting poorly formatted elements and defining relevant relationships across datasets. This process is also known as determining <a href="/facts/Data_quality/Hr9APbfG">data quality</a>.
Data exploration can also refer to the ad hoc querying or visualization of data to identify potential relationships or insights that may be hidden in the data and does not require to formulate assumptions beforehand.
Traditionally, this had been a key area of focus for statisticians, with <a href="/facts/John_Tukey/2a5FbPWb">John Tukey</a> being a key evangelist in the field. Today, data exploration is more widespread and is the focus of data analysts and <a href="/facts/Data_scientists/AHYUo0Rn">data scientists</a>; the latter being a relatively new role within enterprises and larger organizations.

Data exploration open-in-new

Data exploration