Menu
Home
People
Places
Arts
History
Plants & Animals
Science
Life & Culture
Technology
Reference.org
OpenRefine
open-in-new
<h2 id="uses">Uses</h2> <ul><li><i>Cleaning messy data</i>: for example if working with a text file with some semi-structured data, it can be edited using transformations, facets and clustering to make the data cleanly structured.<a class="footnote-ref" id="fnref:6" href="#fn:6"><sup>6</sup></a></li> <li><i>Transformation of data</i>: converting values to other formats, normalizing and denormalizing.</li> <li><i>Parsing data from web sites</i>: OpenRefine has a URL fetch feature and <a href="/facts/Jsoup/4cqaWGLA">jsoup</a> HTML parser and DOM engine.<a class="footnote-ref" id="fnref:7" href="#fn:7"><sup>7</sup></a></li> <li><i>Adding data to dataset by fetching it from web services</i> (i.e. returning <a href="/facts/JSON/bs1rUAhd">JSON</a>).<a class="footnote-ref" id="fnref:8" href="#fn:8"><sup>8</sup></a> For example, can be used for <a href="/facts/Geocoding/KtTELe73">geocoding</a> addresses to <a href="/facts/Geographic_coordinates/RkjfcY6k">geographic coordinates</a>.<a class="footnote-ref" id="fnref:9" href="#fn:9"><sup>9</sup></a></li> <li><i>Aligning to <a href="/facts/Wikidata/5Myy74K5">Wikidata</a></i> (formerly <a href="/facts/Freebase_(database)/oT1kwnQ2">Freebase</a><a class="footnote-ref" id="fnref:10" href="#fn:10"><sup>10</sup></a>): this involves <i>reconciliation</i> — mapping string values in cells to entities in Wikidata.<a class="footnote-ref" id="fnref:11" href="#fn:11"><sup>11</sup></a></li></ul> <h2 id="supported-formats">Supported formats</h2> <p>Import is supported from following formats:<a class="footnote-ref" id="fnref:12" href="#fn:12"><sup>12</sup></a> </p> <ul><li><a href="/facts/Tab-separated_values/TRyPhWXD">TSV</a>, <a href="/facts/Comma-separated_values/dv1pSR1r">CSV</a></li> <li>Text file with custom separators or columns split by fixed width</li> <li><a href="/facts/XML/clNgUfWk">XML</a></li> <li><a href="/facts/Resource_Description_Framework/WMufyukm">RDF</a> triples (<a href="/facts/RDF%2fXML/vsRwLAVS">RDF/XML</a> and <a href="/facts/Notation3/1Lw1bfQc">Notation3</a> serialization formats)</li> <li><a href="/facts/JSON/bs1rUAhd">JSON</a></li> <li><a href="/facts/Google_Spreadsheets/bTTDJYFw">Google Spreadsheets</a><a class="footnote-ref" id="fnref:13" href="#fn:13"><sup>13</sup></a></li></ul> <p>If input data is in a non-standard text format, it can be imported as whole lines, without splitting into columns, and then columns extracted later with OpenRefine's tools. Archived and compressed files are supported (.zip, .tar.gz, .tgz, .tar.bz2, .gz, or .bz2) and Refine can download input files from a <a href="/facts/URL/pPD5g7Wq">URL</a>. To use web pages as input, it is possible to import a list of URLs and then invoke a URL fetch function. </p><p>Export is supported in following formats:<a class="footnote-ref" id="fnref:14" href="#fn:14"><sup>14</sup></a> </p> <ul><li><a href="/facts/Tab-separated_values/TRyPhWXD">TSV</a></li> <li><a href="/facts/Comma-separated_values/dv1pSR1r">CSV</a></li> <li><a href="/facts/Microsoft_Excel/Ezal6G75">Microsoft Excel</a></li> <li><a href="/facts/HTML_table/s4TKzjUP">HTML table</a></li> <li><a href="/facts/Google_Spreadsheets/bTTDJYFw">Google Spreadsheets</a></li> <li>Templating exporter: it is possible to define custom template for outputting data, for example as <a href="/facts/MediaWiki/MbkLUTor">MediaWiki</a> table.</li></ul> <p>Whole OpenRefine projects in native format can be exported as a <a href="/facts/Tar_(file_format)/DCJ37j3d">.tar.gz</a> archive. </p> <h2 id="development">Development</h2> <p>OpenRefine started life as Freebase Gridworks, developed by <a href="/facts/Metaweb/F08ejFfn">Metaweb</a> and has been available as open source since January 2010.<a class="footnote-ref" id="fnref:15" href="#fn:15"><sup>15</sup></a> On 16 July 2010, <a href="/facts/Google/GT9Sugza">Google</a> acquired Metaweb,<a class="footnote-ref" id="fnref:16" href="#fn:16"><sup>16</sup></a> the creators of <a href="/facts/Freebase_(database)/oT1kwnQ2">Freebase</a>, and on 10 November 2010 renamed Freebase Gridwords Google Refine, releasing version 2.0.<a class="footnote-ref" id="fnref:17" href="#fn:17"><sup>17</sup></a> On 2 October 2012, original author David Huynh announced that Google would soon stop its active support of Google Refine.<a class="footnote-ref" id="fnref:18" href="#fn:18"><sup>18</sup></a><a class="footnote-ref" id="fnref:19" href="#fn:19"><sup>19</sup></a><a class="footnote-ref" id="fnref:20" href="#fn:20"><sup>20</sup></a> Since then, the codebase has been in transition to an open source project named OpenRefine.<a class="footnote-ref" id="fnref:21" href="#fn:21"><sup>21</sup></a> </p> <h2 id="external-links">External links</h2> <ul><li><a href="http://openrefine.org/">Official website</a></li> <li><a href="https://commons.wikimedia.org/wiki/File:OpenRefine_Beginners_Tutorial_by_Emma_Carroll.webm#%7B%7Bint%3Alicense-header%7D%7D">OpenRefine Beginners Tutorial by Emma Carroll</a></li></ul> <h2 id="references">References</h2> <ol> <li id="fn:1"><p>"openrefine.github.com". openrefine.org. <a href="http://openrefine.org/" target="_blank">http://openrefine.org/</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></p></li> <li id="fn:2"><p>"Editing by transforming: Cell Editing wiki page from Refine documentation". Retrieved 18 April 2012. <a href="https://code.google.com/p/google-refine/wiki/CellEditing#Editing_by_Transforming" target="_blank">https://code.google.com/p/google-refine/wiki/CellEditing#Editing_by_Transforming</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></p></li> <li id="fn:3"><p>"Comparison with spreadsheet software: Cell Editing wiki page in Refine documentation". Retrieved 18 April 2012. <a href="https://code.google.com/p/google-refine/wiki/CellEditing#Comparison_with_Spreadsheets_Software" target="_blank">https://code.google.com/p/google-refine/wiki/CellEditing#Comparison_with_Spreadsheets_Software</a> <a href="#fnref:3" class="footnote-back-ref">↩</a></p></li> <li id="fn:4"><p>General Refine expression language OpenRefine/OpenRefine Wiki GitHub. Github.com (2013-04-03). Retrieved on 2013-08-16. <a href="https://github.com/OpenRefine/OpenRefine/wiki/General-Refine-Expression-Language" target="_blank">https://github.com/OpenRefine/OpenRefine/wiki/General-Refine-Expression-Language</a> <a href="#fnref:4" class="footnote-back-ref">↩</a></p></li> <li id="fn:5"><p>"Expressions: Refine documentation". Retrieved 18 April 2012. <a href="https://code.google.com/p/google-refine/wiki/DocumentationForUsers#Expressions" target="_blank">https://code.google.com/p/google-refine/wiki/DocumentationForUsers#Expressions</a> <a href="#fnref:5" class="footnote-back-ref">↩</a></p></li> <li id="fn:6"><p>"Screencast: Google Refine 2.0 - Introduction (1 of 3) - editing government data". YouTube. 19 July 2011. Retrieved 18 April 2012. <a href="https://www.youtube.com/watch?v=B70J_H_zAWM" target="_blank">https://www.youtube.com/watch?v=B70J_H_zAWM</a> <a href="#fnref:6" class="footnote-back-ref">↩</a></p></li> <li id="fn:7"><p>"Stripping HTML: Refine documentation wiki page". Retrieved 18 April 2012. <a href="https://code.google.com/p/google-refine/wiki/StrippingHTML" target="_blank">https://code.google.com/p/google-refine/wiki/StrippingHTML</a> <a href="#fnref:7" class="footnote-back-ref">↩</a></p></li> <li id="fn:8"><p>"FetchingURLsFromWebServices wiki page: Refine documentation". Retrieved 18 April 2012. <a href="https://code.google.com/p/google-refine/wiki/FetchingURLsFromWebServices" target="_blank">https://code.google.com/p/google-refine/wiki/FetchingURLsFromWebServices</a> <a href="#fnref:8" class="footnote-back-ref">↩</a></p></li> <li id="fn:9"><p>"Screencast: Google Refine 2.0 - Data Augmentation (3 of 3) - using Openstreetmap Nominatim for geocoding and Freebase for augmentation". YouTube. 19 July 2011. Retrieved 18 April 2012. <a href="https://www.youtube.com/watch?v=5tsyz3ibYzk" target="_blank">https://www.youtube.com/watch?v=5tsyz3ibYzk</a> <a href="#fnref:9" class="footnote-back-ref">↩</a></p></li> <li id="fn:10"><p>"Schema Alignment: Refine documentation wiki page". Retrieved 18 April 2012. <a href="https://code.google.com/p/google-refine/wiki/SchemaAlignment" target="_blank">https://code.google.com/p/google-refine/wiki/SchemaAlignment</a> <a href="#fnref:10" class="footnote-back-ref">↩</a></p></li> <li id="fn:11"><p>"OpenRefine documentation: Reconciliation". GitHub. Retrieved 12 March 2017. <a href="https://github.com/OpenRefine/OpenRefine/wiki/Reconciliation" target="_blank">https://github.com/OpenRefine/OpenRefine/wiki/Reconciliation</a> <a href="#fnref:11" class="footnote-back-ref">↩</a></p></li> <li id="fn:12"><p>"Importers: Refine documentation wiki page". Retrieved 18 April 2012. <a href="https://code.google.com/p/google-refine/wiki/Importers" target="_blank">https://code.google.com/p/google-refine/wiki/Importers</a> <a href="#fnref:12" class="footnote-back-ref">↩</a></p></li> <li id="fn:13"><p>"Changelog for 2.5". Retrieved 18 April 2012. <a href="https://code.google.com/p/google-refine/wiki/ChangesFor2p5" target="_blank">https://code.google.com/p/google-refine/wiki/ChangesFor2p5</a> <a href="#fnref:13" class="footnote-back-ref">↩</a></p></li> <li id="fn:14"><p>"Exporting: Refine documentation wiki page". Retrieved 18 April 2012. <a href="https://code.google.com/p/google-refine/wiki/Exporters" target="_blank">https://code.google.com/p/google-refine/wiki/Exporters</a> <a href="#fnref:14" class="footnote-back-ref">↩</a></p></li> <li id="fn:15"><p>"Google Code Archive - Long-term storage for Google Code Project Hosting". code.google.com. <a href="https://code.google.com/archive/p/google-refine/source" target="_blank">https://code.google.com/archive/p/google-refine/source</a> <a href="#fnref:15" class="footnote-back-ref">↩</a></p></li> <li id="fn:16"><p>"Google Official Blog: Deeper understanding with Metaweb". Retrieved 18 April 2012. <a href="http://googleblog.blogspot.com/2010/07/deeper-understanding-with-metaweb.html" target="_blank">http://googleblog.blogspot.com/2010/07/deeper-understanding-with-metaweb.html</a> <a href="#fnref:16" class="footnote-back-ref">↩</a></p></li> <li id="fn:17"><p>"Google Opensource blog: Announcing Google Refine 2.0, a power tool for data wranglers". Retrieved 18 April 2012. <a href="http://google-opensource.blogspot.com/2010/11/announcing-google-refine-20-power-tool.html" target="_blank">http://google-opensource.blogspot.com/2010/11/announcing-google-refine-20-power-tool.html</a> <a href="#fnref:17" class="footnote-back-ref">↩</a></p></li> <li id="fn:18"><p>"Google Groups". groups.google.com. <a href="https://groups.google.com/forum/#!topic/openrefine/a3R6afKb4-4" target="_blank">https://groups.google.com/forum/#!topic/openrefine/a3R6afKb4-4</a> <a href="#fnref:18" class="footnote-back-ref">↩</a></p></li> <li id="fn:19"><p>"From Freebase Gridworks to Google Refine and now OpenRefine". <a href="http://kb.refinepro.com/2012/10/from-freebase-gridworks-to-google.html" target="_blank">http://kb.refinepro.com/2012/10/from-freebase-gridworks-to-google.html</a> <a href="#fnref:19" class="footnote-back-ref">↩</a></p></li> <li id="fn:20"><p>OpenRefine Archived 2016-09-25 at the Wayback Machine. OpenRefine. Retrieved on 2013-08-16. <a href="http://openrefine.org/OpenRefine/community" target="_blank">http://openrefine.org/OpenRefine/community</a> <a href="#fnref:20" class="footnote-back-ref">↩</a></p></li> <li id="fn:21"><p>google-refine - Google Refine, a power tool for working with messy data (formerly Freebase Gridworks) - Google Project Hosting. Code.google.com. Retrieved on 2013-08-16. <a href="https://code.google.com/p/google-refine/" target="_blank">https://code.google.com/p/google-refine/</a> <a href="#fnref:21" class="footnote-back-ref">↩</a></p></li> </ol>