Menu
Home
People
Places
Arts
History
Plants & Animals
Science
Life & Culture
Technology
Reference.org
Apache Impala
open-in-new
<h2 id="description">Description</h2> <p>Apache Impala is a query engine that runs on Apache Hadoop. The project was announced in October 2012 with a public <a href="/facts/Beta_test/z4Q4RElA">beta test</a> distribution<a class="footnote-ref" id="fnref:3" href="#fn:3"><sup>3</sup></a><a class="footnote-ref" id="fnref:4" href="#fn:4"><sup>4</sup></a> and became generally available in May 2013.<a class="footnote-ref" id="fnref:5" href="#fn:5"><sup>5</sup></a> </p><p>Impala brings scalable <a href="/facts/Parallel_database/AJLtqLf0">parallel database</a> technology to Hadoop, enabling users to issue low-latency <a href="/facts/SQL/UjQLJcYm">SQL</a> queries to data stored in <a href="/facts/HDFS/oUS1G2R7">HDFS</a> and <a href="/facts/Apache_HBase/D4XU1dqK">Apache HBase</a> without requiring data movement or transformation. Impala is integrated with Hadoop to use the same file and data formats, metadata, security and resource management frameworks used by <a href="/facts/MapReduce/KbfxJ5vV">MapReduce</a>, <a href="/facts/Apache_Hive/G3M79J6b">Apache Hive</a>, <a href="/facts/Apache_Pig/y3JVRF1k">Apache Pig</a> and other Hadoop software. </p><p>Impala is promoted for analysts and data scientists to perform analytics on data stored in Hadoop via SQL or <a href="/facts/Business_intelligence/ftThn7Wf">business intelligence</a> tools. The result is that large-scale data processing (via MapReduce) and interactive queries can be done on the same system using the same data and metadata – removing the need to migrate data sets into specialized systems and/or proprietary formats simply to perform analysis. </p><p>Features include: </p> <ul><li>Supports <a href="/facts/HDFS/oUS1G2R7">HDFS</a>, <a href="/facts/Amazon_S3/bqAXYCxq">S3</a>, <a href="/facts/Microsoft_Azure/CCnIHTHd">Microsoft Azure</a> Blob Storage, <a href="/facts/Apache_HBase/D4XU1dqK">Apache HBase</a> and <a href="/facts/Apache_Kudu/zSRaI68D">Apache Kudu</a> storage,</li> <li>Reads Hadoop file formats, including text, <a href="/facts/Lempel%25E2%2580%2593Ziv%25E2%2580%2593Oberhumer/hfS0lRrD">LZO</a>, SequenceFile, <a href="/facts/Apache_Avro/eTn7FdRH">Avro</a>, <a href="/facts/RCFile/caWBPMKx">RCFile</a>, <a href="/facts/Apache_Parquet/BuT8WYeR">Parquet</a> and <a href="/facts/Apache_ORC/INdFBRpm">ORC</a></li> <li>Supports Hadoop security (<a href="/facts/Kerberos_(protocol)/RMXs3P1w">Kerberos authentication</a>, <a href="/facts/Ldap/KgpfD8nR">Ldap</a>),</li> <li>Fine-grained, role-based authorization with Apache Ranger</li> <li>Uses metadata, <a href="/facts/ODBC/2dV2Fm5j">ODBC</a> driver, and SQL syntax from <a href="/facts/Apache_Hive/G3M79J6b">Apache Hive</a>.</li></ul> <p>In early 2013, a <a href="/facts/Column-oriented_DBMS/RHleA9BY">column-oriented file format</a> called <a href="/facts/Apache_Parquet/BuT8WYeR">Parquet</a> was announced for architectures including Impala.<a class="footnote-ref" id="fnref:6" href="#fn:6"><sup>6</sup></a> In December 2013, <a href="/facts/Amazon_Web_Services/UFVN2796">Amazon Web Services</a> announced support for Impala.<a class="footnote-ref" id="fnref:7" href="#fn:7"><sup>7</sup></a> In early 2014, <a href="/facts/MapR/ux6joHSa">MapR</a> added support for Impala.<a class="footnote-ref" id="fnref:8" href="#fn:8"><sup>8</sup></a> In 2015, another format called <a href="/facts/Apache_Kudu/zSRaI68D">Kudu</a> was announced, which <a href="/facts/Cloudera/lJK8e1YY">Cloudera</a> proposed to donate to the <a href="/facts/Apache_Software_Foundation/Xl9Tt9lE">Apache Software Foundation</a> along with Impala.<a class="footnote-ref" id="fnref:9" href="#fn:9"><sup>9</sup></a> Impala graduated to an Apache Top-Level Project (TLP) on 28 November 2017.<a class="footnote-ref" id="fnref:10" href="#fn:10"><sup>10</sup></a> </p> <h2 id="see-also">See also</h2> <ul><li><a href="/facts/Apache_Drill/nY96JM6L">Apache Drill</a> — similar open source project inspired by Dremel</li> <li><a href="/facts/Dremel_(software)/qapTdnLb">Dremel</a> — similar tool from Google</li> <li><a href="/facts/Trino_(SQL_query_engine)/7A007zEx">Trino</a> — open source SQL query engine created by the creators of Presto</li> <li><a href="/facts/Presto_(SQL_query_engine)/c1BlTXQS">Presto</a> — open source SQL query engine created by Facebook and supported by <a href="/facts/Teradata/sarnALYT">Teradata</a></li></ul> <h2 id="external-links">External links</h2> <ul><li><a href="https://impala.apache.org/index.html">Apache Impala</a> project website</li> <li><a href="https://github.com/apache/impala/">Impala GitHub</a> project source code</li></ul> <h2 id="references">References</h2> <ol> <li id="fn:1"><p>"Apache Impala". Retrieved 15 September 2017. <a href="https://impala.apache.org/" target="_blank">https://impala.apache.org/</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></p></li> <li id="fn:2"><p>Cade Metz (October 24, 2012). "Man Busts Out of Google, Rebuilds Top-Secret Query Machine". Wired Magazine. Retrieved October 10, 2016. <a href="https://www.wired.com/2012/10/cloudera-impala-hadoop/" target="_blank">https://www.wired.com/2012/10/cloudera-impala-hadoop/</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></p></li> <li id="fn:3"><p>Larry Digna (October 24, 2012). "Cloudera aims to bring real-time queries to Hadoop, big data". Between the lines blog. ZDNet. Retrieved January 20, 2014. <a href="https://www.zdnet.com/article/cloudera-aims-to-bring-real-time-queries-to-hadoop-big-data/" target="_blank">https://www.zdnet.com/article/cloudera-aims-to-bring-real-time-queries-to-hadoop-big-data/</a> <a href="#fnref:3" class="footnote-back-ref">↩</a></p></li> <li id="fn:4"><p>Andrew Brust (October 25, 2012). "Cloudera's Impala brings Hadoop to SQL and BI". ZDNet. Retrieved January 20, 2014. <a href="https://www.zdnet.com/article/clouderas-impala-brings-hadoop-to-sql-and-bi/" target="_blank">https://www.zdnet.com/article/clouderas-impala-brings-hadoop-to-sql-and-bi/</a> <a href="#fnref:4" class="footnote-back-ref">↩</a></p></li> <li id="fn:5"><p>Marcel Kornacker, Justin Erickson (May 1, 2013). "Cloudera Impala 1.0: It's Here, It's Real, It's Already the Standard for SQL on Hadoop". Archived from the original on April 13, 2014. Retrieved April 10, 2014. <a href="https://web.archive.org/web/20140413155045/http://blog.cloudera.com/blog/2013/05/cloudera-impala-1-0-its-here-its-real-its-already-the-standard-for-sql-on-hadoop/" target="_blank">https://web.archive.org/web/20140413155045/http://blog.cloudera.com/blog/2013/05/cloudera-impala-1-0-its-here-its-real-its-already-the-standard-for-sql-on-hadoop/</a> <a href="#fnref:5" class="footnote-back-ref">↩</a></p></li> <li id="fn:6"><p>"Parquet: Columnar Storage for Hadoop". Project web site. 2013. Retrieved January 20, 2014. <a href="http://parquet.io/" target="_blank">http://parquet.io/</a> <a href="#fnref:6" class="footnote-back-ref">↩</a></p></li> <li id="fn:7"><p>"Announcing Support for Impala with Amazon Elastic MapReduce". Amazon.com. December 12, 2013. Retrieved January 20, 2014. <a href="http://aws.amazon.com/about-aws/whats-new/2013/12/12/announcing-support-for-impala-with-amazon-elastic-mapreduce/" target="_blank">http://aws.amazon.com/about-aws/whats-new/2013/12/12/announcing-support-for-impala-with-amazon-elastic-mapreduce/</a> <a href="#fnref:7" class="footnote-back-ref">↩</a></p></li> <li id="fn:8"><p>"Impala for MapR". MapR.com. February 2, 2014. Retrieved April 10, 2014. <a href="http://doc.mapr.com/display/MapR/Impala+for+MapR" target="_blank">http://doc.mapr.com/display/MapR/Impala+for+MapR</a> <a href="#fnref:8" class="footnote-back-ref">↩</a></p></li> <li id="fn:9"><p>David Ramel (November 18, 2015). "Cloudera to Donate Impala and Kudu Big Data Projects to Apache". Application Development Trends. Retrieved October 10, 2016. <a href="https://adtmag.com/articles/2015/11/18/cloudera-donates-projects.aspx" target="_blank">https://adtmag.com/articles/2015/11/18/cloudera-donates-projects.aspx</a> <a href="#fnref:9" class="footnote-back-ref">↩</a></p></li> <li id="fn:10"><p>"The Apache Software Foundation Announces Apache Impala as a Top-Level Project". November 28, 2017. Retrieved November 30, 2017. <a href="https://blogs.apache.org/foundation/entry/the-apache-software-foundation-announces24" target="_blank">https://blogs.apache.org/foundation/entry/the-apache-software-foundation-announces24</a> <a href="#fnref:10" class="footnote-back-ref">↩</a></p></li> </ol>