Apache Hive is a database built on top of Hadoop and facilitates easy data summarization, ad-hoc queries, and the analysis of large datasets stored in Hadoop compatible distributed file system. Hive provides a mechanism to project structure onto this data and query the data using a SQL-like language called HiveQL. Please DO NOT use this tag for flutter database which is also named Hive, use flutter-hive tag instead.
Apache Hive is a database built on top of Hadoop that provides the following:
- Tools to enable easy data summarization (ETL)
- Ad-hoc querying and analysis of large datasets data stored in Hadoop file system (HDFS)
- A mechanism to put structure on this data
- An advanced query language called Hive Query Language which is based on SQL and some additional features such as DISTRIBUTE BY, TRANSFORM, and which enables users familiar with SQL to query this data.
At the same time, this language also allows traditional map/reduce programmers the ability to plug in their custom mappers and reducers to do more sophisticated analysis that may not be supported by the built-in capabilities of the language.
Since Hive is Hadoop-based, it does not and cannot promise low latencies on queries. The paradigm here is strictly of submitting jobs and being notified when the jobs are completed as opposed to real-time queries. In contrast to the systems such as Oracle where analysis is run on a significantly smaller amount of data, but the analysis proceeds much more iteratively with the response times between iterations being less than a few minutes, Hive queries response times for even the smallest jobs can be of the order of several minutes. However for larger jobs (e.g., jobs processing terabytes of data) in general they may run into hours and days. Many optimizations and improvements were made to spped-up processing such as fetch-only task, LLAP, materialized views, etc
To summarize, while low latency performance is not the top-priority of Hive's design principles, the following are Hive's key features:
- Scalability (scale out with more machines added dynamically to the Hadoop cluster)
- Extensibility (with map/reduce framework and UDF/UDAF/UDTF)
- Fault-tolerance
- Loose-coupling with its input formats
- Rather reach query kanguage with native suport for JSON, XML, regexp, possibility to call java methods, using python and shell transformations, analytics and windowing functions, possibility to connect to different RDBMS using JDBC drivers, Kafka connector.
- Ability to read and write almost any file formats using native and third-party SerDe, RegexSerDe.
- Numerous third-party extensions, for example brickhouse UDFs, etc
How to write good Hive question:
- Add clear textual problem description.
- Provide query and/or table DDL if applicable
- Provide exception message
- Provide input and desired output data example
- Questions about query performance should include EXPLAIN query output.
- Do not use pictures for SQL, DDL, DML, data examples, EXPLAIN output and exception messages.
- Use proper code and text formatting
Official links:
Useful Links:
