1

Lets assume the below table:

as schema: ID,NAME,Country and my partition key is country.

If my query is like:

select * from table where id between 155555756 to 10000000000;

The partition will not work in that case, right? .

On a simple note .What if I do not use partition key in my query . So table full scan will be there, right?

Mayank Porwal
  • 33,470
  • 8
  • 37
  • 58
Varshini
  • 69
  • 10
  • Both answers are yes... You could run an explain plan on the query with and without the partition filter – OneCricketeer Oct 28 '18 at 17:16
  • Think of a partition like a folder in the hdfs hierarchy leading to your storage files. Because that's basically what it is. `hdfs dfs -ls ` should show you that. And you can use `SHOW CREATE TABLE ;` to determine the hdfs ``
     – Elliott Frisch Oct 28 '18 at 21:02

1 Answers1

1

Answer to your first question is yes, this query plan will not do partition pruning.

You can use the following statement to check if the query does partition pruning: explain dependency <your query>

Answer to your second question - It depends!

If the hive.mapred.mode is set to strict, then hive will not allow to do full table scans, and few other "risky" operations like cross joins etc.,

Depending on the version of hive you are using, these settings also affect the number of partitions that can be scanned by a single query

  • hive.metastore.limit.partition.request (or)
  • hive.limit.query.max.table.partition
Jahan Balasubramaniam
  • 360
  • 1
  • 8
  • 22