1

Many users are used to using 'select * from tables' in oracle/mysql

But i should not allow them to query like that in hive

Is there any way to prevent a full_table scan in hive?

Like trigger or somethng else?

Thanks a lot!

leftjoin
  • 36,950
  • 8
  • 57
  • 116
james.peng
  • 373
  • 1
  • 3
  • 13

1 Answers1

1

To prevent full table scan use these settings. 

set hive.mapred.mode=strict;  --prevent full scan
SET hive.limit.query.max.table.partition=10; --limit partitions to 10

Hive will generate exception if partition predicate is not specified:

FAILED: SemanticException [Error 10041]: No partition predicate found

And exception example in case the number of partitions exceeded:

SemanticException Number of partitions scanned (=37) on table
leftjoin
  • 36,950
  • 8
  • 57
  • 116
  • Does `hive.mapred.mode` apply for Tez or Spark execution engine? Also, what if table is not partitioned? – OneCricketeer Feb 08 '18 at 13:37
  • @cricket_007 It is applicable for Tez as well. Do not know about spark. And if table is not partitioned then partition limitation will not work. – leftjoin Feb 08 '18 at 14:48