1

I want to partition my table in hive so that for every unique item in the row it creates a partition. There are ~250 partitions for about a 4 billion row table so I would like to to something like a for loop or a distinct. Here is my thoughts in code (which obviously have not worked)

ALTER TABLE myTable ADD IF NOT EXISTS

PARTITION( myColumn = distinct myColumn);

or is there some kind of loop in Hive? Does this require a UDF? A hive answer would be preferable if possible.

Thanks.

user1807096
  • 25
  • 1
  • 2
  • 7

1 Answers1

0

just use dynamic partitions
https://cwiki.apache.org/confluence/display/Hive/Tutorial#Tutorial-DynamicpartitionInsert

it does the partition creation on the go

dimamah
  • 2,883
  • 18
  • 31