How to apply Partition on hive table which is already partitioned. I am not able to fetch the partitioned data into the folder after the data is loaded.
Asked
Active
Viewed 541 times
0
-
Please add more details like, 1. Is the old partition still existed. 2. Is the new partition done on existing columns, or any new columns added 3. How are you loading the data, once the new partition is created. These can decide the way the data is loaded into partitions. Look [here](https://cwiki.apache.org/confluence/display/Hive/Tutorial#Tutorial-Creating,Showing,Altering,andDroppingTables) for more information with respect to altering the tables, columns and partitions – Ramzy Jun 03 '15 at 21:57
2 Answers
0
1st rule of partitioning in hive is that the partitionioning column should be the last column in the data. since the data is already partitioned lets say we are partitioning data on gender M/F there will be two directories gender=M and gender=F be created inside each of the directories respective gender data will be available and last column again in this data will be gender.
If you want to partiton data again on partitioned table use insert into select and make sure last column you use is the partition column you want to on the partitioned data.
krishna.kadigari
- 77
- 1
- 5
0
Did you add a partition manually with the Hdfs command ? In that case metastore will not keep track of partitions being added unless you specify " alter table add partition "... try this
MSCK REPAIR TABLE table_name;
If that is not the case , then try to drop partitions and create the partitions again . Use alter table command to do this. but you will lose the data . and your partitioning column value should be mentioned as last column in case if you are doing a dynamic partition insert.
prasannads
- 609
- 2
- 14
- 28