0

I am now developping system whose job is to consume data from kafka and put it into hive.Since the table have a partition "day" , so , the location of this partition on hdfs will be /root/tableLocation/day=20161110/adfadfaaf.avro .

But, this location cannot meet my requirement.I want to change this location to /root/tableLocation/20161110/adfadfaaf.avro .

The API I am using is apache hive metastore.Demo code when I create table is like this:

Table table = new Table(database, tableName);
table.setTableType(TableType.EXTERNAL_TABLE);
table.getParameters().put("EXTERNAL", "TRUE");
String tablePath = FileUtils.hiveDirectoryName(url, topicsDir, tableName);
table.setDataLocation(new Path(tablePath));
table.setSerializationLib(avroSerde);
try {
  table.setInputFormatClass(avroInputFormat);
  table.setOutputFormatClass(avroOutputFormat);
} catch (HiveException e) {
  throw new HiveMetaStoreException("Cannot find input/output format:", e);
}
List<FieldSchema> columns = HiveSchemaConverter.convertSchema(schema);
table.setFields(columns);
table.setPartCols(partitioner.partitionFields());

from the code , I can set the table location , but my question is , is there a way to set the partition location?

Ortomala Lokni
  • 56,620
  • 24
  • 188
  • 240
wuchang
  • 3,003
  • 8
  • 42
  • 66

1 Answers1

0

in hive and hdfs, partitions are essentially sub directories under the table directory. The way hive recognizes a partition is by a structure such as, <col name>=<partition value> If you change that sub directory name to just <partition value>, it will not be recognized by hive as a partition.

ameet chaubal
  • 1,440
  • 16
  • 37