5

I have setup a standalone hive-metastore(v3.0.0) backed by postgres and created external tables within spark sql. The external data location is in azure blob. I am able to query these tables by using dbname.tablename instead of the actual location. These are non partitioned tables.

When I check in the metastore postgres db,TBLS table, I can see the field TBL_TYPE set as EXTERNAL_TABLE and there is a key SD_ID which maps to the table SDS. The SDS table has a LOCATION field, which doesn't show the actual blob location. Instead it shows the database path with a PLACEHOLDER appended to it.

Location
file:/home/ash/mydatabase/youraddress4-__PLACEHOLDER__ 

The above local location doesn't even exist.

How does spark or hive metastore resolves the actual location of the tables and where is it actually stored in the metastore database?

bukli
  • 172
  • 2
  • 9

0 Answers0