2

I've a redshift DB setup and we do periodic archival of the data into S3. I would like to create redshift external tables on top of these archived files. AWS documentation suggests that this can be done either via athena or via hive metastore. Since athena is quite expensive, I would like to get this done via Hive metastore. But I'm struggling with the connectivity here.

Below are the links of the steps that I followed: https://docs.aws.amazon.com/redshift/latest/dg/r_CREATE_EXTERNAL_SCHEMA.html https://docs.aws.amazon.com/redshift/latest/dg/r_CREATE_EXTERNAL_TABLE.html

Creating the external schema works out fine; but while creating the table i get the follow error:

Invalid operation: Hive Metastore error. HOST: XX.XXX.XXX.XX PORT: 9083 ERROR: Default TException.

Any idea what can be done here?

skrisshnaswamy
  • 149
  • 1
  • 9
  • 1
    what about Spectrum? you don't need Athena and Hive, just data on S3 and pay per use. The docs you referenced describe Spectrum external schema creation – AlexYes Jan 16 '18 at 11:40
  • Right now, I don't know the pattern (volume) of our data usage. Thats why we decided to go with something like Hive metastore. But the pricing of spectrum doesn't look that bad. Thanks for the suggestion. Will check it out. – skrisshnaswamy Jan 16 '18 at 12:19
  • 1
    you can read this fresh article https://blog.modeanalytics.com/scaling-event-tables-with-redshift-spectrum/ – AlexYes Jan 16 '18 at 12:57
  • Hey.. Did you figure this out? I'm having the exact same problem. – pallupz May 22 '19 at 15:52

0 Answers0