0

I am using Impyla and Python in the CDSW to query data in HDFS and use it. The problem is sometimes to get all of the data I have to go in and manually click on the "Invalidate all metadata and rebuild index" button in HUE.

Is there a way to do this in the workbench with a library or python code?

sectechguy
  • 2,037
  • 4
  • 28
  • 61

1 Answers1

2

I assume you are using something like this to connect to impala via impyla ... try executing the invalidate metadata <table_name> command

from impala.dbapi import connect
conn = connect(host='my.host.com', port=21050)
cursor = conn.cursor()
cursor.execute('INVALIDATE METADATA mytable') # run this
cursor.execute('SELECT * FROM mytable LIMIT 100')
print cursor.description  # prints the result set's schema
results = cursor.fetchall()
thePurplePython
  • 2,621
  • 1
  • 13
  • 34
  • This is a great answer. I knew that there was an 'INVALIDATE METADATA' command but I thought you could only use it from the command line. Thank you very much @thePurplePython – sectechguy Aug 02 '19 at 11:20