0

I'm trying to do a clean exit from the program whenever my python as Hive UDF fails with an exception.
Here is an example:

SELECT TRANSFORM (id, name) USING 'D:\Python27\python.exe streaming.py' AS (id string, name string, count integer) FROM hivesampletable;

#streaming.py
import sys
from datetime import datetime

try:
   for line in sys.stdin.readlines(): 
       fields = line.strip().split('\t') 
       fields.append(len(name))
       print "\t".join(fields)
except:
   #I want program to break/clean exit with out printing (writing back) to table

Ideas appreciated

Alekhya Vemavarapu
  • 1,145
  • 1
  • 10
  • 26
  • 1
    Are you looking for the `pass` statement – Spade Apr 20 '16 at 05:18
  • Thank You, Can you please elaborate on that ? – Alekhya Vemavarapu Apr 20 '16 at 06:13
  • 1
    The `pass` statement lets is the equivalent of NOP or do nothing. One popular application of it is in except blocks where you want to ignore a certain kind of exception. It is not the best practice to not explicitly state your exception when catching it, but `pass` will let you silently restore control flow without interrupting. – Spade Apr 20 '16 at 06:17
  • But I want a clean exit with an exit code from the program, will pass help me do that ? – Alekhya Vemavarapu Apr 20 '16 at 06:20
  • I am not the exception you are having is from python. If it is, `pass` will help you do this. I am not a hive expert, but do you have to have the same number of columns in the TRANSFORM part and columns part of the query? – Spade Apr 20 '16 at 06:28
  • well, errors in the code is little immaterial for me now. I am just trying to know how can I do a clean exit from python/hive UDF. Pass helped me exit neatly but it doesn't allow me to read the exception unlike sys.exit(1). Thanks so much ! Post it in the answers so that users will benefit. – Alekhya Vemavarapu Apr 20 '16 at 06:32
  • use `sys.exc_info()` to see your exception or name it and print it as a `str` – Spade Apr 20 '16 at 06:40

2 Answers2

1

If you essentially want to "swallow" the exception, then I would recommend in the except block you explicitly call sys.exit(0), which will both exit the program and indicate (from a shell level) that the program is "OK".

e.g. You will end up with a a truly clean exit that even a shell, e.g. bash, will see as "success".

Note: If you want to exit without printing anything but allow a shell to know something went awry, pass a non-zero error code to exit.

Response to comment from OP:

Hmm, I wouldn't expect that, since you're explicitly swallowing the exception...

The next step would likely be to print out what the exception is, as was suggested in the other answer, and go from there, depending on what that exception is.

Another thing that may be contributing is I don't think your Python script matches your TRANSFORM statement, so that may be contributing to the issue.

Also, you are referencing name without that being initialized (that might be the exception here -- NameError: name 'name' is not defined).

khampson
  • 14,700
  • 4
  • 41
  • 43
  • It is still breaking with a big run time exception trace but not with a clean exit. "Error: java.lang.RuntimeException: Hive Runtime Error while closing operators" – Alekhya Vemavarapu Apr 20 '16 at 06:11
1

The pass statement will let you ignore the error and return control flow to your program.

except Exception, e:
    pass 
    #To see your exception
    print str(e)
    # Alternately, you could get details of your exception using
    print sys.exc_info()
Spade
  • 2,220
  • 1
  • 19
  • 29