I'm writing a script that retroactively performs an ETL process we previously missed. The bit I'm having trouble with is this piece of code:
feed_string = java_home + '/java -Xmx6000m -jar ' + script_dir + '/Proj/Feeds.jar -o ' + feed + '_ad_hoc_load_' + date + '.csv -s FileSender -d ' + date + ' -f ' + feed + ' -r ' + script_dir + '/Feeds --config {root}/config || { echo \'' + feed + ' process FAILED...\' ; log $AUTO_JOB_NAME "Failure" 1 "' + feed + ' failed..."; exit 1; }'
print('Executing shell command:')
print(feed_string)
print('')
feed_string = str.split(feed_string, ' ')
proc = subprocess.Popen(feed_string)
proc.wait()
print proc.returncode
When I test this I use 3 dates for files; 2 of which exist, one of which doesn't. The first two dates run without error, transform the data and the Python code prints 0
in my standard out. The final date however does not have an accompanying file and as a result, my logs print out a stack trace saying java.io.FileNotFoundException: /path/to/file/file_20170908.data (No such file or directory)
- as expected.
The problem is that the Python code then prints out another 0
despite the fact it throws a FileNotFoundExecption
. Why is this?