Here is my code:
from pysparkling import H2OConf #commenting this line makes it work
import logging
logging.basicConfig(filename='my_log.log',level=logging.INFO)
logging.info('test')
I cannot get the log file to get created, unless I comment the first line of the code. If I do that, then everything works as normal. I'm assuming it's an import inside H2OConf that messes up the handlers. Anybody experience anything similar?
I run the script with
spark-submit --master yarn script.py
Thanks
