0

I only receive this error when I run the task without --local-scheduler, on OS X El Cap. Daemon created with luigid command is currently running as well.

When I run the task with --local-scheduler, it operates as expected. It also runs fine both with and without --local-scheduler on my Windows 7 vm. Curious as to why it only fails in that one case on OS X.

Code snippet comes from this post I was referencing begin learning Luigi:https://marcobonzanini.com/2015/10/24/building-data-pipelines-with-python-and-luigi/

Code:

import luigi


class PrintNumbers(luigi.Task):
    n = luigi.IntParameter()

    def requires(self):
        return []

    def output(self):
        return luigi.LocalTarget("numbers_up_to_{}.txt".format(self.n))

    def run(self):
        with self.output().open('w') as f:
            for i in range(1, self.n+1):
                f.write("{}\n".format(i))


if __name__ == '__main__':
    luigi.run()

The trace:

File "scrape.py", line 24, in luigi.run()

File "/Users/-/.virtualenvs/adwords/lib/python2.7/site-packages/luigi/interface.py", line 210, in run return _run(*args, **kwargs)['success']

File "/Users/-/.virtualenvs/adwords/lib/python2.7/site-packages/luigi/interface.py", line 238, in _run return _schedule_and_run([cp.get_task_obj()], worker_scheduler_factory)

File "/Users/-/.virtualenvs/adwords/lib/python2.7/site-packages/luigi/interface.py", line 194, in _schedule_and_run success &= worker.add(t, env_params.parallel_scheduling)

File "/Users/-/.virtualenvs/adwords/lib/python2.7/site-packages/luigi/worker.py", line 565, in add for next in self._add(item, is_complete):

File "/Users/-/.virtualenvs/adwords/lib/python2.7/site-packages/luigi/worker.py", line 682, in _add retry_policy_dict=_get_retry_policy_dict(task),

File "/Users/-/.virtualenvs/adwords/lib/python2.7/site-packages/luigi/worker.py", line 441, in _add_task self._scheduler.add_task(*args, **kwargs)

File "/Users/-/.virtualenvs/adwords/lib/python2.7/site-packages/luigi/scheduler.py", line 112, in rpc_func return self._request('/api/{}'.format(fn_name), actual_args, **request_args)

File "/Users/-/.virtualenvs/adwords/lib/python2.7/site-packages/luigi/rpc.py", line 145, in _request response = json.loads(page)["response"]

File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/init.py", line 338, in loads return _default_decoder.decode(s)

File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 366, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end())

File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 384, in raw_decode raise ValueError("No JSON object could be decoded") ValueError: No JSON object could be decoded

Michael J Caboose
  • 191
  • 1
  • 12
  • do you have an scheduler instance running? notice that the task you're running is trying to reach that scheduler: its default address is http://localhost:8082. If you're running -say- nginx it must be giving you a forbidden response (html), that's why your task fails to run. – matagus Oct 20 '16 at 11:51
  • Isn't that what the luigid command creates? If so, yes, I have the global scheduler running. I then load up the URL in chrome to look at the dashboard. – Michael J Caboose Oct 20 '16 at 13:45
  • yes, right ...then you should be able to look into luigid log files if there's any error. Also you may see the which endpoints are being requested in console output. – matagus Oct 20 '16 at 15:33
  • I have a theory...whenever I install luigi in a venv on OS X, I have an xcode install asking me to download `clang`. Since the firewall i'm behind wont allow me to install xcode or anything affiliated...I think that is my issue with the OS X side deamon not working, where on my windows VM I have pretty much free reign. – Michael J Caboose Oct 21 '16 at 15:19

0 Answers0