Luigi : Step by Step instructions not working

Question

I'm a newbie to python, I've installed Luigi-2.0.1 on my RHEL linux. Trying to run a sample program

import luigi

class MyTask(luigi.Task) :
        param = luigi.Parameter(default=42)

        def requires(self):
                return SomeOtherTask(self.param)

        def run(self):
                f = self.output.open('w')
                print >>f, 'hello world'
                f.close()

        def output(self):
                return luigi.LocalTarget('/tmp/foo/bar-%s.txt' % self.param)

if __name__ == '__main__':
        luigi.run()

Executed the following command

luigi --module maintask.py MyTask

I get the following error.

Traceback (most recent call last):
  File "/usr/bin/luigi", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 2655, in <module>
    working_set.require(__requires__)
  File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 648, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 546, in resolve
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: python-daemon<3.0

Upon investigating the web, I could not figure out why this happened. I have python 2.6. Is it that luigi is not compatible with 2.6 and only works for python 3.0 onwards?.

Appreciate any help

Forgot to mention, the program should throw an error for the missing SomeOtherTask(self.param) — venBigData, Dec 17 '15 at 17:50
the error tells you that there's a missing dependency, specifically `python-daemon` (but there might be other missing dependencies as well) — Andrea Corbellini, Dec 17 '15 at 18:09
Also you probably will need to use `self.output().open(...)` — Alexey Grigorev, Dec 18 '15 at 06:36
What is the name of the file that contains `MyTask`? [The docs](http://luigi.readthedocs.org/en/stable/command_line.html#running-from-the-command-line) suggest that you should run `luigi --module maintask MyTask` (i.e. no `.py`). — Alasdair, Dec 28 '15 at 23:39

score 5 · Answer 1 · answered Dec 17 '15 at 18:03

5

From Luigi's package page:

Luigi is a Python (2.7, 3.3, 3.4) package that helps you build complex pipelines of batch jobs.

You will have to upgrade from 2.6 if you want to use this.

answered Dec 17 '15 at 18:03

Kevin

74,910
12
133
166

1

Thank you all for your inputs. I have python upgraded to 3.5. Luigi seems to have got installed based on the following response. $ luigi No task specified – venBigData Dec 28 '15 at 20:48

Luigi : Step by Step instructions not working

1 Answers1