1

I am getting started with Kedro, so I created the new kedro project for default iris dataset.

I am able to succesfully run it with kedro run command. My question now is how do I run it as a python command? From the documentation I read that the command kedro run runs the src/project-name/run.py. However, if I run the run.py I get ModuleNotFoundError: No module named 'iris_workflow'. I get the same error if I run the run method from src/project-name/cli.py.

Everything works fine If I run kedro run in terminal.

How do I run kedro run from a python script without subprocess.run(). If I import the run.py or cli.py in a script and run it, I get the same error ModuleNotFoundError: No module named 'iris_workflow'.

This is the default workflow I created with kedro new --starter=pandas-iris

BlueMango
  • 463
  • 7
  • 21

1 Answers1

3

The problem is that your src/ folder which is where your project python package lives isn't on your Python path, so if you modify your PYTHONPATH first, you'll be able to run run.py:

~/code/kedro/test-project
test-project ❯ PYTHONPATH=$PYTHONPATH:$pwd/src python3 src/test_project/run.py

To more concretely answer your question, if you wanted to run Kedro from a Python script, you'd do something like this:

import sys

sys.path.append("<path-to-your-project-src")
with KedroSession.create(package_path.name) as session:
    session.run()
Zain Patel
  • 983
  • 5
  • 14
  • Thank you! If I add `src/` folder to path with `sys.path.append`. I get `ValueError: Given configuration path either does not exist or is not a valid directory: ..... iris\src\iris_workflow\conf\base`. It's true because `conf` is not inside `src`, they are at same level. If I had that folder to path then I again get `ModuleNotFoundError` – BlueMango Mar 06 '21 at 16:01
  • It now looks like this: \n package_path = Path(\__file\__).resolve().parent \n sys.path.append(str(package_path.parent)) \n with KedroSession.create(str(package_path.name)) as session: \n session.run() – BlueMango Mar 06 '21 at 16:05
  • 1
    Make sure that you run your script whilst at the project root, not anywhere else. – Zain Patel Mar 07 '21 at 00:36
  • Thank you. Is there a way to run this from other folder? I want to run kedro project from a script in "python way" from outside the project root. What is the best way to do it? – BlueMango Mar 10 '21 at 09:34
  • 1
    That's fine, as long as you move your `conf/` folder to sit alongside your working directory. – Zain Patel Mar 10 '21 at 11:08