Stack Overflow
Stack Overflow

Questions tagged [kedro]

Kedro is an open source Python library that helps you build production-ready data and analytics pipelines

202 questions
0
votes
2 answers

Jupyter notebooks as Kedro node

How can I use a Jupyter Notebook as a node in Kedro pipeline? This is different from converting functions from Jupyter Notebooks into Kedro nodes. What I want to do is using the full notebook as the node.
MCK
  • 11
0
votes
1 answer

In Kedro, how to pick up intermediate dataset in a pipeline?

I'm working on my pipeline and testing it manually on jupyter notebook. Here is my situation. I wanted to pick up example_train and example_valid from it, so I wrote like this. context.pipeline.to_outputs("example_train", "example_valid") and…
Koichi MIYAMOTO
  • 53
  • 5
0
votes
1 answer

How to connect with Impala with Kedro?

How to connect with Impala with Kedro? Is there any integration with it? I am using a Window machine. I try Impyla and ibis and both doesn't work.
mediumnok
  • 180
  • 1
  • 9
0
votes
1 answer

In Kedro, How to specify layer to parameters.yml?

Currently, I'm using kedro and kedro-viz. I can specify a layer of dataset from catalog.yml. hoge: type: MemoryDataSet layer: raw but I don't know how to do it with parameters.yml step_size: 1 learning_rate: 0.01 if it can be done not in…
Koichi MIYAMOTO
  • 53
  • 5
0
votes
1 answer

using gunicorn for nested folders

I'm new to gunicorn and heroku so I would appreciate any help. I want to deploy my python Dash app on to heroku and I know I need a Procfile. The thing is that my project structure uses the Kedro structure and my structure looks like…
Daniel Leong
  • 13
  • 4
0
votes
1 answer

Using dictionary rather than parameter.yml for Kedro

Is there a way to use dictionary rather than using a yaml config for parameters.yml? I want to keep it as a Python Object because my IDE can then track the dependency easily. For my parameters, I am injecting functions in it. If i need to use yml, I…
mediumnok
  • 180
  • 1
  • 9
0
votes
1 answer

Kedro airflow on spark

Looking for kedro+ airflow implementation on spark. Is the plugin now available for spark ? Looked at PipelineX but couldn't find relevant examples on spark ?
Brilliant
  • 33
  • 1
  • 2
  • 7
0
votes
1 answer

How to organize inference results by model ID that produced it?

I am creating a deep ensemble, and I need to keep track of which inference results came from which neural network. However, enabling versioning via the Kedro catalog only organizes my inference results by datetime, with each file having an identical…
crypdick
  • 16,152
  • 7
  • 51
  • 74
0
votes
0 answers

kedro-airflow creates DAGs that throw errors

I am using kedro-airflow to create a DAG for airflow but the DAG created throws an error (see below). The flow is just a test flow - very simple - and it runs without errors with kedro run. Airflow also runs other DAGS without any problem. …
nmduarte
  • 61
  • 1
  • 6
0
votes
2 answers

Data versioning of "Hello_World" tutorial

i have added "versioned: true" in the "catalog.yml" file of the "hello_world" tutorial. example_iris_data: type: pandas.CSVDataSet filepath: data/01_raw/iris.csv versioned: true Then when I used "kedro run" to run the tutorial, it has error…
Y. huang
  • 53
  • 5
0
votes
0 answers

Running jupyter lab in kedro project in vscode under windows not possible

On our Windows 10 machines I tried to setup kedro projects setup on Ubuntu configured for VSCode. On Ubuntu they are working just fine. However if I run kedro jupyter lab in the VSCode integrated terminal on Windows I get the following error: [C…
thinwybk
  • 4,193
  • 2
  • 40
  • 76
0
votes
1 answer

kedro error: Pipeline does not contain nodes named ['preprocess_companies']

I was following the kedro pipelines tutorial1, create all needed files, started the kedro with kedro run --node=preprocess_companies It returns the following error ValueError: Pipeline does not contain nodes named ['preprocess_companies']. did try…
Andrea Jaeger
  • 1
  • 1
0
votes
1 answer

How to snap a python package with plugin packages?

I'd like to bundle the Python package kedro which provides a command line interface (kedro). In addition I'd like to put the Python package kedro-docker into the snap as well. This second package extends the first package's command line interface…
thinwybk
  • 4,193
  • 2
  • 40
  • 76
0
votes
2 answers

'kedro' is not recognized as an internal or external command, operable program or batch file

I am trying to install Kedro but I am getting this error. I know most of the time this error arises because kedro is not in my PATH. I tried adding the file path to my PATH and still getting the same error. When I run: pip show kedro output: Name:…
vincenzo Scotto Di Uccio
  • 25
  • 1
  • 9
0
votes
1 answer

Access Kedro context from decorator

I am trying to create a decorator in which I need some information about the project and/or catalog. Is it possible to access the project context from inside of the decorator? I am looking for things like project_name, catalog entry name, and…
Waylon Walker
  • 543
  • 3
  • 10
1 2 3
…
13
14