How can I tell Prefect, to not rerun all previous tasks in DAG?
Asked
Active
Viewed 190 times
1 Answers
0
Targets will get you where you want to go. From the docs:
Targets in Prefect are templatable location strings that are used to check for the existence of a task Result. This is useful in cases where you might want a task to only write data to a location once or merely not rerun if some piece of data is present. If a result exists at that location then the task run will enter a cached state.
In your task decorator just add a target like so:
@task(result=LocalResult(), target="{task_name}-{today}")
George Coyne
- 21
- 2