How do I write a Dagster asset that depends on an earlier partition of itself?

Question

I was using depends_on_past with Airflow. I'm now using Dagster, with software-defined assets, and I was told that the way to get similar functionality is with build_asset_reconciliation_sensor and a daily-partitioned asset where each partition depends on past partitions of itself. How do I write a such an asset?

I tried creating a daily-partitioned asset, but I wasn't sure how to make it depend on earlier partitions of itself

score 1 · Answer 1 · answered Jan 27 '23 at 18:15

You can make an asset depend on earlier partitions of itself by using a TimeWindowPartitionMapping with a start_offset and end_offset:

from dagster import asset, AssetIn, DailyPartitionsDefinition, TimeWindowPartitionMapping, Nothing


@asset(
    partitions_def=DailyPartitionsDefinition(start_date="2020-01-01"),
    ins={
        "asset1": AssetIn(
            partition_mapping=TimeWindowPartitionMapping(start_offset=-1, end_offset=-1),
            dagster_type=Nothing,
        )
    },
)
def asset1() -> None:
    ...

How do I write a Dagster asset that depends on an earlier partition of itself?

1 Answers1