0

I am struggling on a great_expectations integration problem.
I obviously use RunGreatExpectationsValidation task with:

validation_task = RunGreatExpectationsValidation()
with Flow(
    "GE_pull_and_run",
) as GE_pull_and_run_flow:
    .......
    validation_task(
        context_root_dir=root_dir,
        checkpoint_name=expectation_checkpoint_name
    )

When I run the command on GE (great_expectations --V3-api checkpoint run my_checkpoint), it works, but on prefect task, I have an exception: With GE V3 api:

    .....
   File "c:\Users\vincent2\DK\prefect.data.pipeline\venv\lib\site- 
   packages\prefect\tasks\great_expectations\checkpoints.py", line 246, in run
        for batch in ge_checkpoint["batches"]:
    TypeError: 'Checkpoint' object is not subscriptable

The same with GE V2 api

    ...
        for batch in ge_checkpoint["batches"]:
    TypeError: 'LegacyCheckpoint' object is not subscriptable

Great_expectations=0.13.43 (also tried with 0.12.10 version)
prefect=0.15.9
Anyone experienced this pb? Thanks

vBob
  • 3
  • 1
  • 1
    Community Manager for Great Expectations here... There's currently an open issue for this here: https://github.com/great-expectations/great_expectations/issues/3609 Our community working on making the Prefect operator work with V3 right now. And we'll be updating the progress of the necessary work in that issue if you'd like to follow it. It will involve a PR on Prefect and Great Expectations, which we'll link to once we have them. – Kyle Eaton Dec 02 '21 at 19:49

1 Answers1

0

To provide an update on this: the issue has been fixed as part of this PR in Prefect. Feel free to give it a try now and if something still doesn't work for you, let us know.

Anna Geller
  • 1,653
  • 7
  • 10