2

I am using python great expectation to validate my data using Apache spark. Basically I would like to add a flag is_valid against each row of the data frame. To add this flag, I need to apply multiple checks on each column with in the row. Great expectation is giving result for a column considering the whole rows. Is there any way I can achieve this using Great expectations?

sunitha
  • 1,468
  • 14
  • 18

1 Answers1

1

If you are trying to append to the dataframe you are validating with Great Expectations, currently this is not directly possible from within Great Expectations. You could add a step to your pipeline which would parse the Validation Result produced by your Checkpoint, and then append to your dataframe based on the results of that.

t. gl.
  • 31
  • 2