Great expectations saving custom queries

Question

Creating a batch and using a custom query does not save the query in the json file

For example:

batch_kwargs = {'data_asset_name': 'pgsql.data_asset_name',
            'query': 'select entity_id,attribute  from table_name where attribute > 50 and',
            'datasource': 'datasource',
            'schema': 'public',
            'table': 'table_name',
            }


expectation_suite_name = "test.error"

suite = context.create_expectation_suite(
    expectation_suite_name, overwrite_existing=True  # Configure these parameters for your needs)
batch = context.get_batch(batch_kwargs, suite)

batch.expect_column_values_to_not_be_null('attribute', result_format={'result_format': 'COMPLETE','include_unexpected_rows': True})                                                                           
batch.save_expectation_suite(discard_failed_expectations=False)

will only yield a json containing the expectations and not the batch_kwargs.

Is there a configuration to save the query per se?

score 0 · Answer 1 · answered Mar 01 '22 at 12:11

One way to achieve this is to run 'great_expectations init' in terminal, choose what fits your case and when prompted with this qestion:

'You have selected a datasource that is a SQL database. How would you like to specify the data?'

Select 2.Enter a custom SQL query

Then the query will be saved to the .json

Great expectations saving custom queries

1 Answers1