How to add a data quality check utility in awe glue.?

Question

How to add a job that just checks for data quality like null, correct data type etc in aws glue

score 1 · Answer 1 · answered May 11 '21 at 15:47

1

you can check pyDeequ library too. It enables you to implement python code to DQ checks.

answered May 11 '21 at 15:47

Udit Vardhan

score -1 · Answer 2 · answered Sep 10 '19 at 17:22

-1

You can write pyspark glue job for quality check. You can also use AWS Deequ library to do it but it is in scala.

answered Sep 10 '19 at 17:22

Sandeep Fatangare

2 Answers2