I have a csv file with 8 columns. within the columns i purposely deleted some cells.
When i tried to run a Glue DataQuality job, for IsComplete, the result passed (which is not supposed to) for one column , but the rest of the columns failed as expected. Why is this behaviour shown?
Sample data:
Rule used in DQDL :
Rules = [
IsComplete "Gender",
Completeness "Gender" = 1,
IsComplete "Age",
IsComplete "Spending Score (1-100)",
IsComplete "annual income ($)",
IsComplete "Family Size",
IsComplete "work experience"
]
DQ outcome:
By right Rule 1 and Rule 2 should fail as well..
