We have jobs moving data from one bucket to another bucket. The job is converting CSV to parquet. While moving glue is dropping records with most null column. Let say we have 50 column table and if 20 subsequent columns are empty or null then that record is dropped. I am not able understand this behavior. Please let me know if someone have see this behavior and have rectified it. Below order of the code.
datasource0 = applymapping1 = resolvechoice2 = datasink =
I don't have dropnullfields, I took it out because I suspected it. please shed some light Regards, Prakash
