I used to work with Azure but I'm very very new on AWS. I do have a situation similar with this question . I have lots of json files on S3 bucket and in the files I have structs and arrays. I need to flatten both of them and keep it S3 again. End of the day I have to load data from S3 to Snowflake.
For now, I have used AWS Glue Crawler and it can recognize my json files' structure. But I couldn't any solution to flatten both structs and arrays at the same time. It has been one year after last comment on the question which I shared, maybe there is a new feature that I can use it and solve my situation easily. Any help/comment would be amazing. Thank you.
I have created the structure of json files via AWS Glue Crawler I tried to flatten via AWS Glue DataBrew but it doesn't work well on big files. It would be great if it works.
