resolve choice for Glue dataframe not working

Question

I have a Glue data frame with the following structure, due to some historical data we have differences in the structure. When I try to change the structure the resolveChoice is not working.

|-- logs: array
|    |-- element: struct
|    |    |-- ...
|    |    |-- target: struct
|    |    |    |-- ...
|    |    |    |-- details: struct
|    |    |    |    |-- ...
|    |    |    |    |-- reviews: struct
|    |    |    |    |    |-- comment: string
|    |    |    |    |    |-- generalReason: choice
|    |    |    |    |    |    |-- array
|    |    |    |    |    |    |    |-- element: array
|    |    |    |    |    |    |    |    |-- element: choice
|    |    |    |    |    |    |    |    |    |-- int
|    |    |    |    |    |    |    |    |    |-- string
|    |    |    |    |    |    |-- string
|    |    |    |    |    |    |-- struct
|    |    |    |    |    |    |    |-- deleted: long
|    |    |    |    |    |    |    |-- brandId: string
|    |    |    |    |    |    |    |-- brand: string
|    |    |    |    |    |    |    |-- name_JP: string
|    |    |    |    |    |    |    |-- status: boolean
|    |    |    |    |    |    |    |-- modifiedDate: string
|    |    |    |    |    |    |    |-- name_EN: string
|    |    |    |    |    |    |    |-- id: string
|    |    |-- ...
|-- SK: string
|-- PK: string

The code I am running:

case_logs_temp = case_logs.resolveChoice(
    specs=[
        ("logs.target.details.reviews.generalReason", "project:struct")
        ],
    transformation_ctx="case_logs_resolveChoice",
)

# case_logs.printSchema()
case_logs_temp.printSchema()

The printschema looks exactly like the one above. Any ideas how to resolve the issue?

score 0 · Answer 1 · answered Jun 30 '21 at 06:35

The following code works:

    specs=[
        ("logs[].target.details.reviews.generalReason", "project:struct")
        ],
    transformation_ctx="case_logs_resolveChoice",
)

Thanks to issue In AWS Glue, how do I apply resolveChoice to a struct element within an array in a DynamicFrame?

resolve choice for Glue dataframe not working

1 Answers1