0

I'm trying to get the appropriate values in this list of dictionaries, which includes calling classes from 'table_name'. But I keep getting multiple errors, currently I'm seeing:

copy_VE_to_s3 = {group: S3CopyObjectsOperator( TypeError: unhashable type: 'dict'

data_groups = [
    {'name': 'commissions', 'dtype': 'snapshot_date', 'table_name': SkCommission},
    {'name': 'clicks', 'dtype': 'date', 'table_name': SkClicks},
    {'name': 'products', 'dtype': 'snapshot_date', 'table_name': SkProducts}
]

co_name = [
    'VE',
    'AB']

with dag:
    with TaskGroup(group_id='group1') as tg1:

        copy_VE_to_s3 = {group: S3CopyObjectsOperator(
            task_id=f'copy_{group["name"]}_data_to_VE_s3',
            partition=SkimlinksS3Partition(
                location_base=f'VE/{group["name"]}',
                obj=f'{group["name"]}',
            ),
            dest_prefix=f'VE/{group["name"]}/{group["dtype"]}={ds}'
        ) for group in data_groups}
KristiLuna
  • 1,601
  • 2
  • 18
  • 52

1 Answers1

0

You are trying to create a dictionary with dictionaries as keys. Apparently the group object is a dictionary. However, one requirement for dictionary keys is that they are hashable, which is not the case for dictionaries. This explains your error

Simon Hawe
  • 3,968
  • 6
  • 14
  • thank you! how would I fix this then and still loop through each value in my list of dictionaries? – KristiLuna Jan 18 '22 at 19:04
  • That depends on what copy_VE_to_s3 should contain. If you just want to store pairs of group and S3CopyObjectsOperator, you can do something like [(group, S3CopyObjectsOperator( task_id=f'copy_{group["name"]}_data_to_VE_s3', partition=SkimlinksS3Partition( location_base=f'VE/{group["name"]}', obj=f'{group["name"]}', ), dest_prefix=f'VE/{group["name"]}/{group["dtype"]}={ds}' )) for group in data_groups] But again, that depends on how you want to use the result. – Simon Hawe Jan 18 '22 at 19:07