2

I am working on an AWS Glue job where I have a function "some_function" that I want to apply on DynamicFrame dy_f, but I also want to pass an input param to some_function.

Map.apply(frame=products_combination, f=search)

where some_function's definition is:

some_function(record, k)

What I've tried so far: Works:

Map.apply(frame=products_combination, f=search) ##provided i'm not taking k as input in some_function def as well

What is giving error:

Map.apply(frame=products_combination, f=search(k=10))

This returns "TypeError: search() missing 1 required positional argument: 'record'"

How can I pass a parameter to Map.apply function? I have gone through the documentation [here] but couldn't find my solution there.1

MAC
  • 172
  • 1
  • 8

2 Answers2

3

You can always pass in additional parameters with lambda:

Map.apply(frame=products_combination, f=lambda record: search(record, k=10))
bzu
  • 1,242
  • 1
  • 8
  • 14
0

I eventually used Spark udf instead of DynamicFrame.Map.apply. Worked for me.

MAC
  • 172
  • 1
  • 8
  • This requires you to leave the dynamicframe guardrails. any resolution that doesn't require to you use a dataframe? – Brian Nov 23 '21 at 17:34