How to Pass data between lambda functions in Step function?

Question

I have spend some time evaluating Step functions for a scheduling engine that we are trying to build. My initial understanding is that:

We will build the step function as a series of lambda functions which will be called in sequence. To trigger the step function to run once every night, I would create a CloudWatch event rule. I however have a couple of questions:

a. How would we change the output returned from a first lambda function to an input to a second lambda function. Can we do this in a "Pass" element? I see that the Pass element has ResultPath, Result fields but I am not able to understand how I would implement this if the output of the first lambda function is completely different from the input of the second lambda function. Or it is a recommended practice to write the different lambda functions (in a step function) such that they cannot be used anywhere else?

b. How does the community handle source control for step functions? It is recommended practice to check in the generated source code for the step functions in source control?

Answer 1

A) You can use Pass state however it is very limited (and you would get the same effect by using InputPath in Task state). It is possible to add additional Task state which convert execution state into input of next Task. Anyway I would suggest to design Lambda functions to be able to just read execution data as is (so it would just extract needed fields from it, ignore others and throw an error if required field is missing).

I won't say that this is recommended practice. In fact I haven't found any recommended practices on this. However that method works in my case and if any Task implementation (be it Lambda or Activity) is shared between multiple state machines then those state machines are similar enough that their data model is very similar (thus Lambda works for both state machines).

B) State machine structure is part of the project so it is versioned along with rest of the code. Using YAML in Cloud Formation templates its JSON can be easily embedded as multiline string. In my case it is also inside Fn::Sub so ARNs of resources (Lambda, Activities) can be automatically set on deployment.