AWS Data Pipeline: setting local variable in shell command

Question

I am trying to make use of the uuid library within a shell command invoked by an AWS data pipeline. It seems like the uuid function works fine, but when I try to pass this value to a variable, the data is lost.

A snippet of my testing script below:

sudo yum -y install uuid-devel
myExportId=$(uuid)
echo 'myExportId:' $myExportId
uuid

When I look at the activity log for the pipeline I see that the uuid function seems to be working, but the variable does not seem to contain anything?

myExportId:
b6cf791a-1d5e-11e6-a581-122c089c2e25

I notice this same behavior with other local variables in my scripts. Am I expressing these incorrectly?

My pipeline parameters are working fine within the script, so no issues there.

score 0 · Accepted Answer · answered May 19 '16 at 19:15

Shortly after posting this I realized that I had encoded some of the above steps in subshells within my pipeline definition. It can be difficult debugging shell scripts embedded within JSON, so I think I will move on to using the scriptUri parameter pointing to a bash file.

AWS Data Pipeline: setting local variable in shell command

1 Answers1