0

I have a Lambda function that connects to EMR using boto3. I want to run "aws s3 cp" command on EMR using my lambda function to copy files from S3 to EMR's local directory.

Is there a way to run aws cli commands on EMR using Lambda?

John Rotenstein
  • 241,921
  • 22
  • 380
  • 470
Rhiya
  • 271
  • 6
  • 21
  • Pasted the solution to copy from S3 to EMR in this post - https://stackoverflow.com/questions/56623774/copy-files-from-s3-to-emr-local-using-lambda/56761459#56761459 – Rhiya Jun 25 '19 at 20:22

1 Answers1

0

No.

Amazon Lambda runs from the Internet. It does not have access to run commands on the EMR cluster instances.

You could, theoretically, install the Systems Manager Agent on EMR. (I haven't tried, it, but it should work.) Your AWS Lambda function can then call the Systems Manager send_command() function to execute some code on the instance.

See: AWS Systems Manager Run Command - AWS Systems Manager

John Rotenstein
  • 241,921
  • 22
  • 380
  • 470