1

Is there a way to know when a comprehend asynchronous job finishes?

For example:

I need to read the output of a sentiment analysis job when it finishes, but not by blocking the program and waiting. I think there should be a way to trigger a lambda function or something similar.

Note that when the job finishes it writes the results to s3_bucket in a file called output.tar.gz

Thanks.

John Rotenstein
  • 241,921
  • 22
  • 380
  • 470
Osama Abuhamdan
  • 159
  • 8

2 Answers2

3

You can trigger a lambda function when the file is created.

Configuring Amazon S3 Event Notifications has general information on triggering events from S3.

Configuring Notifications with Object Key Name Filtering shows how to use the filtering to match an object key.

Jason Wadsworth
  • 8,059
  • 19
  • 32
  • I have done this but the lambda functions didn't trigger, I made it trigger on all object creation on my bucket. Prefix and suffix was put properly. Any other suggestions? – Osama Abuhamdan Mar 03 '20 at 23:42
  • If you're having issues getting that working I would post a question about that. It's a known solution (I have triggers on S3 buckets), so if it's not working it's not because the solution isn't valid, it's because you have something configured incorrectly. – Jason Wadsworth Mar 04 '20 at 01:07
  • Thanks a lot! Another question, is creating the file happens after the job finishes? in other words, does the lambda get triggered when the job finishes? or in some other stages of the sentiment analysis job? – Osama Abuhamdan Mar 04 '20 at 19:49
  • Lamda would be triggered when the file appears in S3, because you're using event notifications as the trigger. – alex067 Mar 04 '20 at 20:53
  • @alex067 And this event happen after the job ends? – Osama Abuhamdan Mar 05 '20 at 09:36
  • No. The event will trigger WHEN S3 detects the file is in the bucket, which sends a notification to lambda which causes lambda to run. – alex067 Mar 05 '20 at 16:01
  • Tested it and everything worked as expected! @JasonWadsworth Thanks a lot! – Osama Abuhamdan Mar 07 '20 at 14:15
  • @alex067 and this happens 1 second (in my test case) after the file created (which means the result of the job is ready) – Osama Abuhamdan Mar 08 '20 at 08:52
2

I'm not sure how creating a lambda function would help. Are you trying to view the analysis job's logging / print contents?

Perhaps you could maybe create a script that checks every so often for the output.tar.gz file in s3, to know the job is finished.

alex067
  • 3,159
  • 1
  • 11
  • 17
  • Lambda creation is my trial to do what needed. I was looking for another solution than you suggest, if there wasn't, I will consider it. Thanks. – Osama Abuhamdan Mar 03 '20 at 23:44