18

I'm trying to get a toy example up and running with an AWS Lambda function, f, that's triggered by a message on one SQS queue, sqs, publishes to another queue sqs', and then a worker, f' reads from sqs' and processes the message where the entire "request" is traced with X-Ray.

sqs -> f -> sqs' -> f'

Currently, I have the queues in place and the functions writing and receiving from the queue. I also have X-Ray tracing the request from the the first function f to the sqs queue.

My current challenge is: how do I propagate the trace to the final worker so I can see the entire process in x-ray.


Here are my current functions:

public class Hello implements RequestHandler<SQSEvent, Void> {
    String OUTPUT_QUEUE_URL = "...";

    private AmazonSQS sqs = AmazonSQSClientBuilder.standard()
        .withRequestHandlers(new TracingHandler(AWSXRay.getGlobalRecorder()))
        .build();

    public Void handleRequest(SQSEvent event, Context context)
    {
        for(SQSMessage msg : event.getRecords()){
            System.out.println(new String(msg.getBody()));
        }

        SendMessageRequest send_msg_request = new SendMessageRequest()
            .withQueueUrl(OUTPUT_QUEUE_URL)
            .withMessageBody("hello world")
            .withDelaySeconds(5);
        sqs.sendMessage(send_msg_request);
        return null;
    }
}

public class World implements RequestHandler<SQSEvent, Void>{
    public Void handleRequest(SQSEvent event, Context context)
    {
        for(SQSMessage msg : event.getRecords()){
            System.out.println(new String(msg.getBody()));
        }
        return null;
    }
}
Justin Bell
  • 800
  • 7
  • 20
  • 2
    have exactly the same problem with a node.js lambda. SQS seems to be treated the same way as API Gateway (shows up as "User" in the X-Ray view) – WrRaThY Oct 03 '18 at 22:25

4 Answers4

6

Even though SQS supports X-Ray tracing now, it doesn't propagate the trace to a lambda function. That's why with SQS->Lambda setup, Lambda always starts a new trace. There is an issue for it in the nodejs xray SDK https://github.com/aws/aws-xray-sdk-node/issues/208

Mahendra R
  • 101
  • 2
  • 5
3

As of today, AWS X-Ray now has native support for Amazon SQS: https://aws.amazon.com/about-aws/whats-new/2019/08/aws-x-ray-now-supports-amazon-sqs/

matwer1
  • 161
  • 1
  • 7
  • 2
    @justin-bell did this solve your problem? I feel like I have the exact same setup but Lambda is always starting its own trace. – Graham Lea Nov 12 '19 at 02:16
2

Although as Matthew Werber rightly says this is not officially supported at the moment, you can work around this by creating a new AWS-Xray segment inside the Lambda Function and using the incoming TraceID from the SQS message. This will result in two Segments for your lambda invocation. One which Lambda itself creates, and one which you create to extend the existing trace. Whether that's acceptable or not for your use case is something you'll have to decide for yourself!

If you're working with Python you can do it with aws-xray-lambda-segment-shim.
If you're working with NodeJS you can follow this guide on dev.to.
If you're working with .NET there are some examples on this GitHub issue.

Sam Martin
  • 1,238
  • 10
  • 19
  • Hey Sam, thanks for pointing to this blog post. Here is a repo I created based on the suggested fix: https://github.com/renatoargh/lambda-sqs-lambda-xray-poc It uses serverless so just running "npx sls deploy" is enough to deploy the entire POC setup to AWS – Renato Gama Mar 09 '22 at 16:20
0

I'm planing to build the same prototype and I noticed the following announcement (date 2022-11-21): https://aws.amazon.com/about-aws/whats-new/2022/11/aws-x-ray-trace-linking-event-driven-applications-amazon-sqs-lambda/

It seems the propagation is supported now.

MarcC
  • 413
  • 3
  • 12