Stack Overflow
Stack Overflow

Questions tagged [amazon-sqs]

Amazon Simple Queue Service (Amazon SQS) offers a reliable, highly scalable, hosted queue for storing messages as they travel between computers.

Resources

Tutorial

4134 questions
1
vote
1 answer

Do I need to delete duplicate SQS messages?

In a use case that precludes the use of SQS FIFO queues, duplicate messages are a possibility. If a consumer receives a duplicate message that it has already deleted, does it need to delete the duplicate? I'm guessing that the answer is no, since…
Dancrumb
  • 26,597
  • 10
  • 74
  • 130
1
vote
1 answer

Auto-scaling processors based on SQS, with different backlog SLOs

I have this problem where we may have different SLOs (service level objectives) based on request. Some requests we want to process within 5 minutes, and some requests we can take longer to process (like 2 hours etc). I was going to use Amazon SQS as…
de1337ed
  • 3,113
  • 12
  • 37
  • 55
1
vote
0 answers

Track completion status of batch of tasks in AWS SQS

I have queue where I publish multiple batches of tasks to same queue. Each batch is associated with a unique identifier. Is there a way to track completion status of the batch? I can think of attaching message group ID to each message of the batch…
Aditya Abhas
  • 174
  • 11
1
vote
0 answers

How to reset VisibilityTimeout for entire input AWS SQS queue if golang application is shutdown?

My golang-application uses aws-sdk-go to interact with an input queue. One message from the queue is processed about 7-10 min by a goroutine and then deleted. I would like to hide the message in the input queue setting VisibilityTimeout to 10min…
DisplayName
  • 219
  • 4
  • 23
1
vote
1 answer

Does Amazon SQS Extended Client Library support zipping of uploaded file in S3?

I am trying to figure out if Amazon SQS Extended Client Library supports zipping of uploaded file in S3. I Googled quite a lot but found no evidence of such support. The idea is to tell SQS in the message request to store in S3 as a Zip. So looking…
Jacobs2000
  • 856
  • 2
  • 15
  • 25
1
vote
0 answers

How to filter values in a status history panel with two queries in Grafana

I am currently building a dashboard in Grafana (9.0.6) and try to visualize "AWS Lambda Invocations" and "AWS Visible SQS Messages" with two queries (Metric Search) in a status history panel over the last 24 hours. Problem is that the SQS Metric has…
systrance
  • 37
  • 1
  • 6
1
vote
1 answer

How do I achieve one by one message delivery for each user individually (in parallel)?

I need a queue/message broker that would allow me to receive by 1 message for each user. Previously I've been using FIFO SQS and each user has its own message group id, which allows me to have 1 message inflight for each user. But I've ended up…
Vladlen Gladis
  • 1,699
  • 5
  • 19
  • 41
1
vote
1 answer

Why am I not able to read the request body when I construct a HTTP request from a dump?

I am trying to construct a HTTP request out of a SQS event within a Lambda using the code below. I am able to get the headers but the request body is empty. If the incoming request has a Content-Length header, then there is some content in the body…
Arvind
  • 33
  • 2
1
vote
0 answers

Amazon SQS Listener on Spring Boot calling REST endpoint fails with "Are you referring to request attributes outside an actual web request"

I have an Amazon SQS Listener configured in a Spring Boot application. After receiving a message of the queue, it has to call a REST API. The call fails with this error java.lang.IllegalStateException: No thread-bound request found: Are you…
Do Will
  • 711
  • 1
  • 9
  • 18
1
vote
0 answers

How can I ensure fixed TPS while triggering lambda with SQS?

I have a use case where I want to call a downstream API for all records in S3. The API has a rate limit of 250 TPS only, so I can call the API only 250 times in one second. Now, the records can arrive in S3 all at the same time (within few seconds).…
Let us see.
  • 81
  • 8
1
vote
0 answers

Spring Cloud AWS messaging with a circuit breaker

I would like some pointers/guide in how to configure a circuit breaker when working with "Spring Cloud AWS messaging" with an annotation driven approach: I'm using spring cloud AWS to work with an SQS and I wan't to use the "SqsListener"…
barakcaf
  • 1,294
  • 2
  • 16
  • 27
1
vote
0 answers

Is possible to make Step Functions wait for messages in SQS to be empty

In the Amazon SQS queue there are many of messages. Is it possible in the AWS Step Functions workflow receiving message from the SQS queues until empty then continue the workflow?
Chea Chandarith
  • 11
  • 2
1
vote
1 answer

Understanding lambda concurrency

I am trying to understand how a lambda triggered by SQS fifo queue handles load and concurrency. To test, I've created a lambda that publishes X number of messages to an SQS fifo queue (single MessageGroupId). FIFO queue has a batch size 1. The…
user1738539
  • 874
  • 2
  • 11
  • 23
1
vote
1 answer

Cannot parse SQS Json response

I have created a consumer to gather bounces and complaints in an SQS Queue. It is working for the majority of my messages, but there are some messages coming in that are throwing an Exception. The JSON output that…
DevOpsSauce
  • 1,319
  • 1
  • 20
  • 52
1
vote
0 answers

How to achieve dynamic fair processing between batch tasks?

Our use case is that our system supports scheduling multiple multi-channel send jobs at any time. Multi-channel send meaning send emails, send notifications, send sms etc. How it currently works is we have a SQS queue per channel. Whenever a job is…
uptoNoGood
  • 566
  • 5
  • 20
1 2 3
…
99
100