Member-only story
Lambda SQS Triggers and Concurrency
Revisiting the limits of Lambda SQS triggers
AWS provides the capability to trigger a lambda function with incoming events in the SQS queue. Lambda polls the queue for the messages and invokes your function synchronously. Lambda picks these messages in batches and passes these as input to a function. When a function successfully processes the batch, it deletes the events from the queue.
If there is an error in processing the messages, the messages appear in the queue again after visibility timeout and can finally land in the dead-letter queue if the error keeps occurring.
Where’s the Issue?
When an SQS trigger is enabled, Lambda starts long polling with five parallel connections. Each connection picks a batch of messages from the SQS queue and passes it to a lambda function. If there are still messages in queue, Lambda increases the polling processes by up to 60 more instances every minute. The maximum number of instances can reach up to 1,000.
The problem is that this scaling of polling instances is not directly connected to your lambda function concurrency. So if your lambda function reaches its concurrency limit with increasing polling instances and can’t handle incoming requests, it will throttle the messages. The throttled messages will go back to the queue after the visibility timeout and can eventually end up in the dead-letter queue. This issue is likely to occur when Lambda’s concurrency is set to a low value like 1 to 30.
Reproducing the Issue
To understand it better, I did a small POC in which I created a lambda with concurrency set as 5. The task of my lambda was to take an input and then sleep for five seconds. Here’s the code for the lambda function:
