Create a Lambda function that polls an SQS queue
This pattern deploys deploys a Lambda function, an SQS queue and an EventBridge rule.
The Lambda function is used control how many jobs to submit to a downstream service for each incoming message in an Amazon SQS queue. We may need to control the number of concurrent job executions due to a business requirement or due to scalability limitation of the downstream service. We are controlling the max number of concurrent executions by using a SQS queue. An Amazon EventBridge rule triggers an AWS Lambda function every 2 minutes to pick messages from SQS queue, one by one, until until we hit the max executions of our downstream service.
This does not use the SQS trigger for Lambda because the purpose is to slow down the creation of new executions. Whereas the Amazon SQS trigger would push messages to the AWS Lambda function more quickly.
< Back to all patterns
Visit the GitHub repo for this pattern.
Download
git clone https://github.com/aws-samples/serverless-patterns/ cd serverless-patterns/sqs-lambda-eb-cdk-pythonDeploy
cdk deployTesting
Use the AWS CLI to send a message to the SQS queue and observe the event delivered to the Lambda function:
1. Delete the stack:
sam delete --stack-name STACK_NAME.1. Send several SQS messages by running this command multiple times:
aws sqs send-message --queue-url ENTER_YOUR_SQS_QUEUE_URL --message-body "Test message"2. Retrieve the logs from the Lambda function:
aws logs describe-log-streams --log-group-name '/aws/lambda/ENTER_YOUR_FUNCTION_NAME' --query logStreams[*].logStreamName.Cleanup
1. Delete the stack:
cdk destroy.2. Confirm the stack has been deleted:
aws cloudformation list-stacks --query "StackSummaries[?contains(StackName,'STACK_NAME')].StackStatus"