Event-Driven Data Pipeline with AWS Lambda Durable Functions

Amazon SQS queue → AWS Lambda durable function

This serverless pattern demonstrates building an event-driven pipeline using AWS Lambda Durable Functions with direct SQS Event Source Mapping

This pattern demonstrates an event-driven data processing pipeline using AWS Lambda Durable Functions with direct SQS Event Source Mapping. When a message arrives in the SQS queue, it directly triggers the durable function (no intermediary Lambda needed). The durable function then orchestrates a series of specialized processing steps using Lambda invoke chaining, first validating the incoming data, then transforming it , and finally storing the processed results in DynamoDB. Throughout this process, the durable function automatically creates checkpoints, enabling fault-tolerant execution that can recover from failures without losing progress. The entire pipeline operates within the 15-minute ESM execution limit, making it ideal for reliable batch processing workflows.

< Back to all patterns

GitHub icon Download this pattern (.zip)

GitHub icon View this pattern on GitHub


Clone repo

git clone https://github.com/aws-samples/serverless-patterns/cd serverles-patterns/lambda-durable-esm-and-chaining

Deploy

sam buildsam deploy --guided


Testing

See the GitHub repo for detailed testing instructions.

Cleanup

Delete the stack: sam delete.

Created by:

Sahithi Ginjupalli

Sahithi Ginjupalli

Sahithi Ginjupalli is a Cloud Engineer with a passion for continuous learning and in-depth exploration of her field. She helps optimize client applications through strategic implementation of AWS services, enhancing their efficiency and performance.

Follow on LinkedIn