Fan out across Amazon S3 items using Distributed Map, invoking a Lambda durable function per item with per-item idempotency.
This pattern uses AWS Step Functions Distributed Map to read items from an Amazon S3 JSON file and fan out processing across Lambda durable functions.
Each item is processed by a durable function invoked via the AWS SDK service integration (CallAwsService targeting lambda:invoke).
The DurableExecutionName parameter is derived from each item's ID, providing per-item idempotency across re-runs.
The durable function executes a three-step workflow: validate the item, wait for rate limiting, and update the catalog entry.
< Back to all patterns
Clone repo
git clone https://github.com/aws-samples/serverless-patterns/cd serverless-patterns/cdk-sfn-dmap-dfDeploy
Clone the repository: <code>git clone https://github.com/aws-samples/serverless-patterns</code>Change directory: <code>cd cdk-sfn-dmap-df</code>Install dependencies: <code>npm install</code>Run tests: <code>npm test</code>Deploy the CDK stack: <code>cdk deploy</code>Testing
Run local tests:
npm test — validates CDK stack assertions and runs the durable function locally using LocalDurableTestRunner (no AWS deployment needed).Start the state machine:
aws stepfunctions start-execution --state-machine-arn <StateMachineArn from stack output> --name "catalog-update-$(date +%s)"Monitor the execution in the AWS Step Functions console to see all 50 child executions.
Re-run with the same input to demonstrate idempotency — durable functions return cached results.
Cleanup
Delete the stack:
cdk destroyCreated by:
