Amazon API Gateway to Amazon Kinesis Data Stream to AWS Lambda to Amazon DynamoDB

Amazon API Gateway → Kinesis Data Streams → AWS Lambda → Amazon DynamoDB

This pattern deploy an AWS SAM application with Amazon API Gateway, Amazon Kinesis Data Stream, AWS Lambda, and Amazon DynamoDB.

This pattern deploys an Amazon API Gateway REST API with route /prod/submit/{streamName}/{eventId} configured with basic authentication.
On receiving a request, API Gateway invokes a Lambda authorizer which validates the request by checking basic auth credentials from AWS Secrets Manager and returns a policy informing API Gateway to accept or deny the request.
When the request is accepted, API Gateway sends the message payload to Kinesis Data Stream. This pattern is useful in cases of large payloads since Kinesis Data Stream can support up to 1MB payload size.
Lambda process messages from the Kinesis Data Streams. Lambda uses a SQS queue as Dead Letter Queue to send the messages in case of continued failures to process the messages.
Lambda saves the received messages into a DynamoDB table.

< Back to all patterns

GitHub icon Download this pattern (.zip)

GitHub icon View this pattern on GitHub


Clone repo

git clone https://github.com/aws-samples/serverless-patterns/cd serverless-patterns/apigw-kinesis-lambda-ddb

Deploy

sam deploy


Testing

See the GitHub repo for detailed testing instructions.

Cleanup

Delete the stack: sam delete.

Created by:

Ravi Kiran Ganji

Ravi Kiran Ganji

I am a Senior Cloud Application Architect at AWS Professional Services, and Serverless Enthusiast.

Follow on LinkedIn