Create a Lambda function that subscribes to an MSK topic
Lambda is a consumer application for your Kafka topic. It processes records from one or more partitions and sends the payload to the target function. Lambda continues to process batches until there are no more messages in the topic.
Lambda internally polls for new messages from the event source and then synchronously invokes the target Lambda function. Lambda reads the messages in batches and provides these to your function as an event payload. The maximum batch size is configurable. (The default is 100 messages.)
The CDK template creates a Lambda Function that uses a AWSLambdaMSKExecutionRole IAM role to poll a certain topic within a certain MSK Stream as defined as the Event source in the template.
< Back to all patterns
Clone repo
git clone https://github.com/aws-samples/serverless-patterns/cd serverless-patterns/msk-lambda-cdkDeploy
npx cdk deployTesting
1. Publish messages to the Amazon MSK topic. Follow the steps as outlined in Step 4 to create a client machine and publish messages to the MSK Topic.
2. Retrieve the logs from the Lambda function:
sam logs -n ENTER_YOUR_CONSUMER_FUNCTION_NAME.Cleanup
1. Delete the stack:
cdk destroy.2. Confirm the stack has been deleted:
aws cloudformation list-stacks --query "StackSummaries[?contains(StackName,'STACK_NAME')].StackStatus".Created by:
