AWS Lambda function subscribed to an Amazon MSK topic with Avro and Schema Registry (Java)

AWS Lambda → Amazon MSK → AWS Lambda

Creates Lambda functions that produce and consume Avro-formatted messages to Amazon MSK topics using AWS Glue Schema Registry and IAM authentication.

This pattern provides two Lambda functions: a consumer that processes messages from a MSK topic, and a producer that generates Avro-formatted messages using AWS Glue Schema Registry.
There is a Amazon CloudFormation template provided in this pattern to deploys a MSK Cluster and create a Kafka topic.
The pattern works with either a Provisioned or Serverless MSK cluster as long as the cluster is configured to use IAM authentication.
This pattern demonstrates how to use Avro serialization with AWS Glue Schema Registry for efficient message handling in a Kafka-based architecture.

< Back to all patterns

GitHub icon Download this pattern (.zip)

GitHub icon View this pattern on GitHub


Clone repo

git clone https://github.com/aws-samples/serverless-patterns/cd serverless-patterns/msk-lambda-schema-avro-java-sam

Deploy

sam deploy --guided


Testing

See the GitHub repo for detailed testing instructions.

Cleanup

Delete the Lambda functions: sam delete.
Delete the CloudFormation stack that created the MSK cluster and EC2 client machine.

Created by:

Indranil Banerjee

Indranil Banerjee

AWS - Senior Solutions Architect

Follow on LinkedIn

Vaibhav Jain

Vaibhav Jain

AWS - Sr. Application Architect

Follow on LinkedIn

Adam Wagner

Adam Wagner

AWS - Principal Serverless Solutions Architect

Follow on LinkedIn

Philipp Page

Philipp Page

AWS - SA Engineer

Follow on LinkedIn

Leandro Cavalcante Damascena

Leandro Cavalcante Damascena

AWS - Sr. SA Engineer

Follow on LinkedIn