[{"data":1,"prerenderedAt":92},["ShallowReactive",2],{"content-guides-lambda-kafka-resources":3},{"markdown":4,"frontMatterAttributes":5,"bodyRaw":8,"contributorPaths":9,"menuType":11,"contentName":12,"slug":13,"readingTime":14,"menu":15,"nextDocItem":91,"previousDocItem":80},"\u003Ch3>Streaming\u003C\u002Fh3>\n\u003Cul>\n\u003Cli>\u003Ca href=\"https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Fcompute\u002Fbuilding-serverless-applications-with-streaming-data-part-1\u002F\">Building serverless applications with streaming data\u003C\u002Fa>\u003C\u002Fli>\n\u003Cli>\u003Ca href=\"https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Fbig-data\u002Fbest-practices-for-right-sizing-your-apache-kafka-clusters-to-optimize-performance-and-cost\u002F\">Best practices for right-sizing your Apache Kafka clusters to optimize performance and cost\u003C\u002Fa>\u003C\u002Fli>\n\u003Cli>\u003Ca href=\"https:\u002F\u002Fd1.awsstatic.com\u002Fserverless\u002FWhitepaper\u002FStream%20Processing%20Whitepaper.pdf\">Serverless Streaming Architectures and Best Practices\u003C\u002Fa>\u003C\u002Fli>\n\u003Cli>\u003Ca href=\"https:\u002F\u002Fwww.confluent.io\u002Fblog\u002Fserverless-kafka-streaming-with-confluent-cloud-and-aws-lambda\u002F\">Confluent: Trigger AWS Lambda Functions Directly from an Apache Kafka Topic\u003C\u002Fa>\u003C\u002Fli>\n\u003Cli>\u003Ca href=\"https:\u002F\u002Fdocs.confluent.io\u002Fplatform\u002Fcurrent\u002Fconnect\u002Findex.html\">Confluent: Kafka Connect\u003C\u002Fa>\u003C\u002Fli>\n\u003C\u002Ful>\n\u003Ch3>Self-managed Kafka\u003C\u002Fh3>\n\u003Cul>\n\u003Cli>\u003Ca href=\"https:\u002F\u002Fdocs.aws.amazon.com\u002Flambda\u002Flatest\u002Fdg\u002Fwith-kafka.html#services-kafka-scaling\">Using Lambda with self-managed Apache Kafka\u003C\u002Fa>\u003C\u002Fli>\n\u003Cli>\u003Ca href=\"https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Fcompute\u002Fusing-self-hosted-apache-kafka-as-an-event-source-for-aws-lambda\u002F\">Using self-hosted Apache Kafka as an event source for AWS Lambda\u003C\u002Fa>\u003C\u002Fli>\n\u003C\u002Ful>\n\u003Ch3>Amazon MSK\u003C\u002Fh3>\n\u003Cul>\n\u003Cli>\u003Ca href=\"https:\u002F\u002Fdocs.aws.amazon.com\u002Flambda\u002Flatest\u002Fdg\u002Fwith-msk.html#services-msk-vpc-config\">Using Lambda with Amazon MSK\u003C\u002Fa>\u003C\u002Fli>\n\u003Cli>\u003Ca href=\"https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Fcompute\u002Fusing-amazon-msk-as-an-event-source-for-aws-lambda\u002F\">Using Amazon MSK as an event source for AWS Lambda\u003C\u002Fa>\u003C\u002Fli>\n\u003Cli>\u003Ca href=\"https:\u002F\u002Faws.amazon.com\u002Fpt\u002Fblogs\u002Fcompute\u002Fcreating-a-serverless-apache-kafka-publisher-using-aws-lambda\">Creating a serverless Apache Kafka publisher using AWS Lambda\u003C\u002Fa>\u003C\u002Fli>\n\u003Cli>\u003Ca href=\"https:\u002F\u002Fcatalog.us-east-1.prod.workshops.aws\u002Fworkshops\u002Fc2b72b6f-666b-4596-b8bc-bafa5dcca741\u002Fen-US\">Amazon MSK Labs Workshop\u003C\u002Fa>\u003C\u002Fli>\n\u003Cli>\u003Ca href=\"https:\u002F\u002Fdocs.aws.amazon.com\u002Fmsk\u002Findex.html\">Amazon Managed Streaming for Apache Kafka Documentation\u003C\u002Fa>\u003C\u002Fli>\n\u003Cli>\u003Ca href=\"https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Fcompute\u002Fintroducing-aws-lambda-native-support-for-avro-and-protobuf-formatted-apache-kafka-events\u002F\">AWS Lambda native support for Avro and Protobuf formatted Apache Kafka events\u003C\u002Fa>\u003C\u002Fli>\n\u003Cli>\u003Ca href=\"https:\u002F\u002Fserverlessland.com\u002Fpatterns\u002Fmsk-lambda-schema-avro-java-sam\">Java Lambda Avro Kafka Event Sample Code\u003C\u002Fa>\u003C\u002Fli>\n\u003Cli>\u003Ca href=\"https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Fcompute\u002Fintroducing-provisioned-mode-for-kafka-event-source-mappings-with-aws-lambda\u002F\">Provisioned Mode for Kafka Event Source Mappings with AWS Lambda\u003C\u002Fa>\u003C\u002Fli>\n\u003C\u002Ful>\n\u003Ch3>Networking\u003C\u002Fh3>\n\u003Cul>\n\u003Cli>\u003Ca href=\"https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Fcompute\u002Fsetting-up-aws-lambda-with-an-apache-kafka-cluster-within-a-vpc\u002F\">Setting up AWS Lambda with an Apache Kafka cluster within a VPC\u003C\u002Fa>\u003C\u002Fli>\n\u003Cli>\u003Ca href=\"https:\u002F\u002Fdocs.aws.amazon.com\u002Flambda\u002Flatest\u002Fdg\u002Fconfiguration-vpc.html#vpc-internet\">Connecting outbound networking to resources in a VPC\u003C\u002Fa>\u003C\u002Fli>\n\u003Cli>\u003Ca href=\"https:\u002F\u002Frepost.aws\u002Fknowledge-center\u002Flambda-trigger-msk-kafka-cluster\">How do I troubleshoot Lambda triggers that poll from MSK and self-managed Kafka clusters?\u003C\u002Fa>\u003C\u002Fli>\n\u003Cli>\u003Ca href=\"https:\u002F\u002Fdocs.aws.amazon.com\u002Fwhitepapers\u002Flatest\u002Faws-privatelink\u002Fwhat-are-vpc-endpoints.html\">What are VPC endpoints?\u003C\u002Fa>\u003C\u002Fli>\n\u003Cli>\u003Ca href=\"https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Fbig-data\u002Fconnect-kafka-client-applications-securely-to-your-amazon-msk-cluster-from-different-vpcs-and-aws-accounts\u002F\">MSK Cross VPC connectivity\u003C\u002Fa>\u003C\u002Fli>\n\u003C\u002Ful>\n",{"title":6,"order":7},"Resources",9,"### Streaming\n\n- [Building serverless applications with streaming data](https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Fcompute\u002Fbuilding-serverless-applications-with-streaming-data-part-1\u002F)\n- [Best practices for right-sizing your Apache Kafka clusters to optimize performance and cost](https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Fbig-data\u002Fbest-practices-for-right-sizing-your-apache-kafka-clusters-to-optimize-performance-and-cost\u002F)\n- [Serverless Streaming Architectures and Best Practices](https:\u002F\u002Fd1.awsstatic.com\u002Fserverless\u002FWhitepaper\u002FStream%20Processing%20Whitepaper.pdf)\n- [Confluent: Trigger AWS Lambda Functions Directly from an Apache Kafka Topic](https:\u002F\u002Fwww.confluent.io\u002Fblog\u002Fserverless-kafka-streaming-with-confluent-cloud-and-aws-lambda\u002F)\n- [Confluent: Kafka Connect](https:\u002F\u002Fdocs.confluent.io\u002Fplatform\u002Fcurrent\u002Fconnect\u002Findex.html)\n\n### Self-managed Kafka\n\n- [Using Lambda with self-managed Apache Kafka](https:\u002F\u002Fdocs.aws.amazon.com\u002Flambda\u002Flatest\u002Fdg\u002Fwith-kafka.html#services-kafka-scaling)\n- [Using self-hosted Apache Kafka as an event source for AWS Lambda](https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Fcompute\u002Fusing-self-hosted-apache-kafka-as-an-event-source-for-aws-lambda\u002F)\n\n### Amazon MSK\n\n- [Using Lambda with Amazon MSK](https:\u002F\u002Fdocs.aws.amazon.com\u002Flambda\u002Flatest\u002Fdg\u002Fwith-msk.html#services-msk-vpc-config)\n- [Using Amazon MSK as an event source for AWS Lambda](https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Fcompute\u002Fusing-amazon-msk-as-an-event-source-for-aws-lambda\u002F)\n- [Creating a serverless Apache Kafka publisher using AWS Lambda](https:\u002F\u002Faws.amazon.com\u002Fpt\u002Fblogs\u002Fcompute\u002Fcreating-a-serverless-apache-kafka-publisher-using-aws-lambda)\n- [Amazon MSK Labs Workshop](https:\u002F\u002Fcatalog.us-east-1.prod.workshops.aws\u002Fworkshops\u002Fc2b72b6f-666b-4596-b8bc-bafa5dcca741\u002Fen-US)\n- [Amazon Managed Streaming for Apache Kafka Documentation](https:\u002F\u002Fdocs.aws.amazon.com\u002Fmsk\u002Findex.html)\n- [AWS Lambda native support for Avro and Protobuf formatted Apache Kafka events](https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Fcompute\u002Fintroducing-aws-lambda-native-support-for-avro-and-protobuf-formatted-apache-kafka-events\u002F)\n- [Java Lambda Avro Kafka Event Sample Code](https:\u002F\u002Fserverlessland.com\u002Fpatterns\u002Fmsk-lambda-schema-avro-java-sam)\n- [Provisioned Mode for Kafka Event Source Mappings with AWS Lambda](https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Fcompute\u002Fintroducing-provisioned-mode-for-kafka-event-source-mappings-with-aws-lambda\u002F)\n\n### Networking\n\n- [Setting up AWS Lambda with an Apache Kafka cluster within a VPC](https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Fcompute\u002Fsetting-up-aws-lambda-with-an-apache-kafka-cluster-within-a-vpc\u002F)\n- [Connecting outbound networking to resources in a VPC](https:\u002F\u002Fdocs.aws.amazon.com\u002Flambda\u002Flatest\u002Fdg\u002Fconfiguration-vpc.html#vpc-internet)\n- [How do I troubleshoot Lambda triggers that poll from MSK and self-managed Kafka clusters?](https:\u002F\u002Frepost.aws\u002Fknowledge-center\u002Flambda-trigger-msk-kafka-cluster)\n- [What are VPC endpoints?](https:\u002F\u002Fdocs.aws.amazon.com\u002Fwhitepapers\u002Flatest\u002Faws-privatelink\u002Fwhat-are-vpc-endpoints.html)\n- [MSK Cross VPC connectivity](https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Fbig-data\u002Fconnect-kafka-client-applications-securely-to-your-amazon-msk-cluster-from-different-vpcs-and-aws-accounts\u002F)",[10],"content\u002Fcontributors\u002Fjulian-wood.json","LESSON","Using AWS Lambda to process Apache Kafka streams","\u002Fcontent\u002Fguides\u002Flambda-kafka\u002Fresources","1 min",[16,25,34,42,51,59,68,77,86],{"title":17,"content":18},"Introducing streaming",[19],{"title":17,"order":20,"time":21,"path":22,"id":23,"link":24},1,"4 min","introduction","introduction.md","\u002Fcontent\u002Fguides\u002Flambda-kafka\u002Fintroduction",{"title":26,"content":27},"Using Kafka to build your streaming application",[28],{"title":26,"order":29,"time":30,"path":31,"id":32,"link":33},2,"6 min","using-kafka","using-kafka.md","\u002Fcontent\u002Fguides\u002Flambda-kafka\u002Fusing-kafka",{"title":35,"content":36},"Using Lambda to consume records from Kafka",[37],{"title":35,"order":38,"time":21,"path":39,"id":40,"link":41},3,"using-lambda","using-lambda.md","\u002Fcontent\u002Fguides\u002Flambda-kafka\u002Fusing-lambda",{"title":43,"content":44},"Configuring Kafka and Lambda",[45],{"title":43,"order":46,"time":47,"path":48,"id":49,"link":50},4,"9 min","configuring-kafka-lambda","configuring-kafka-lambda.md","\u002Fcontent\u002Fguides\u002Flambda-kafka\u002Fconfiguring-kafka-lambda",{"title":52,"content":53},"Processing Kafka streams using the Lambda ESM",[54],{"title":52,"order":55,"time":47,"path":56,"id":57,"link":58},5,"processing-esm","processing-esm.md","\u002Fcontent\u002Fguides\u002Flambda-kafka\u002Fprocessing-esm",{"title":60,"content":61},"Scaling and throughput",[62],{"title":60,"order":63,"time":64,"path":65,"id":66,"link":67},6,"5 min","scaling-throughput","scaling-throughput.md","\u002Fcontent\u002Fguides\u002Flambda-kafka\u002Fscaling-throughput",{"title":69,"content":70},"Monitoring and observability",[71],{"title":69,"order":72,"time":73,"path":74,"id":75,"link":76},7,"2 min","monitoring-observability","monitoring-observability.md","\u002Fcontent\u002Fguides\u002Flambda-kafka\u002Fmonitoring-observability",{"title":78,"content":79},"Troubleshooting",[80],{"title":78,"order":81,"time":82,"path":83,"id":84,"link":85},8,"3 min","troubleshooting","troubleshooting.md","\u002Fcontent\u002Fguides\u002Flambda-kafka\u002Ftroubleshooting",{"title":6,"content":87},[88],{"title":6,"order":7,"time":14,"path":89,"id":90,"link":13},"resources","resources.md",null,1792665264850]