[{"data":1,"prerenderedAt":63},["ShallowReactive",2],{"pattern-smk-lambda-queue-mode-python-sam":3,"pattern-group-smk-lambda-queue-mode-python-sam":62},{"id":4,"title":5,"architectureURL":6,"cleanup":7,"contributors":10,"deploy":13,"description":16,"extension":17,"framework":18,"gitHub":19,"highlight":6,"introBox":23,"language":29,"level":30,"meta":31,"patternArch":32,"patternGroup":6,"patternGroupLabel":6,"resources":46,"stem":57,"testing":58,"videoId":6,"__hash__":61},"patterns\u002Fpatterns\u002Fsmk-lambda-queue-mode-python-sam.json","Self-managed Apache Kafka Queue mode to AWS Lambda (KIP-932)",null,{"text":8},[9],"Delete the ESM first: \u003Ccode>aws lambda delete-event-source-mapping --uuid <esm-uuid>\u003C\u002Fcode>, then delete stacks in reverse order: \u003Ccode>kafka-queue-observability\u003C\u002Fcode>, \u003Ccode>kafka-queue-app\u003C\u002Fcode>, \u003Ccode>kafka-queue-broker\u003C\u002Fcode>, \u003Ccode>kafka-queue-network\u003C\u002Fcode>.",[11,12],"content\u002Fcontributors\u002Fvaibhav-jain.json","content\u002Fcontributors\u002Fhardith-murari.json",{"text":14},[15],"See README.md for full deployment instructions including three paths: full deployment, bring-your-own Kafka, and bring-your-own Kafka and VPC.","Lambda ESM for self-managed Kafka 4.2+ using Queue mode (KIP-932 Share Groups). Multiple pollers process the same partition concurrently, breaking the partition-count ceiling.","json","AWS SAM",{"template":20},{"patternPath":21,"templateFile":22},"smk-lambda-queue-mode\u002Fpython\u002Fsam","stacks\u002F3-app.yaml",{"headline":24,"text":25},"How it works",[26,27,28],"With Queue consumption mode (ConsumptionMode: Queue), Lambda event pollers use Kafka Share Groups (KIP-932) instead of traditional consumer groups. Multiple pollers can read from the same partition concurrently — parallelism is decoupled from partition count.","This pattern deploys: a dedicated VPC with private subnets and VPC endpoints, a self-managed Kafka 4.2+ broker on EC2 with share groups enabled, a Python Lambda worker function with partial batch response, a producer Lambda for testing, and CloudWatch observability.","Deployment is split into four independent stacks so you can skip the network and broker stacks if you have existing infrastructure.","Python","300",{},{"icon1":33,"icon2":38,"line1":42},{"x":34,"y":35,"service":36,"label":37},20,50,"msk","Self-managed Apache Kafka",{"x":39,"y":35,"service":40,"label":41},80,"lambda-event-functions","AWS Lambda",{"from":43,"to":44,"label":45},"icon1","icon2","Queue mode enabled",{"bullets":47},[48,51,54],{"text":49,"link":50},"Using Lambda with self-managed Apache Kafka","https:\u002F\u002Fdocs.aws.amazon.com\u002Flambda\u002Flatest\u002Fdg\u002Fwith-kafka.html",{"text":52,"link":53},"Apache Kafka event poller scaling modes in Lambda","https:\u002F\u002Fdocs.aws.amazon.com\u002Flambda\u002Flatest\u002Fdg\u002Fkafka-scaling-modes.html",{"text":55,"link":56},"Configuring error handling controls for Kafka event sources","https:\u002F\u002Fdocs.aws.amazon.com\u002Flambda\u002Flatest\u002Fdg\u002Fkafka-retry-configurations.html","patterns\u002Fsmk-lambda-queue-mode-python-sam",{"text":59},[60],"See the GitHub repo for detailed testing instructions.","pOWwc0_9XlsyzJyUXM_3grY3F7tQ8ybMhCG7KrKddJ0",[],1793356471530]