[{"data":1,"prerenderedAt":91},["ShallowReactive",2],{"content-guides-lambda-kafka-troubleshooting":3},{"markdown":4,"frontMatterAttributes":5,"bodyRaw":8,"contributorPaths":9,"menuType":11,"contentName":12,"slug":13,"readingTime":14,"menu":15,"nextDocItem":85,"previousDocItem":71},"\u003Ch4>Network security issues\u003C\u002Fh4>\n\u003Cp>If an ESM's networking, authentication, or authorization settings prevent communication with the cluster, then setup fails before it can invoke a function. The trigger shows an error record that can help troubleshoot the root cause.\u003C\u002Fp>\n\u003Cp>When the ESM sends a request to the broker endpoints and doesn't receive a response, the ESM considers the request as timed out. When a timeout to the broker endpoint occurs, the trigger displays the following error message:\u003C\u002Fp>\n\u003Cblockquote>\n\u003Cp>\u003Cstrong>"PROBLEM: Connection error. Please check your event source connection configuration. If your event source lives in a VPC, try setting up a new Lambda function or EC2 instance with the same VPC, Subnet, and Security Group settings. Connect the new device to the Kafka cluster and consume messages to ensure that the issue is not related to VPC or Endpoint configuration. If the new device is able to consume messages, please contact Lambda customer support for further investigation."\u003C\u002Fstrong>\u003C\u002Fp>\n\u003C\u002Fblockquote>\n\u003Cp>Follow the steps in the error message. Also, check the networking configurations to be sure that your ESM is properly configured.\u003C\u002Fp>\n\u003Ch4>Initialization, polling, and invocation errors\u003C\u002Fh4>\n\u003Cp>You receive the following message if your ESM is configured in a VPC but cannot call the STS API or Secrets Manager API (if required). You also receive the message if the ESM can access the Kafka cluster but can't invoke the function using the Lambda API.\u003C\u002Fp>\n\u003Cblockquote>\n\u003Cp>\u003Cstrong>"PROBLEM: Connection error. Your VPC must be able to connect to Lambda and STS, as well as Secrets Manager if authentication is required. You can provide access by configuring PrivateLink or a NAT Gateway."\u003C\u002Fstrong>\u003C\u002Fp>\n\u003C\u002Fblockquote>\n\u003Cp>Check your VPC settings that prevent your ESM from reaching other services like AWS STS and AWS Secrets Manager. Follow the steps in \u003Ca href=\"https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Fcompute\u002Fsetting-up-aws-lambda-with-an-apache-kafka-cluster-within-a-vpc\u002F\">Setting up AWS Lambda with an Apache Kafka cluster within a VPC\u003C\u002Fa> to properly configure your VPC settings.\u003C\u002Fp>\n\u003Cp>If calls to the STS API fail or timeout, then your VPC settings prevent your ESM from reaching the Regional Lambda endpoint on port 443. To resolve this issue, see \u003Ca href=\"https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Fcompute\u002Fsetting-up-aws-lambda-with-an-apache-kafka-cluster-within-a-vpc\u002F\">Setting up AWS Lambda with an Apache Kafka cluster within a VPC\u003C\u002Fa>.\u003C\u002Fp>\n\u003Cp>If  \u003Cstrong>SourceAccessConfiguration\u003C\u002Fstrong>  contains a secret, then be sure to retrieve that secret from Secrets Manager.\u003C\u002Fp>\n\u003Cp>For additional troubleshooting steps, see \u003Ca href=\"https:\u002F\u002Frepost.aws\u002Fknowledge-center\u002Flambda-trigger-msk-kafka-cluster\">Troubleshoot Lambda triggers with MSK and Kafka clusters\u003C\u002Fa>.\u003C\u002Fp>\n\u003Ch4>Amazon MSK consumer group rebalancing\u003C\u002Fh4>\n\u003Cp>If your Amazon MSK consumer group is rebalancing continuously, see the following \u003Ca href=\"https:\u002F\u002Frepost.aws\u002Fknowledge-center\u002Fmsk-consumer-group-rebalance\">troubleshooting steps\u003C\u002Fa>.\u003C\u002Fp>\n\u003Ch4>SASL\u002FSCRAM authentication\u003C\u002Fh4>\n\u003Cp>To troubleshoot issues when using Amazon MSK with SASL\u002FSCRAM authentication, see the following \u003Ca href=\"https:\u002F\u002Frepost.aws\u002Fknowledge-center\u002Fmsk-sasl-scram-issues\">troubleshooting steps\u003C\u002Fa>.\u003C\u002Fp>\n\u003Ch4>Lambda ESM consumer cannot join existing consumer group\u003C\u002Fh4>\n\u003Cp>ESM consumers cannot join a consumer group if the existing consumers are using a different consumer strategy than the ESM consumers. Ensure that all consumers in the group are using the same consumer strategy\u003C\u002Fp>\n",{"title":6,"order":7},"Troubleshooting",8,"#### Network security issues\n\nIf an ESM's networking, authentication, or authorization settings prevent communication with the cluster, then setup fails before it can invoke a function. The trigger shows an error record that can help troubleshoot the root cause.\n\nWhen the ESM sends a request to the broker endpoints and doesn't receive a response, the ESM considers the request as timed out. When a timeout to the broker endpoint occurs, the trigger displays the following error message:\n\n> **\"PROBLEM: Connection error. Please check your event source connection configuration. If your event source lives in a VPC, try setting up a new Lambda function or EC2 instance with the same VPC, Subnet, and Security Group settings. Connect the new device to the Kafka cluster and consume messages to ensure that the issue is not related to VPC or Endpoint configuration. If the new device is able to consume messages, please contact Lambda customer support for further investigation.\"**\n\nFollow the steps in the error message. Also, check the networking configurations to be sure that your ESM is properly configured.\n\n#### Initialization, polling, and invocation errors\n\nYou receive the following message if your ESM is configured in a VPC but cannot call the STS API or Secrets Manager API (if required). You also receive the message if the ESM can access the Kafka cluster but can't invoke the function using the Lambda API.\n\n> **\"PROBLEM: Connection error. Your VPC must be able to connect to Lambda and STS, as well as Secrets Manager if authentication is required. You can provide access by configuring PrivateLink or a NAT Gateway.\"**\n\nCheck your VPC settings that prevent your ESM from reaching other services like AWS STS and AWS Secrets Manager. Follow the steps in [Setting up AWS Lambda with an Apache Kafka cluster within a VPC](https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Fcompute\u002Fsetting-up-aws-lambda-with-an-apache-kafka-cluster-within-a-vpc\u002F) to properly configure your VPC settings.\n\nIf calls to the STS API fail or timeout, then your VPC settings prevent your ESM from reaching the Regional Lambda endpoint on port 443. To resolve this issue, see [Setting up AWS Lambda with an Apache Kafka cluster within a VPC](https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Fcompute\u002Fsetting-up-aws-lambda-with-an-apache-kafka-cluster-within-a-vpc\u002F).\n\nIf  **SourceAccessConfiguration**  contains a secret, then be sure to retrieve that secret from Secrets Manager.\n\nFor additional troubleshooting steps, see [Troubleshoot Lambda triggers with MSK and Kafka clusters](https:\u002F\u002Frepost.aws\u002Fknowledge-center\u002Flambda-trigger-msk-kafka-cluster).\n\n#### Amazon MSK consumer group rebalancing\n\nIf your Amazon MSK consumer group is rebalancing continuously, see the following [troubleshooting steps](https:\u002F\u002Frepost.aws\u002Fknowledge-center\u002Fmsk-consumer-group-rebalance).\n\n#### SASL\u002FSCRAM authentication\n\nTo troubleshoot issues when using Amazon MSK with SASL\u002FSCRAM authentication, see the following [troubleshooting steps](https:\u002F\u002Frepost.aws\u002Fknowledge-center\u002Fmsk-sasl-scram-issues).\n\n#### Lambda ESM consumer cannot join existing consumer group\n\nESM consumers cannot join a consumer group if the existing consumers are using a different consumer strategy than the ESM consumers. Ensure that all consumers in the group are using the same consumer strategy\n",[10],"content\u002Fcontributors\u002Fjulian-wood.json","LESSON","Using AWS Lambda to process Apache Kafka streams","\u002Fcontent\u002Fguides\u002Flambda-kafka\u002Ftroubleshooting","3 min",[16,25,34,42,51,59,68,77,82],{"title":17,"content":18},"Introducing streaming",[19],{"title":17,"order":20,"time":21,"path":22,"id":23,"link":24},1,"4 min","introduction","introduction.md","\u002Fcontent\u002Fguides\u002Flambda-kafka\u002Fintroduction",{"title":26,"content":27},"Using Kafka to build your streaming application",[28],{"title":26,"order":29,"time":30,"path":31,"id":32,"link":33},2,"6 min","using-kafka","using-kafka.md","\u002Fcontent\u002Fguides\u002Flambda-kafka\u002Fusing-kafka",{"title":35,"content":36},"Using Lambda to consume records from Kafka",[37],{"title":35,"order":38,"time":21,"path":39,"id":40,"link":41},3,"using-lambda","using-lambda.md","\u002Fcontent\u002Fguides\u002Flambda-kafka\u002Fusing-lambda",{"title":43,"content":44},"Configuring Kafka and Lambda",[45],{"title":43,"order":46,"time":47,"path":48,"id":49,"link":50},4,"9 min","configuring-kafka-lambda","configuring-kafka-lambda.md","\u002Fcontent\u002Fguides\u002Flambda-kafka\u002Fconfiguring-kafka-lambda",{"title":52,"content":53},"Processing Kafka streams using the Lambda ESM",[54],{"title":52,"order":55,"time":47,"path":56,"id":57,"link":58},5,"processing-esm","processing-esm.md","\u002Fcontent\u002Fguides\u002Flambda-kafka\u002Fprocessing-esm",{"title":60,"content":61},"Scaling and throughput",[62],{"title":60,"order":63,"time":64,"path":65,"id":66,"link":67},6,"5 min","scaling-throughput","scaling-throughput.md","\u002Fcontent\u002Fguides\u002Flambda-kafka\u002Fscaling-throughput",{"title":69,"content":70},"Monitoring and observability",[71],{"title":69,"order":72,"time":73,"path":74,"id":75,"link":76},7,"2 min","monitoring-observability","monitoring-observability.md","\u002Fcontent\u002Fguides\u002Flambda-kafka\u002Fmonitoring-observability",{"title":6,"content":78},[79],{"title":6,"order":7,"time":14,"path":80,"id":81,"link":13},"troubleshooting","troubleshooting.md",{"title":83,"content":84},"Resources",[85],{"title":83,"order":86,"time":87,"path":88,"id":89,"link":90},9,"1 min","resources","resources.md","\u002Fcontent\u002Fguides\u002Flambda-kafka\u002Fresources",1792578866099]