[{"data":1,"prerenderedAt":567},["ShallowReactive",2],{"content-svc-lambda-guides-aws-lambda-operator-guide-troubleshooting-walkthrough":3},{"markdown":4,"frontMatterAttributes":5,"bodyRaw":8,"menu":9,"menuType":563,"isCollapseableMenu":12,"nextDocItem":468,"previousDocItem":458,"contributorPaths":564,"contentName":565,"slug":467,"readingTime":566},"\u003Cp>In the \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Faws-samples\u002Faws-lambda-operators-guide%5BCoffee\">https:\u002F\u002Fgithub.com\u002Faws-samples\u002Faws-lambda-operators-guide[Coffee\u003C\u002Fa> Lookup example] in the companion GitHub repository, users have provided feedback that the front-end application is slow and frequently showing errors. This section walks through the process of reproducing and diagnosing the cause of the problem.\u003C\u002Fp>\n\u003Cp>\u003Cstrong>Topics\u003C\u002Fstrong>\u003C\u002Fp>\n\u003Cul>\n\u003Cli>\u003Ca href=\"#troubleshooting-walkthrough-isolating-and-resolving-issues\">Troubleshooting walkthrough: isolating and resolving issues\u003C\u002Fa>\u003Cul>\n\u003Cli>\u003Ca href=\"#reproducing-the-problem\">Reproducing the problem\u003C\u002Fa>\u003C\u002Fli>\n\u003Cli>\u003Ca href=\"#isolating-the-cause-of-errors\">Isolating the cause of errors\u003C\u002Fa>\u003C\u002Fli>\n\u003Cli>\u003Ca href=\"#isolating-the-performance-issue\">Isolating the performance issue\u003C\u002Fa>\u003C\u002Fli>\n\u003Cli>\u003Ca href=\"#measuring-and-improving-the-performance-of-outliers\">Measuring and improving the performance of outliers\u003C\u002Fa>\u003C\u002Fli>\n\u003C\u002Ful>\n\u003C\u002Fli>\n\u003C\u002Ful>\n\u003Ch2>Reproducing the problem\u003C\u002Fh2>\n\u003Cp>The first step is to reproduce the problem. Invoking the API using curl or Postman shows that the API is operating but does not highlight any errors. By running a load test against the backend API with \u003Ca href=\"https:\u002F\u002Fartillery.io\u002F\">Artillery\u003C\u002Fa>, simulating the level of load in production, I can reproduce both issues.\u003C\u002Fp>\n\u003Cp>In a test with 1500 requests over 5 minutes, 25% of the requests are failing with 502 errors and the overall performance is highly variable. While the median response time is 522 ms, which is worse than expected, the p95 response times are over 5.7 seconds. The Artillery report breaks down these findings further:\u003C\u002Fp>\n\u003Cp>\u003Cimg src=\"\u002Fassets\u002Fexternal\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fassets\u002Fimages\u002Fmonitoring-observability-figure-16.png\" alt=\"\">\u003C\u002Fp>\n\u003Cp>To learn more about load testing serverless APIs, see \u003Ca href=\"https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Fcompute\u002Fload-testing-a-web-applications-serverless-backend\u002F\">https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Fcompute\u002Fload-testing-a-web-applications-serverless-backend\u002F\u003C\u002Fa>.\u003C\u002Fp>\n\u003Ch2>Isolating the cause of errors\u003C\u002Fh2>\n\u003Cp>While the Artillery load test reports round-trip timings for API calls, it’s important to know the portion of latency caused by the backend application. I use X-Ray to investigate latency between the various AWS services used in the application. This helps to exclude networking delays or issues that are local to the client device. After a load test, I can check X-Ray or \u003Ca href=\"https:\u002F\u002Fdocs.aws.amazon.com\u002FAmazonCloudWatch\u002Flatest\u002Fmonitoring\u002FServiceLens.html\">ServiceLens\u003C\u002Fa> to verify the roundtrip performance once the requests are received by the API.\u003C\u002Fp>\n\u003Cp>\u003Cimg src=\"\u002Fassets\u002Fexternal\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fassets\u002Fimages\u002Fmonitoring-observability-figure-17.png\" alt=\"\">\u003C\u002Fp>\n\u003Cp>The service map shows service latency and failures in about 25% of the requests. X-Ray allows you to drill into requests by status code to further identify the causes of 502 errors. In this case, the error traces show that DynamoDB performance is normal but the Lambda function is raising an error after 5 seconds:\u003C\u002Fp>\n\u003Cp>\u003Cimg src=\"\u002Fassets\u002Fexternal\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fassets\u002Fimages\u002Fmonitoring-observability-figure-18.png\" alt=\"\">\u003C\u002Fp>\n\u003Cp>Investigating further in the CloudWatch Logs for the failing function, the error messages indicate that the function has timed out. The function is configured with a maximum duration of 5 seconds. While I can increase this timeout value to eliminate the error, the underlying cause of the error is still unknown. When I increase the timeout or the memory allocated to the Lambda function and run the load test again, this results in a higher percentage of 502 errors.\u003C\u002Fp>\n\u003Cp>When Lambda functions generate more errors as resources are increased, this can often indicate a downstream issue. For this application, the next downstream service is DynamoDB. Looking at the metrics for resource consumption on the DynamoDB table, this indicates the source of the problem:\u003C\u002Fp>\n\u003Cp>\u003Cimg src=\"\u002Fassets\u002Fexternal\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fassets\u002Fimages\u002Fmonitoring-observability-figure-19.png\" alt=\"\">\u003C\u002Fp>\n\u003Cp>The read capacity of the table is set to 5 RCUs in the AWS SAM template for the application. The load test is driving RCU usage to above 20 and the DynamoDB table is throttling read requests. This is not handled by the custom code in the Lambda function, resulting in a time-out error. By setting the DynamoDB table’s RCU value to 25 and re-running the test, I can compare the results in X-Ray:\u003C\u002Fp>\n\u003Cp>\u003Cimg src=\"\u002Fassets\u002Fexternal\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fassets\u002Fimages\u002Fmonitoring-observability-figure-20.png\" alt=\"\">\u003C\u002Fp>\n\u003Cp>All of the requests are now returning without errors but the performance of the Lambda function is still below expectations.\u003C\u002Fp>\n\u003Ch2>Isolating the performance issue\u003C\u002Fh2>\n\u003Cp>\u003Cimg src=\"\u002Fassets\u002Fexternal\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fassets\u002Fimages\u002Fmonitoring-observability-figure-21.png\" alt=\"\">\u003C\u002Fp>\n\u003Cp>In testing the function from an AWS Cloud9 instance and a local development machine, the average performance is faster, averaging 250 ms. The comparatively slow performance when running in Lambda is likely caused by the configuration instead of the code.\u003C\u002Fp>\n\u003Cp>First, I check if the Lambda function is running out of memory, since this is a major cause of slow-running functions. Using CloudWatch Insights, I can analyze the amount of memory used by the function during the load test. I use the \u003Cem>Percentile report of Lambda memory usage\u003C\u002Fem> query from the previous section.\u003C\u002Fp>\n\u003Cp>The results indicate that the function is using a maximum of 154.5 MB and 5% of all invocations use above 151.6 MB. The function is allocated 256 MB so it is not close to the memory maximum. However, memory settings in Lambda also determine the amount of virtual CPU available, so act as a power control. The function is computationally heavy so may benefit from more memory since this provides more computational power.\u003C\u002Fp>\n\u003Cp>I increase the function’s memory to 1024 MB and re-run the load test. The load test results show an improvement, with a median response time of 395 ms.\u003C\u002Fp>\n\u003Cp>\u003Cimg src=\"\u002Fassets\u002Fexternal\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fassets\u002Fimages\u002Fmonitoring-observability-figure-22.png\" alt=\"\">\u003C\u002Fp>\n\u003Cp>The X-Ray service map now also shows faster performance, reducing the average Lambda invocation duration from 739 ms to 284 ms, an improvement of more than 61%.\u003C\u002Fp>\n\u003Cp>\u003Cimg src=\"\u002Fassets\u002Fexternal\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fassets\u002Fimages\u002Fmonitoring-observability-figure-23.png\" alt=\"\">\u003C\u002Fp>\n\u003Cp>I can rerun the same test with different Lambda memory settings to determine which configuration provides the fastest performance. While I can do this manually, I can also automate this process to find the ideal balance between duration and cost, as described in chapter 6.\u003C\u002Fp>\n\u003Ch2>Measuring and improving the performance of outliers\u003C\u002Fh2>\n\u003Cp>In the latest test, although the average performance is considerably better, the p95 duration is 517 ms and the p99 duration is 747 ms. This means that 5% of all invocations are taking longer than 517 ms and 1% exceed 747 ms. The X-Ray service map shows a response distribution graph for the Lambda function:\u003C\u002Fp>\n\u003Cp>\u003Cimg src=\"\u002Fassets\u002Fexternal\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fassets\u002Fimages\u002Fmonitoring-observability-figure-24.png\" alt=\"\">\u003C\u002Fp>\n\u003Cp>Often, the greatest impact contributor to latency in these performance outliers is the effect of cold starts. By running the \u003Cem>Percentage of cold starts in total invocations\u003C\u002Fem> query in CloudWatch Insights, I can find how many cold starts occurred during the load test, and their average duration.\u003C\u002Fp>\n\u003Cp>\u003Cimg src=\"\u002Fassets\u002Fexternal\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fassets\u002Fimages\u002Fmonitoring-observability-figure-25.png\" alt=\"\">\u003C\u002Fp>\n\u003Cp>In this case, almost 2% of invocations were cold starts with an average total duration of 1618 ms, which explains the outlier performance. There are many additional optimizations that can be made to the Lambda function to improve the cold start latency, and these are covered in detail in \u003Ca href=\"perf-optimize\">Performance Optimization\u003C\u002Fa>.\u003C\u002Fp>\n",{"title":6,"order":7},"Troubleshooting walkthrough: isolating and resolving issues",74,"In the https:\u002F\u002Fgithub.com\u002Faws-samples\u002Faws-lambda-operators-guide[Coffee Lookup example] in the companion GitHub repository, users have provided feedback that the front-end application is slow and frequently showing errors. This section walks through the process of reproducing and diagnosing the cause of the problem.\n\n**Topics**\n\n- [Troubleshooting walkthrough: isolating and resolving issues](#troubleshooting-walkthrough-isolating-and-resolving-issues)\n  - [Reproducing the problem](#reproducing-the-problem)\n  - [Isolating the cause of errors](#isolating-the-cause-of-errors)\n  - [Isolating the performance issue](#isolating-the-performance-issue)\n  - [Measuring and improving the performance of outliers](#measuring-and-improving-the-performance-of-outliers)\n\n## Reproducing the problem\n\nThe first step is to reproduce the problem. Invoking the API using curl or Postman shows that the API is operating but does not highlight any errors. By running a load test against the backend API with [Artillery](https:\u002F\u002Fartillery.io\u002F), simulating the level of load in production, I can reproduce both issues.\n\nIn a test with 1500 requests over 5 minutes, 25% of the requests are failing with 502 errors and the overall performance is highly variable. While the median response time is 522 ms, which is worse than expected, the p95 response times are over 5.7 seconds. The Artillery report breaks down these findings further:\n\n![](\u002Fassets\u002Fexternal\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fassets\u002Fimages\u002Fmonitoring-observability-figure-16.png)\n\nTo learn more about load testing serverless APIs, see https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Fcompute\u002Fload-testing-a-web-applications-serverless-backend\u002F.\n\n## Isolating the cause of errors\n\nWhile the Artillery load test reports round-trip timings for API calls, it’s important to know the portion of latency caused by the backend application. I use X-Ray to investigate latency between the various AWS services used in the application. This helps to exclude networking delays or issues that are local to the client device. After a load test, I can check X-Ray or [ServiceLens](https:\u002F\u002Fdocs.aws.amazon.com\u002FAmazonCloudWatch\u002Flatest\u002Fmonitoring\u002FServiceLens.html) to verify the roundtrip performance once the requests are received by the API.\n\n![](\u002Fassets\u002Fexternal\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fassets\u002Fimages\u002Fmonitoring-observability-figure-17.png)\n\nThe service map shows service latency and failures in about 25% of the requests. X-Ray allows you to drill into requests by status code to further identify the causes of 502 errors. In this case, the error traces show that DynamoDB performance is normal but the Lambda function is raising an error after 5 seconds:\n\n![](\u002Fassets\u002Fexternal\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fassets\u002Fimages\u002Fmonitoring-observability-figure-18.png)\n\nInvestigating further in the CloudWatch Logs for the failing function, the error messages indicate that the function has timed out. The function is configured with a maximum duration of 5 seconds. While I can increase this timeout value to eliminate the error, the underlying cause of the error is still unknown. When I increase the timeout or the memory allocated to the Lambda function and run the load test again, this results in a higher percentage of 502 errors.\n\nWhen Lambda functions generate more errors as resources are increased, this can often indicate a downstream issue. For this application, the next downstream service is DynamoDB. Looking at the metrics for resource consumption on the DynamoDB table, this indicates the source of the problem:\n\n![](\u002Fassets\u002Fexternal\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fassets\u002Fimages\u002Fmonitoring-observability-figure-19.png)\n\nThe read capacity of the table is set to 5 RCUs in the AWS SAM template for the application. The load test is driving RCU usage to above 20 and the DynamoDB table is throttling read requests. This is not handled by the custom code in the Lambda function, resulting in a time-out error. By setting the DynamoDB table’s RCU value to 25 and re-running the test, I can compare the results in X-Ray:\n\n![](\u002Fassets\u002Fexternal\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fassets\u002Fimages\u002Fmonitoring-observability-figure-20.png)\n\nAll of the requests are now returning without errors but the performance of the Lambda function is still below expectations.\n\n## Isolating the performance issue\n\n![](\u002Fassets\u002Fexternal\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fassets\u002Fimages\u002Fmonitoring-observability-figure-21.png)\n\nIn testing the function from an AWS Cloud9 instance and a local development machine, the average performance is faster, averaging 250 ms. The comparatively slow performance when running in Lambda is likely caused by the configuration instead of the code.\n\nFirst, I check if the Lambda function is running out of memory, since this is a major cause of slow-running functions. Using CloudWatch Insights, I can analyze the amount of memory used by the function during the load test. I use the _Percentile report of Lambda memory usage_ query from the previous section.\n\nThe results indicate that the function is using a maximum of 154.5 MB and 5% of all invocations use above 151.6 MB. The function is allocated 256 MB so it is not close to the memory maximum. However, memory settings in Lambda also determine the amount of virtual CPU available, so act as a power control. The function is computationally heavy so may benefit from more memory since this provides more computational power.\n\nI increase the function’s memory to 1024 MB and re-run the load test. The load test results show an improvement, with a median response time of 395 ms.\n\n![](\u002Fassets\u002Fexternal\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fassets\u002Fimages\u002Fmonitoring-observability-figure-22.png)\n\nThe X-Ray service map now also shows faster performance, reducing the average Lambda invocation duration from 739 ms to 284 ms, an improvement of more than 61%.\n\n![](\u002Fassets\u002Fexternal\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fassets\u002Fimages\u002Fmonitoring-observability-figure-23.png)\n\nI can rerun the same test with different Lambda memory settings to determine which configuration provides the fastest performance. While I can do this manually, I can also automate this process to find the ideal balance between duration and cost, as described in chapter 6.\n\n## Measuring and improving the performance of outliers\n\nIn the latest test, although the average performance is considerably better, the p95 duration is 517 ms and the p99 duration is 747 ms. This means that 5% of all invocations are taking longer than 517 ms and 1% exceed 747 ms. The X-Ray service map shows a response distribution graph for the Lambda function:\n\n![](\u002Fassets\u002Fexternal\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fassets\u002Fimages\u002Fmonitoring-observability-figure-24.png)\n\nOften, the greatest impact contributor to latency in these performance outliers is the effect of cold starts. By running the _Percentage of cold starts in total invocations_ query in CloudWatch Insights, I can find how many cold starts occurred during the load test, and their average duration.\n\n![](\u002Fassets\u002Fexternal\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fassets\u002Fimages\u002Fmonitoring-observability-figure-25.png)\n\nIn this case, almost 2% of invocations were cold starts with an average total duration of 1618 ms, which explains the outlier performance. There are many additional optimizations that can be made to the Lambda function to improve the cold start latency, and these are covered in detail in [Performance Optimization](perf-optimize).\n",[10,21,139,247,314,384,486,555],{"title":-1,"collapsible":11,"isCollapsed":12,"content":13},false,true,[14],{"title":15,"order":16,"time":17,"path":18,"id":19,"link":20},"Introduction",1,"4 min","intro","intro.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fintro",{"title":22,"collapsible":12,"isCollapsed":12,"content":23},"Event-driven architectures",[24,30,37,43,49,55,62,68,74,80,86,92,98,104,110,116,122,128,133],{"title":22,"order":25,"time":26,"path":27,"id":28,"link":29},2,"1 min","event-driven-architectures","event-driven-architectures.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fevent-driven-architectures",{"title":31,"order":32,"time":33,"path":34,"id":35,"link":36},"How Lambda fits into the event-driven paradigm",3,"3 min","lambda-event-driven-paradigm","lambda-event-driven-paradigm.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Flambda-event-driven-paradigm",{"title":38,"order":39,"time":33,"path":40,"id":41,"link":42},"The benefits of event-driven architectures",4,"event-driven-benefits","event-driven-benefits.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fevent-driven-benefits",{"title":44,"order":45,"time":17,"path":46,"id":47,"link":48},"Trade-offs of event-driven architectures",5,"tradeoffs-event-driven","tradeoffs-event-driven.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Ftradeoffs-event-driven",{"title":50,"order":51,"time":26,"path":52,"id":53,"link":54},"Design principles",6,"design-principles","design-principles.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fdesign-principles",{"title":56,"order":57,"time":58,"path":59,"id":60,"link":61},"Use services instead of custom code",7,"2 min","services-custom-code","services-custom-code.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fservices-custom-code",{"title":63,"order":64,"time":26,"path":65,"id":66,"link":67},"Understanding the level of abstraction",8,"level-abstraction","level-abstraction.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Flevel-abstraction",{"title":69,"order":70,"time":26,"path":71,"id":72,"link":73},"Implementing statelessness in functions",9,"statelessness-functions","statelessness-functions.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fstatelessness-functions",{"title":75,"order":76,"time":26,"path":77,"id":78,"link":79},"Lambda function design",10,"function-design","function-design.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Ffunction-design",{"title":81,"order":82,"time":58,"path":83,"id":84,"link":85},"Building for on-demand data instead of batches",11,"on-demand-batches","on-demand-batches.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fon-demand-batches",{"title":87,"order":88,"time":26,"path":89,"id":90,"link":91},"Orchestration",12,"orchestration","orchestration.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Forchestration",{"title":93,"order":94,"time":58,"path":95,"id":96,"link":97},"Developing for retries and failures",13,"retries-failure","retries-failure.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fretries-failure",{"title":99,"order":100,"time":26,"path":101,"id":102,"link":103},"Anti-patterns in Lambda-based applications",14,"anti-patterns","anti-patterns.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fanti-patterns",{"title":105,"order":106,"time":17,"path":107,"id":108,"link":109},"The Lambda monolith",15,"monolith","monolith.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fmonolith",{"title":111,"order":112,"time":58,"path":113,"id":114,"link":115},"Lambda as orchestrator",16,"orchestrator","orchestrator.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Forchestrator",{"title":117,"order":118,"time":58,"path":119,"id":120,"link":121},"Recursive patterns that cause run-away Lambda functions",17,"recursive-runaway","recursive-runaway.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Frecursive-runaway",{"title":123,"order":124,"time":33,"path":125,"id":126,"link":127},"Lambda functions calling Lambda functions",19,"functions-calling-functions","functions-calling-functions.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Ffunctions-calling-functions",{"title":129,"order":124,"time":26,"path":130,"id":131,"link":132},"Synchronous waiting within a single Lambda function","synchronous-waiting","synchronous-waiting.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fsynchronous-waiting",{"title":134,"order":135,"time":33,"path":136,"id":137,"link":138},"Frequently asked questions",20,"event-driven-faq","event-driven-faq.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fevent-driven-faq",{"title":140,"collapsible":12,"isCollapsed":12,"content":141},"Application design",[142,147,153,159,165,171,177,182,188,194,200,206,212,218,224,230,236,242],{"title":140,"order":143,"time":26,"path":144,"id":145,"link":146},21,"application-design","application-design.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fapplication-design",{"title":148,"order":149,"time":33,"path":150,"id":151,"link":152},"Understanding quotas",22,"quotas","quotas.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fquotas",{"title":154,"order":155,"time":58,"path":156,"id":157,"link":158},"Architecting with Service Quotas",23,"service-quotas","service-quotas.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fservice-quotas",{"title":160,"order":161,"time":58,"path":162,"id":163,"link":164},"Using multiple AWS accounts for managing quotas",24,"multiple-accounts","multiple-accounts.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fmultiple-accounts",{"title":166,"order":167,"time":26,"path":168,"id":169,"link":170},"Scaling and concurrency in Lambda",25,"scaling-concurrency","scaling-concurrency.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fscaling-concurrency",{"title":172,"order":173,"time":58,"path":174,"id":175,"link":176},"On-demand scaling example",26,"on-demand-scaling","on-demand-scaling.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fon-demand-scaling",{"title":178,"order":173,"time":58,"path":179,"id":180,"link":181},"Provisioned Concurrency scaling example","provisioned-scaling","provisioned-scaling.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fprovisioned-scaling",{"title":183,"order":184,"time":58,"path":185,"id":186,"link":187},"Using service integrations and asynchronous processing",28,"integrations-asynchronous","integrations-asynchronous.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fintegrations-asynchronous",{"title":189,"order":190,"time":58,"path":191,"id":192,"link":193},"Reserved concurrency",29,"reserved-concurrency","reserved-concurrency.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Freserved-concurrency",{"title":195,"order":196,"time":26,"path":197,"id":198,"link":199},"Choosing and managing runtimes in Lambda functions",30,"runtimes-functions","runtimes-functions.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fruntimes-functions",{"title":201,"order":202,"time":26,"path":203,"id":204,"link":205},"Runtimes and performance",31,"runtimes-performance","runtimes-performance.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fruntimes-performance",{"title":207,"order":208,"time":26,"path":209,"id":210,"link":211},"Multiple runtimes in single applications",32,"runtimes-multiple","runtimes-multiple.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fruntimes-multiple",{"title":213,"order":214,"time":26,"path":215,"id":216,"link":217},"Managing AWS SDKs in Lambda functions",33,"sdks-functions","sdks-functions.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fsdks-functions",{"title":219,"order":220,"time":33,"path":221,"id":222,"link":223},"Networking and VPC configurations",34,"networking-vpc","networking-vpc.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fnetworking-vpc",{"title":225,"order":226,"time":33,"path":227,"id":228,"link":229},"Comparing Lambda invocation modes",35,"invocation-modes","invocation-modes.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Finvocation-modes",{"title":231,"order":232,"time":58,"path":233,"id":234,"link":235},"Understanding SQS retries",36,"sqs-retries","sqs-retries.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fsqs-retries",{"title":237,"order":238,"time":58,"path":239,"id":240,"link":241},"Controlling traffic flow for server-based resources",37,"traffic-server-based","traffic-server-based.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Ftraffic-server-based",{"title":134,"order":243,"time":33,"path":244,"id":245,"link":246},38,"application-design-faq","application-design-faq.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fapplication-design-faq",{"title":248,"collapsible":12,"isCollapsed":12,"content":249},"Security",[250,255,261,267,273,279,285,291,297,303,309],{"title":248,"order":251,"time":58,"path":252,"id":253,"link":254},39,"security-ops","security-ops.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fsecurity-ops",{"title":256,"order":257,"time":33,"path":258,"id":259,"link":260},"Understanding the Lambda execution environment",40,"execution-environment","execution-environment.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fexecution-environment",{"title":262,"order":263,"time":58,"path":264,"id":265,"link":266},"Applying the principles of least privilege",41,"least-privilege","least-privilege.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fleast-privilege",{"title":268,"order":269,"time":58,"path":270,"id":271,"link":272},"Developing least privilege IAM roles",42,"least-privilege-iam","least-privilege-iam.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fleast-privilege-iam",{"title":274,"order":275,"time":26,"path":276,"id":277,"link":278},"Access to CloudWatch Logs",43,"access-logs","access-logs.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Faccess-logs",{"title":280,"order":281,"time":58,"path":282,"id":283,"link":284},"Avoiding granting wildcard permissions in IAM policies",44,"wildcard-permissions-iam","wildcard-permissions-iam.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fwildcard-permissions-iam",{"title":286,"order":287,"time":26,"path":288,"id":289,"link":290},"Specialized Lambda functions compared with all-purpose functions",45,"specialized-all-purpose","specialized-all-purpose.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fspecialized-all-purpose",{"title":292,"order":293,"time":17,"path":294,"id":295,"link":296},"Securing workloads with public endpoints",46,"public-endpoints","public-endpoints.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fpublic-endpoints",{"title":298,"order":299,"time":58,"path":300,"id":301,"link":302},"Encrypting data in Lambda-based applications",47,"data-in-applications","data-in-applications.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fdata-in-applications",{"title":304,"order":305,"time":33,"path":306,"id":307,"link":308},"Security governance controls",48,"governance-controls","governance-controls.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fgovernance-controls",{"title":134,"order":310,"time":58,"path":311,"id":312,"link":313},49,"security-ops-faq","security-ops-faq.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fsecurity-ops-faq",{"title":315,"collapsible":12,"isCollapsed":12,"content":316},"Debugging",[317,322,328,334,340,347,354,360,366,372,378],{"title":315,"order":318,"time":26,"path":319,"id":320,"link":321},50,"debugging-ops","debugging-ops.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fdebugging-ops",{"title":323,"order":324,"time":33,"path":325,"id":326,"link":327},"Standardizing a debugging approach",51,"debugging-approach","debugging-approach.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fdebugging-approach",{"title":329,"order":330,"time":58,"path":331,"id":332,"link":333},"General types of error",52,"general-types","general-types.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fgeneral-types",{"title":335,"order":336,"time":17,"path":337,"id":338,"link":339},"Troubleshooting payloads",53,"payload","payload.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fpayload",{"title":341,"order":342,"time":343,"path":344,"id":345,"link":346},"Troubleshooting integration errors",54,"5 min","integration-errors","integration-errors.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fintegration-errors",{"title":348,"order":349,"time":350,"path":351,"id":352,"link":353},"Troubleshooting Lambda configurations",55,"9 min","configurations","configurations.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fconfigurations",{"title":355,"order":356,"time":58,"path":357,"id":358,"link":359},"Troubleshooting queue processing by Lambda functions",56,"queue-processing","queue-processing.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fqueue-processing",{"title":361,"order":362,"time":58,"path":363,"id":364,"link":365},"Identifying and managing throttling",57,"throttling","throttling.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fthrottling",{"title":367,"order":368,"time":26,"path":369,"id":370,"link":371},"Errors in the processing function",58,"processing-function","processing-function.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fprocessing-function",{"title":373,"order":374,"time":26,"path":375,"id":376,"link":377},"Identifying and handling backpressure",59,"backpressure","backpressure.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fbackpressure",{"title":379,"order":380,"time":33,"path":381,"id":382,"link":383},"Best practices for your debugging environment",60,"best-practices-debugging","best-practices-debugging.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fbest-practices-debugging",{"title":385,"collapsible":12,"isCollapsed":11,"content":386},"Monitoring and observability",[387,392,398,404,410,416,422,428,434,440,446,452,458,464,468,474,480],{"title":385,"order":388,"time":26,"path":389,"id":390,"link":391},61,"monitoring-observability","monitoring-observability.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fmonitoring-observability",{"title":393,"order":394,"time":33,"path":395,"id":396,"link":397},"Monitoring concepts in Lambda-based applications",62,"monitoring-concepts","monitoring-concepts.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fmonitoring-concepts",{"title":399,"order":400,"time":26,"path":401,"id":402,"link":403},"Logging and metrics with Amazon CloudWatch",63,"logging-metrics","logging-metrics.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Flogging-metrics",{"title":405,"order":406,"time":58,"path":407,"id":408,"link":409},"How CloudWatch structures logs",64,"log-structure","log-structure.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Flog-structure",{"title":411,"order":412,"time":33,"path":413,"id":414,"link":415},"Important metrics for CloudWatch",65,"important-metrics","important-metrics.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fimportant-metrics",{"title":417,"order":418,"time":58,"path":419,"id":420,"link":421},"Custom metrics",66,"custom-metrics","custom-metrics.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fcustom-metrics",{"title":423,"order":424,"time":26,"path":425,"id":426,"link":427},"Using AWS Resource Groups to organize your workload",67,"organize-workload","organize-workload.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Forganize-workload",{"title":429,"order":430,"time":58,"path":431,"id":432,"link":433},"Searching across logs with CloudWatch Logs Insights",68,"search-logs","search-logs.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fsearch-logs",{"title":435,"order":436,"time":33,"path":437,"id":438,"link":439},"Parsing logs and structured logging",69,"parse-logs","parse-logs.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fparse-logs",{"title":441,"order":442,"time":26,"path":443,"id":444,"link":445},"Querying AWS-generated events",70,"query-events","query-events.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fquery-events",{"title":447,"order":448,"time":26,"path":449,"id":450,"link":451},"Log visualization and dashboard",71,"visualization","visualization.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fvisualization",{"title":453,"order":454,"time":58,"path":455,"id":456,"link":457},"Useful Insights queries",72,"useful-queries","useful-queries.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fuseful-queries",{"title":459,"order":460,"time":58,"path":461,"id":462,"link":463},"Tracing requests with AWS X-Ray",73,"trace-requests","trace-requests.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Ftrace-requests",{"title":6,"order":7,"time":343,"path":465,"id":466,"link":467},"troubleshooting-walkthrough","troubleshooting-walkthrough.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Ftroubleshooting-walkthrough",{"title":469,"order":470,"time":58,"path":471,"id":472,"link":473},"A general approach to debugging Lambda performance issues and errors",75,"general-approach","general-approach.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fgeneral-approach",{"title":475,"order":476,"time":26,"path":477,"id":478,"link":479},"Monitoring Lambda code storage",76,"code-storage","code-storage.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fcode-storage",{"title":481,"order":482,"time":33,"path":483,"id":484,"link":485},"Best practices for managing code storage",77,"code-storage-best-practice","code-storage-best-practice.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fcode-storage-best-practice",{"title":487,"collapsible":12,"isCollapsed":12,"content":488},"Performance optimization",[489,494,501,507,513,519,525,531,537,543,549],{"title":487,"order":490,"time":26,"path":491,"id":492,"link":493},78,"perf-optimize","perf-optimize.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fperf-optimize",{"title":495,"order":496,"time":497,"path":498,"id":499,"link":500},"Lambda execution environments",79,"7 min","execution-environments","execution-environments.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fexecution-environments",{"title":502,"order":503,"time":58,"path":504,"id":505,"link":506},"Memory and computing power",80,"computing-power","computing-power.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fcomputing-power",{"title":508,"order":509,"time":58,"path":510,"id":511,"link":512},"Profiling functions with AWS Lambda Power Tuning",81,"profile-functions","profile-functions.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fprofile-functions",{"title":514,"order":515,"time":33,"path":516,"id":517,"link":518},"Optimizing static initialization",82,"static-initialization","static-initialization.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fstatic-initialization",{"title":520,"order":521,"time":58,"path":522,"id":523,"link":524},"Comparing the effect of global scope",83,"global-scope","global-scope.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fglobal-scope",{"title":526,"order":527,"time":26,"path":528,"id":529,"link":530},"Static initialization and Provisioned Concurrency",84,"initialization-pc","initialization-pc.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Finitialization-pc",{"title":532,"order":533,"time":26,"path":534,"id":535,"link":536},"Architecture and Best Practices",85,"architecture-best-practice","architecture-best-practice.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Farchitecture-best-practice",{"title":538,"order":539,"time":58,"path":540,"id":541,"link":542},"Comparing performance of interactive and asynchronous workloads",86,"interactive-asynchronous","interactive-asynchronous.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Finteractive-asynchronous",{"title":544,"order":545,"time":58,"path":546,"id":547,"link":548},"When not to use a Lambda function",87,"no-function","no-function.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fno-function",{"title":550,"order":551,"time":58,"path":552,"id":553,"link":554},"Cost optimization",88,"cost-optimize","cost-optimize.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fcost-optimize",{"title":-1,"collapsible":11,"isCollapsed":12,"content":556},[557],{"title":558,"order":559,"time":26,"path":560,"id":561,"link":562},"Document history",89,"doc-history","doc-history.md","\u002Fcontent\u002Fservice\u002Flambda\u002Fguides\u002Faws-lambda-operator-guide\u002Fdoc-history","LIST",[],"AWS Lambda Operator Guide","6 min",1792578871477]