Optimizing AWS Lambda costs by avoiding paying for idle

Lambda scales both up and down automatically based on the traffic patterns, and as such autonomously adjusts allocated capacity to match required capacity. It is a great fit for cost optimizations in scenarios when you don’t want to deal with infrastructure management, have a spiky traffic pattern, and know you’re over-provisioning for worst-case scenario. Having additional insights on how Lambda works can help you to optimize costs even further.

Multi-step and long-running tasks have a great potential for optimization. When running Lambda functions, you are billed for the allocated memory and invocation duration. If your function runs for 500ms - you will be billed for 500ms, regardless of whether your code was busy with computations, or just idle waiting for a nested dependency to respond. A good question to ask yourself is do you have synchronous workflows where you need to wait for task completion before proceeding to the next step? Or do you have multiple nested Lambda functions calls? Imagine you have four functions with synchronous nested invokes. This setup would work, but the problem with this is your functions are sitting idle, waiting for nested downstream function to respond.

There are several optimization approaches that can be applied here. First of all - avoid nested functions. Having multiple levels of nested invokes implies paying for idle, so try to analyze the impact even if you have one level of nesting. If you have a complex workflow - see if you can rebuild it in an asynchronous, event-driven way. Remove the idle time of synchronously waiting for downstream dependencies to respond. Replace it with asynchronous request and subscribe for completion events using EventBridge. This way you're maximizing the time your functions spend actually doing meaningful work.

Secondly, try to avoid orchestrating workflows with Lambda. Use services like AWS Step Functions for orchestrating complex workflows. With Step Functions you build your workflows in a visual editor. This allows you to visualize complex flows, make it much easier to design and less error-prone than writing code. Step Functions provide you with several ways of invoking downstream services - you can either wait for the invocation to complete, periodically polling the state, or you can pause the workflow until the downstream service notifies you of completion with a callback. All that with zero lines of code!

Step Functions have two flavors - Standard and Express. Standard are great for workflows that run for long time - hours, days, weeks, months. With Standard you're paying for state transitions, regardless of the actual execution time. Express workflows, on the other hand, and used when you need your workflows to complete fast in under 5 minutes. With Express you'll be paying for execution time, regardless of how many state transitions occurred.

In conclusion, you can optimize your costs by eliminating unnecessary idle waiting time, which usually occurs when building multi-step long-running synchronous workflows. Convert your synchronous workflow into asynchronous by using event-driven architecture. If you do need to have end-to-end orchestration for a long, complex workflow - evaluate using Step Functions instead of coding it.