Kafka Event Source Mapping (ESM) is a Lambda resource that read messages from a Kafka topic and invokes a target Lambda function. It has three main components. The first is the poller process that uses Kafka client libraries to poll the Kafka topic. The ESM resource initializes with a single poller process for reading from all the partitions. As messages are read from the topic, they are passed on to the next component that does batching based on the batch size and interval. Once the batch is ready, it is passed on to a invoker which calls the target Lambda function. Each poller is associated with one instance of a Lambda function and the invocations are synchronous. This pattern allows processing messages in the order in which they are stored within a partition.

For a more deep dive learning journey follow the Kafka Lambda learning guide.