Process High-Volume Messages from Amazon SQS

This sample project demonstrates how to use an AWS Step Functions Express Workflow to process messages or data from a high-volume event source, such as Amazon Simple Queue Service (Amazon SQS).

When messages are sent to the SQS queue, a Lambda function that has event source mapping to the SQS queue will invoke the state machine.
The state machine will then invoke a Lambda function that will decode the base64 string recieved from the SQS queue. The Lambda function's output will be sent to the next state.
Next, the state machine will invoke a Lambda function that will generate statistics based on the string input recieved. Statistics include text length, average word length, number of digits and number of special characters. The Lambda function's output will be sent to the next state.
Next, The state machine will invoke a Lambda function that will remove special characters from the string input recieved. The Lambda function's output will be sent to the next state.
Finally, the state machine will invoke a Lambda function that will tokenize and count the occurance of each word in the string input recieved.

< Back to all workflows

GitHub icon View this workflow on GitHub


Clone repo

git clone https://github.com/aws-samples/step-functions-workflows-collection/tree/main/text-processing-sqs-express-cdkcd step-functions-workflows-collection/text-processing-sqs-express-cdk

Deploy

cdk deploy


Testing

See the GitHub repo for detailed testing instructions.

Cleanup

1. Delete the stack: cdk destroy.

Additional resources

Created by:

Zainab Allawi

Zainab Allawi

Zainab Allawi is a Solutions Architect at Amazon Web Services (AWS) based in Bahrain. She enjoys collaborating with customers to build automated solutions in AWS to solve every-day challenges.

Follow on LinkedIn