Amazon API Gateway to AWS Step Functions Express to Amazon Athena

Client → Amazon API Gateway → AWS Step Functions Express → Amazon Athena → Amazon S3

Run an Amazon Athena query through an AWS Step Functions Express Workflow and return the rows in one synchronous Amazon API Gateway call, with no Lambda function.

This pattern runs an Amazon Athena query and returns its rows in one synchronous Amazon API Gateway request. The REST API starts an AWS Step Functions Express Workflow with StartSyncExecution, and the workflow calls Athena through native service integrations, so there's no AWS Lambda function in the request path.
Express Workflows don't support the .sync integration pattern, so the workflow starts the query with StartQueryExecution and polls GetQueryExecution in a Wait and Choice loop. When the query succeeds, GetQueryResults reads the rows, which must fit in the 256 KiB Step Functions payload. After 10 polls the workflow calls StopQueryExecution and fails, so the caller gets an HTTP 500 before the 29-second integration timeout.
The method uses IAM authorization, because the caller's SQL goes straight to Athena, and the stage is throttled to 5 requests per second. Athena reads a sample CSV file through an AWS Glue table and writes results to an Amazon S3 prefix set by its workgroup.

< Back to all patterns

GitHub icon Download this pattern (.zip)

GitHub icon View this pattern on GitHub


Clone repo

git clone https://github.com/aws-samples/serverless-patterns/cd serverless-patterns/apigw-sfn-athena/typescript/cdk

Deploy

cd apigw-sfn-athena/typescript/cdk/cdknpm installnpx cdk deploy


Testing

Send a SigV4-signed POST with a queryString to the ApiEndpoint output, for example with curl --aws-sigv4. See the README in the GitHub repo for the commands.

Cleanup

Delete the stack: cd apigw-sfn-athena/typescript/cdk/cdk then npx cdk destroy.
Warning: this deletes the Amazon S3 bucket and all sample data and Amazon Athena query results.

Created by:

Nithin Chandran R

Nithin Chandran R

Technical Account Manager at AWS