[{"data":1,"prerenderedAt":53},["ShallowReactive",2],{"workflow-query-large-datasets-cdk":3},{"id":4,"title":5,"cleanup":6,"contributors":10,"deploy":12,"description":15,"diagram":16,"extension":17,"framework":18,"gitHub":19,"introBox":28,"level":32,"meta":33,"resources":34,"s3URL":37,"services":38,"simplicity":44,"stem":45,"testing":46,"type":50,"usecase":51,"videoId":27,"__hash__":52},"workflows\u002Fworkflows\u002Fquery-large-datasets-cdk.json","Query large datasets",{"headline":7,"text":8},"Cleanup",[9],"Delete the stack: \u003Ccode>cdk destroy\u003C\u002Fcode>.",[11],"content\u002Fcontributors\u002Fpajtim-matoshi.json",{"text":13},[14],"cdk deploy","Ingest large datasets in S3, partition with Glue Crawlers, and perform Athena queries. Includes Step Functions, S3, Glue, and SNS.","\u002Fassets\u002Fimages\u002Fworkflows\u002Fquery-large-datasets-cdk.png","json","AWS CDK",{"template":20,"payloads":25},{"repoURL":21,"templateDir":22,"templateFile":23,"ASL":24},"https:\u002F\u002Fgithub.com\u002Faws-samples\u002Fstep-functions-workflows-collection\u002Ftree\u002Fmain\u002Fsfn-query-large-datasets-cdk\u002F","sfn-query-large-datasets-sam","app.py","statemachine\u002Fstatemachine.asl.json",[26],{"headline":27,"payloadURL":27},"",{"headline":29,"text":30},"How it works",[31],"In this project, the Step Functions state machine invokes an AWS Glue crawler that partitions a large dataset in Amazon S3. Once the AWS Glue crawler returns a success message, the workflow executes Athena queries against that partition. Once query execution is successfully complete, an Amazon SNS notification is sent to an Amazon SNS topic.","200",{},{"headline":35,"bullets":36},"Additional resources",[],null,[39,40,41,42,43],"sfn","glue","s3","athena","sns","2 - Pattern","workflows\u002Fquery-large-datasets-cdk",{"headline":47,"text":48},"Testing",[49],"See the GitHub repo for detailed testing instructions.","Standard","Data Processing","e7yUd0qXnqVH9K64dRKhhKVjVScGCU09JEl1NWOg_b4",1778846889364]