[{"data":1,"prerenderedAt":67},["ShallowReactive",2],{"pattern-s3-eventbridge-glue":3},{"id":4,"title":5,"architectureURL":6,"cleanup":7,"contributors":10,"deploy":12,"description":15,"extension":16,"framework":17,"gitHub":18,"highlight":6,"introBox":24,"language":31,"level":32,"meta":33,"patternArch":34,"resources":54,"s3URL":6,"services":6,"stem":62,"testing":63,"videoId":6,"__hash__":66},"patterns\u002Fpatterns\u002Fs3-eventbridge-glue.json","Automatic data transformation with Amazon S3, AWS Glue and AWS Step Functions",null,{"text":8},[9],"Delete the stack: \u003Ccode>cdk destroy\u003C\u002Fcode>.",[11],"content\u002Fcontributors\u002Felie-elmalem.json",{"text":13},[14],"cdk deploy","This pattern helps transform data by using an event-driven architecture to trigger a data transformation job","json","AWS CDK",{"template":19},{"repoURL":20,"templateURL":21,"projectFolder":22,"templateFile":23},"https:\u002F\u002Fgithub.com\u002Faws-samples\u002Fserverless-patterns\u002Ftree\u002Fmain\u002Fs3-eventbridge-glue","serverless-patterns\u002Fs3-eventbridge-glue","s3-eventbridge-glue","s3_eventbridge_crawler\u002Fs3_eventbridge_crawler_stack.py",{"headline":25,"text":26},"How it works",[27,28,29,30],"When you upload a .csv file to the Input S3 bucket, it matches with an Amazon EventBridge rule that triggers a State Machine. The State Machine is composed of two steps:","1. Starts a AWS Glue Crawler to crawl the input bucket. This allows you to have the raw data (before transformation) inside the Data Catalog","2. Start a AWS Glue ETL Job, that runs a simple transformation job and drops empty columns in the dataset","The processed\u002Ftransformed data is then added to the Data Catalog and to an S3 bucket to allow for further processing.","Python","200",{},{"icon1":35,"icon2":40,"icon3":43,"line1":47,"line2":51},{"x":36,"y":37,"service":38,"label":39},20,50,"s3","Amazon S3",{"x":37,"y":37,"service":41,"label":42},"sfn","AWS Step Functions",{"x":44,"y":37,"service":45,"label":46},80,"glue","AWS Glue",{"from":48,"to":49,"label":50},"icon1","icon2","EventBridge rule",{"from":49,"to":52,"label":53},"icon3","",{"bullets":55},[56,59],{"text":57,"link":58},"Using an Amazon S3 trigger with Amazon EventBridge","https:\u002F\u002Fdocs.aws.amazon.com\u002FAmazonS3\u002Flatest\u002Fuserguide\u002FEventBridge.html",{"text":60,"link":61},"Getting started with AWS Glue","https:\u002F\u002Fdocs.aws.amazon.com\u002Fglue\u002Flatest\u002Fdg\u002Fwhat-is-glue.html","patterns\u002Fs3-eventbridge-glue",{"text":64},[65],"See the GitHub repo for detailed testing instructions.","tkcTWDSIWuQ3qoUGPmaU5zVH1HsM0ImkAsHsLHis0IM",1778846887188]