[{"data":1,"prerenderedAt":98},["ShallowReactive",2],{"content-svc-step-functions-guides-integrating-step-functions-with-containerized-workloads-50-run-a-job":3},{"markdown":4,"frontMatterAttributes":5,"bodyRaw":16,"menu":17,"menuType":95,"isCollapseableMenu":20,"nextDocItem":60,"previousDocItem":40,"contributorPaths":96,"contentName":19,"slug":59,"readingTime":36},"\u003Ch4>Invoking Amazon ECS\u002FAmazon EKS task using Run a Job integrations\u003C\u002Fh4>\n\n\u003Cp>You can use Amazon ECS RunTask API to run short running tasks in containers. With Step Functions Run a Job integration, you can invoke Amazon ECS RunTask API and wait for the task to complete. Step Functions starts a new task using the specified task definition, waits for the task to complete and then calls the next step in the workflow.\u003C\u002Fp>\n\u003Cp>Similar to ECS integration, Step Functions can interact with Kubernetes API and run jobs as part of your application workflows. \u003Ca href=\"https:\u002F\u002Fdocs.aws.amazon.com\u002Fstep-functions\u002Flatest\u002Fdg\u002Fconnect-eks.html\">Step Functions EKS integration\u003C\u002Fa> supports only Kubernetes API server with \u003Ccode>public endpoint access\u003C\u002Fcode>. \u003C\u002Fp>\n\u003Cp>Sample code using Amazon ECS Runtask API\u003C\u002Fp>\n\u003Cpre>\u003Ccode class=\"language-json\">          "ECS RunTask": {\n            "Type": "Task",\n            "Resource": "arn:aws:states:::ecs:runTask.sync",\n            "Parameters": {\n              "LaunchType": "FARGATE",\n              "Cluster": "arn:aws:ecs:{region}:{account}:cluster\u002F{cluster}",\n              "TaskDefinition": "arn:aws:ecs:{region}:{account}:task-definition\u002Fecsdemo-sfn:2",\n              "Overrides": {\n                "ContainerOverrides": [\n                  {\n                    "Name": "ecsdemo-sfn",\n                    "Environment": [\n                      {\n                        "Name": "ITEMS",\n                        "Value.$": "States.JsonToString($)"\n                      },\n                      {\n                        "Name": "INPUT_BUCKET",\n                        "Value": "bucket1s50db36y1g4h"\n                      }\n                    ]\n                  }\n                ]\n              }\n            },\n            "End": true\n          }\n\n\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cp>Notice the Resource attribute in the above code. If .sync is not added in the resource, the request follows \u003Ccode>Request Response\u003C\u002Fcode> integration pattern which will not wait for the task to complete.\u003C\u002Fp>\n\u003Cp>You can pass parameters to ECS task using Environment attribute within ContainerOverrides.  \u003C\u002Fp>\n\u003Cp>\u003Cstrong>Considerations\u003C\u002Fstrong>\u003C\u002Fp>\n\u003Cul>\n\u003Cli>Container overrides JSON has a max size limit of 8192 symbols. If you need to pass more data to the ECS task, you need to consider storing the payload in external storage or activity integration.\u003C\u002Fli>\n\u003Cli>You can not send response back to the Step Functions workflow from the ECS task\u002FEKS job. You need to use mechanisms such as Amazon S3 or DynamoDB to store the data for later use in the Step Functions workflow or use \u003Ccode>Wait for a Callback\u003C\u002Fcode> or \u003Ccode>Activity\u003C\u002Fcode> integration.\u003C\u002Fli>\n\u003Cli>In order to determine if the ECS task is complete, Step Functions call the ECS:DescribeTasks API regularly. At high volume, you might observe throttles on the DescribeTasks API. To avoid such throttles, you can remove ECS:DescribeTasks IAM permission from state machine’s execution role. Step Functions will use \u003Ca href=\"https:\u002F\u002Fdocs.aws.amazon.com\u002FAmazonECS\u002Flatest\u002Fdeveloperguide\u002Fecs_cwe_events.html\">Amazon ECS events for Amazon EventBridge\u003C\u002Fa> to determine if the task is complete.\u003C\u002Fli>\n\u003C\u002Ful>\n",{"title":6,"order":7,"resources":8},"Run a job Integration",50,[9,13],{"link":10,"text":11,"type":12},"https:\u002F\u002Fdocs.aws.amazon.com\u002Fstep-functions\u002Flatest\u002Fdg\u002Fconnect-ecs.html","Manage Amazon ECS or Fargate Tasks with Step Functions","documentation",{"link":14,"text":15,"type":12},"https:\u002F\u002Fdocs.aws.amazon.com\u002Fstep-functions\u002Flatest\u002Fdg\u002Fconnect-eks.html","Call Amazon EKS with Step Functions","\u003Ch4>Invoking Amazon ECS\u002FAmazon EKS task using Run a Job integrations\u003C\u002Fh4>\n\nYou can use Amazon ECS RunTask API to run short running tasks in containers. With Step Functions Run a Job integration, you can invoke Amazon ECS RunTask API and wait for the task to complete. Step Functions starts a new task using the specified task definition, waits for the task to complete and then calls the next step in the workflow.\n\nSimilar to ECS integration, Step Functions can interact with Kubernetes API and run jobs as part of your application workflows. [Step Functions EKS integration](https:\u002F\u002Fdocs.aws.amazon.com\u002Fstep-functions\u002Flatest\u002Fdg\u002Fconnect-eks.html) supports only Kubernetes API server with `public endpoint access`. \n\nSample code using Amazon ECS Runtask API\n\n```json\n          \"ECS RunTask\": {\n            \"Type\": \"Task\",\n            \"Resource\": \"arn:aws:states:::ecs:runTask.sync\",\n            \"Parameters\": {\n              \"LaunchType\": \"FARGATE\",\n              \"Cluster\": \"arn:aws:ecs:{region}:{account}:cluster\u002F{cluster}\",\n              \"TaskDefinition\": \"arn:aws:ecs:{region}:{account}:task-definition\u002Fecsdemo-sfn:2\",\n              \"Overrides\": {\n                \"ContainerOverrides\": [\n                  {\n                    \"Name\": \"ecsdemo-sfn\",\n                    \"Environment\": [\n                      {\n                        \"Name\": \"ITEMS\",\n                        \"Value.$\": \"States.JsonToString($)\"\n                      },\n                      {\n                        \"Name\": \"INPUT_BUCKET\",\n                        \"Value\": \"bucket1s50db36y1g4h\"\n                      }\n                    ]\n                  }\n                ]\n              }\n            },\n            \"End\": true\n          }\n\n\n```\n\nNotice the Resource attribute in the above code. If .sync is not added in the resource, the request follows `Request Response` integration pattern which will not wait for the task to complete.\n\nYou can pass parameters to ECS task using Environment attribute within ContainerOverrides.  \n\n**Considerations**\n\n* Container overrides JSON has a max size limit of 8192 symbols. If you need to pass more data to the ECS task, you need to consider storing the payload in external storage or activity integration.\n* You can not send response back to the Step Functions workflow from the ECS task\u002FEKS job. You need to use mechanisms such as Amazon S3 or DynamoDB to store the data for later use in the Step Functions workflow or use `Wait for a Callback` or `Activity` integration.\n* In order to determine if the ECS task is complete, Step Functions call the ECS:DescribeTasks API regularly. At high volume, you might observe throttles on the DescribeTasks API. To avoid such throttles, you can remove ECS:DescribeTasks IAM permission from state machine’s execution role. Step Functions will use [Amazon ECS events for Amazon EventBridge](https:\u002F\u002Fdocs.aws.amazon.com\u002FAmazonECS\u002Flatest\u002Fdeveloperguide\u002Fecs_cwe_events.html) to determine if the task is complete.\n\n",[18],{"title":19,"collapsible":20,"isCollapsed":20,"content":21},"Integrating Step Functions with containerized workloads",false,[22,29,40,53,60,74,90],{"title":23,"order":24,"time":25,"path":26,"id":27,"link":28},"Introduction",10,"1 min","10-introduction","10-introduction.md","\u002Fcontent\u002Fservice\u002Fstep-functions\u002Fguides\u002Fintegrating-step-functions-with-containerized-workloads\u002F10-introduction",{"title":30,"order":31,"resources":32,"time":36,"path":37,"id":38,"link":39},"Service Integration patterns and AWS service integrations ",20,[33],{"link":34,"text":35,"type":12},"https:\u002F\u002Fdocs.aws.amazon.com\u002Fstep-functions\u002Flatest\u002Fdg\u002Fconnect-to-resource.html","Service integration patterns","2 min","20-patterns","20-patterns.md","\u002Fcontent\u002Fservice\u002Fstep-functions\u002Fguides\u002Fintegrating-step-functions-with-containerized-workloads\u002F20-patterns",{"title":41,"order":42,"resources":43,"time":25,"path":50,"id":51,"link":52},"Connecting to systems outside AWS",40,[44,47],{"link":45,"text":46,"type":12},"https:\u002F\u002Fdocs.aws.amazon.com\u002Fstep-functions\u002Flatest\u002Fdg\u002Fconnect-third-party-apis.html","Third party API integration",{"link":48,"text":49,"type":12},"https:\u002F\u002Fdocs.aws.amazon.com\u002Fstep-functions\u002Flatest\u002Fdg\u002Fconcepts-activities.html","Activity Integration","40-external-system-integration","40-external-system-integration.md","\u002Fcontent\u002Fservice\u002Fstep-functions\u002Fguides\u002Fintegrating-step-functions-with-containerized-workloads\u002F40-external-system-integration",{"title":6,"order":7,"resources":54,"time":36,"path":57,"id":58,"link":59},[55,56],{"link":10,"text":11,"type":12},{"link":14,"text":15,"type":12},"50-run-a-job","50-run-a-job.md","\u002Fcontent\u002Fservice\u002Fstep-functions\u002Fguides\u002Fintegrating-step-functions-with-containerized-workloads\u002F50-run-a-job",{"title":61,"order":62,"callout":63,"resources":66,"time":36,"path":71,"id":72,"link":73},"Wait for Callback integration",60,{"title":64,"description":64,"link":65},"Parallel video processing using AWS Fargate tasks","https:\u002F\u002Fcatalog.workshops.aws\u002Fstepfunctions\u002Fen-US\u002Fuse-cases\u002Fdata-processing\u002Fmap-ecs",[67],{"link":68,"text":69,"type":70},"https:\u002F\u002Fserverlessland.com\u002Fpatterns\u002Fsfn-ecs-sam","Sample pattern for Wait for Callback","pattern","60-callback","60-callback.md","\u002Fcontent\u002Fservice\u002Fstep-functions\u002Fguides\u002Fintegrating-step-functions-with-containerized-workloads\u002F60-callback",{"title":49,"order":75,"resources":76,"callout":83,"time":36,"path":87,"id":88,"link":89},70,[77,79],{"link":48,"text":78,"type":12},"Activities",{"link":80,"text":81,"type":82},"https:\u002F\u002Fcatalog.us-east-1.prod.workshops.aws\u002Fworkshops\u002F2a22e604-2f2e-4d7b-85a8-33b38c999234\u002Fen-US\u002Fuse-cases\u002Fmonte-carlo-simulation","Run Monte Carlo Simulation with AWS Fargate and Step Functions Activity","workshop",{"title":84,"description":85,"link":86},"Using Activities with AWS Fargate","This blog post demostrates how you can use Activities with AWS Fargate at scale.","https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Fcontainers\u002Frun-monte-carlo-simulations-at-scale-with-aws-step-functions-and-aws-fargate\u002F","70-activity","70-activity.md","\u002Fcontent\u002Fservice\u002Fstep-functions\u002Fguides\u002Fintegrating-step-functions-with-containerized-workloads\u002F70-activity",{"title":91,"order":24,"time":25,"path":92,"id":93,"link":94},"Conclusion","80-conclusion","80-conclusion.md","\u002Fcontent\u002Fservice\u002Fstep-functions\u002Fguides\u002Fintegrating-step-functions-with-containerized-workloads\u002F80-conclusion","LIST",[97],"content\u002Fcontributors\u002Fuma-ramadoss.json",1792665273930]