[{"data":1,"prerenderedAt":125},["ShallowReactive",2],{"content-svc-step-functions-paved-path-distributed-data-processing-with-step-functions-patterns-usecases":3},{"markdown":4,"frontMatterAttributes":5,"bodyRaw":21,"menu":22,"menuType":120,"isCollapseableMenu":25,"nextDocItem":88,"previousDocItem":51,"contributorPaths":121,"contentName":124,"slug":84,"readingTime":81},"\u003Cp>\u003Cstrong>Single source pattern\u003C\u002Fstrong>\u003C\u002Fp>\n\u003Cp>In a single source pattern, Distributed Map reads data from an Amazon S3 object or JSON array which is given as an input to the map. It then fans out highly concurrent processing by distributing batches of the data to child workflow defined within the map. With each iteration, the child workflow runs as many parallel workflows as configured with \u003Ccode>MaxConcurrency\u003C\u002Fcode> attribute. The results of the processing are aggregated and stored in S3 bucket for further processing.  \u003C\u002Fp>\n\u003Cp>\u003Cimg src=\"\u002Fassets\u002Fexternal\u002Fservice\u002Fstep-functions\u002Fpaved-path\u002Fdistributed-data-processing-with-step-functions\u002Fassets\u002Fsinglesource.png\" alt=\"\">\u003C\u002Fp>\n\u003Cp>Many organizations use flat files to aggregate, store, and transfer data across disparate systems. For example, a third party partner sends pricing data across all the products periodically in .csv file format. You need to process the records in the file at scale so that the required changes are processed on time. \u003C\u002Fp>\n\u003Cpre>\u003Ccode> "DistributedMap": {\n      "Type": "Map",\n      "ItemReader": {\n        "ReaderConfig": {\n          "InputType": "CSV",\n          "CSVHeaderLocation": "FIRST_ROW"\n        },\n        "Resource": "arn:aws:states:::s3:getObject",\n        "Parameters": {\n          "Bucket.$": "$.bucket",\n          "Key.$": "$.manifest"\n        }\n      } ...\n  }\n\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cp>You create an AWS Lambda function that processes one or more of these records. You call the Lambda function inside the Distributed Map with a batch of records using \u003Ccode>ItemBatcher\u003C\u002Fcode>. You also control how many instances of Lambda functions are invoked in parallel using \u003Ccode>MaxConcurrency\u003C\u002Fcode>.\u003C\u002Fp>\n\u003Cpre>\u003Ccode> "DistributedMap": {...\n        "StartAt": "Lambda Invoke",\n        "States": {\n          "Lambda Invoke": {\n            "Type": "Task",\n            "Resource": "arn:aws:states:::lambda:invoke",\n            "OutputPath": "$.Payload",\n            "Parameters": {\n              "Payload.$": "$",\n              "FunctionName": "arn:aws:lambda:us-east-1:1234567890:function:priceupdate:12"\n            },\n            "Retry": [\n              {\n                "ErrorEquals": [\n                  "Lambda.ServiceException",\n                  "Lambda.AWSLambdaException",\n                  "Lambda.SdkClientException",\n                  "Lambda.TooManyRequestsException"\n                ],\n                "IntervalSeconds": 2,\n                "MaxAttempts": 6,\n                "BackoffRate": 2\n              }\n            ],\n            "End": true\n          }\n        }\n      },\n      "MaxConcurrency": 1000,\n      "ItemBatcher": {\n        "MaxItemsPerBatch": 500\n      },\n      "Next": "Notifier"\n    }\n\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cp>The file can have up to 100M records or 10GB in size. The file type supported are .csv and JSON.\u003C\u002Fp>\n\u003Cp>\u003Cem>\u003Cstrong>Use cases\u003C\u002Fstrong>\u003C\u002Fem>\u003C\u002Fp>\n\u003Col>\n\u003Cli>Payroll processing for millions of associates with records stored in .csv or JSON format.\u003C\u002Fli>\n\u003Cli>Migrate data from one data store to another using a data export from S3.\u003C\u002Fli>\n\u003C\u002Fol>\n\u003Cp>To test out single source pattern in action, see the \u003Ca href=\"https:\u002F\u002Fserverlessland.com\u002Fworkflows\u002Fdistributed-map-state?ref=search\">Distributed Map state gif generator\u003C\u002Fa>.\u003C\u002Fp>\n\u003Cp>\u003Cstrong>Multi file pattern\u003C\u002Fstrong>\u003C\u002Fp>\n\u003Cp> In the multi file pattern, Distributed Map accepts S3 file prefix, collects metadata about files, and distributes the metadata to child workflow for further processing. With each iteration, the child workflow runs as many parallel workflows as configured with \u003Ccode>MaxConcurrency\u003C\u002Fcode> attribute. The results of the processing are aggregated and stored in an S3 bucket for further processing.  \u003C\u002Fp>\n\u003Cp>\u003Cimg src=\"\u002Fassets\u002Fexternal\u002Fservice\u002Fstep-functions\u002Fpaved-path\u002Fdistributed-data-processing-with-step-functions\u002Fassets\u002Fmultifile.png\" alt=\"\">\u003C\u002Fp>\n\u003Cp>Consider an example where sensors in manufacturing units send operational metrics every 5 minutes. You have as many files as the number of machines in the 5 minute period. At every 5 minute, you run am AWS Step Functions workflow that process all the files, identify anomalies, and take immediate action. \u003C\u002Fp>\n\u003Cpre>\u003Ccode>    "DistributedMap": {\n      "Type": "Map",\n      "ItemReader": {\n        "Resource": "arn:aws:states:::s3:listObjectsV2",\n        "Parameters": {\n          "Bucket": "$.sensor_data_bucket"\n          "Prefix.$": "$.fivemin_data_prefix"\n        }\n      }...\n\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cp>The resource provided in this pattern uses the \u003Ccode>listObjectsV2\u003C\u002Fcode> API whereas the single file pattern uses \u003Ccode>getObject\u003C\u002Fcode>. You write a Lambda function to run the calculation to detect anomalies.\u003C\u002Fp>\n\u003Cpre>\u003Ccode>        "States": {\n          "Lambda Invoke": {\n            "Type": "Task",\n            "Resource": "arn:aws:states:::lambda:invoke",\n            "OutputPath": "$.Payload",\n            "Parameters": {\n              "Payload.$": "$",\n              "FunctionName": "arn:aws:lambda:us-east-1:1234567890:function:process-sensordata--C1x7u12ryFZ2"\n            }....\n        }\n \n\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cp>The Lambda function in the child workflow receives one or more file metadata read from the configured bucket\u002Fprefix location.\u003C\u002Fp>\n\u003Cpre>\u003Ccode>{\n  "Items": [\n    {\n      "Etag": "\\"6653728b15bff8bd5d33ebd492500273\\"",\n      "Key": "98430099999_sensor1.xml",\n      "LastModified": 1675979708,\n      "Size": 8821,\n      "StorageClass": "STANDARD"\n    },\n    {\n      "Etag": "\\"ee80f36288e29747af43d081dc9d05a0\\"",\n      "Key": "98431099999_sensor2.xml",\n      "LastModified": 1675979708,\n      "Size": 9391,\n      "StorageClass": "STANDARD"\n    }\n    \n }\n\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cp>\u003Cem>\u003Cstrong>Use cases\u003C\u002Fstrong>\u003C\u002Fem>\u003C\u002Fp>\n\u003Cul>\n\u003Cli>Scan thousands of files for security vulnerabilities .\u003C\u002Fli>\n\u003Cli>Run an ML model inferencing to predict financial risk scores for your customers with input data stored in individual files in S3.\u003C\u002Fli>\n\u003Cli>Backfill data transformation for past dates when you onboard a new customer.\u003C\u002Fli>\n\u003C\u002Ful>\n\u003Cp>To test out the multi file pattern, see the \u003Ca href=\"https:\u002F\u002Fcatalog.workshops.aws\u002Fstepfunctions\u002Fen-US\u002Fuse-cases\u002Fdata-processing\u002Fdistributed-map\">Large-Scale Parallelization with Distributed Map workshop\u003C\u002Fa>. This is a step-by-step example of how to use a Step Functions workflow in Distributed mode to process a list of S3 objects in an S3 bucket.\u003C\u002Fp>\n\u003Cp>\u003Cstrong>Nested single source pattern\u003C\u002Fstrong>\u003C\u002Fp>\n\u003Cp>In a nested single source pattern, Distributed Map is inside another Distributed Map. The outer map reads metadata of multiple files from an S3 bucket, and sends batches of metadata to the inner map. Inner map reads the individual file records and runs child workflows to process them. The data can be aggregated at each Distributed Map for further processing.\u003C\u002Fp>\n\u003Cp>\u003Cimg src=\"\u002Fassets\u002Fexternal\u002Fservice\u002Fstep-functions\u002Fpaved-path\u002Fdistributed-data-processing-with-step-functions\u002Fassets\u002Fnested.png\" alt=\"\">\u003C\u002Fp>\n\u003Cp>Imagine you are getting end of day sales records from point of sales systems across your restaurants. Each individual files has 10s of 1000s of sales data for a single restaurant. You need to clean, verify, process payments, and generate a report. \u003C\u002Fp>\n\u003Cpre>\u003Ccode>    "OuterMap to iterate sales across restaurants": {\n      "Type": "Map",\n      "ItemProcessor": {\n        "ProcessorConfig": {\n          "Mode": "DISTRIBUTED",\n          "ExecutionType": "STANDARD"\n        },\n        "StartAt": "InnerMap to iterate sales records",\n        "States": {\n          "InnerMap to iterate sales records": {\n            "Type": "Map",\n            "ItemProcessor": {\n              "ProcessorConfig": {\n                "Mode": "DISTRIBUTED",\n                "ExecutionType": "EXPRESS"\n              },...\n\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cp>The outer map iterates through each S3 objects located at the prefix using S3 \u003Ccode>listObject\u003C\u002Fcode>, sends the file metadata to the child workflow which is a Distributed Map itself.\u003C\u002Fp>\n\u003Cpre>\u003Ccode>      "ItemReader": {\n        "Resource": "arn:aws:states:::s3:listObjectsV2",\n        "Parameters": {\n          "Bucket": "endofday_sales",\n          "Prefix.$": "$.day"\n        }\n\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cp>The inner map reads a single S3 object from the outer map, iterates on the content which is sales records, and sends it to the Lambda function in the child workflow.\u003C\u002Fp>\n\u003Cpre>\u003Ccode>            "ItemReader": {\n              "ReaderConfig": {\n                "InputType": "CSV",\n                "CSVHeaderLocation": "FIRST_ROW"\n              },\n              "Resource": "arn:aws:states:::s3:getObject",\n              "Parameters": {\n                "Bucket": "endofday_sales",\n                "Key.$": "$.Key"\n              }\n            }\n\u003C\u002Fcode>\u003C\u002Fpre>\n\u003Cp>\u003Cem>\u003Cstrong>Use cases\u003C\u002Fstrong>\u003C\u002Fem>\u003C\u002Fp>\n\u003Cul>\n\u003Cli>End of day financial transactions from multiple locations that need to be processed, verified for fraud, reviewed manually by agent if needed.\u003C\u002Fli>\n\u003Cli>Process thousands of invoices each consisting of 100s of sales records.\u003C\u002Fli>\n\u003C\u002Ful>\n",{"title":6,"order":7,"callout":8,"learningGoals":12,"resources":14},"Patterns and Use cases",3,{"title":9,"description":10,"link":11},"Workshop","Hands-on experience on building multiple real-world use cases of distributed map","https:\u002F\u002Fcatalog.us-east-1.prod.workshops.aws\u002Fworkshops\u002F2a22e604-2f2e-4d7b-85a8-33b38c999234\u002Fen-US",[13],"Learn different patterns and explore use cases",[15,17],{"link":11,"text":16,"type":9},"Distributed Map workshop",{"link":18,"text":19,"type":20},"https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=Fp-F8ehBUFY","Advanced serverless workflow patterns","video","**Single source pattern**\n \nIn a single source pattern, Distributed Map reads data from an Amazon S3 object or JSON array which is given as an input to the map. It then fans out highly concurrent processing by distributing batches of the data to child workflow defined within the map. With each iteration, the child workflow runs as many parallel workflows as configured with `MaxConcurrency` attribute. The results of the processing are aggregated and stored in S3 bucket for further processing.  \n\n![](\u002Fassets\u002Fexternal\u002Fservice\u002Fstep-functions\u002Fpaved-path\u002Fdistributed-data-processing-with-step-functions\u002Fassets\u002Fsinglesource.png)\n\n\nMany organizations use flat files to aggregate, store, and transfer data across disparate systems. For example, a third party partner sends pricing data across all the products periodically in .csv file format. You need to process the records in the file at scale so that the required changes are processed on time. \n\n```\n \"DistributedMap\": {\n      \"Type\": \"Map\",\n      \"ItemReader\": {\n        \"ReaderConfig\": {\n          \"InputType\": \"CSV\",\n          \"CSVHeaderLocation\": \"FIRST_ROW\"\n        },\n        \"Resource\": \"arn:aws:states:::s3:getObject\",\n        \"Parameters\": {\n          \"Bucket.$\": \"$.bucket\",\n          \"Key.$\": \"$.manifest\"\n        }\n      } ...\n  }\n```\nYou create an AWS Lambda function that processes one or more of these records. You call the Lambda function inside the Distributed Map with a batch of records using `ItemBatcher`. You also control how many instances of Lambda functions are invoked in parallel using `MaxConcurrency`.\n```\n \"DistributedMap\": {...\n        \"StartAt\": \"Lambda Invoke\",\n        \"States\": {\n          \"Lambda Invoke\": {\n            \"Type\": \"Task\",\n            \"Resource\": \"arn:aws:states:::lambda:invoke\",\n            \"OutputPath\": \"$.Payload\",\n            \"Parameters\": {\n              \"Payload.$\": \"$\",\n              \"FunctionName\": \"arn:aws:lambda:us-east-1:1234567890:function:priceupdate:12\"\n            },\n            \"Retry\": [\n              {\n                \"ErrorEquals\": [\n                  \"Lambda.ServiceException\",\n                  \"Lambda.AWSLambdaException\",\n                  \"Lambda.SdkClientException\",\n                  \"Lambda.TooManyRequestsException\"\n                ],\n                \"IntervalSeconds\": 2,\n                \"MaxAttempts\": 6,\n                \"BackoffRate\": 2\n              }\n            ],\n            \"End\": true\n          }\n        }\n      },\n      \"MaxConcurrency\": 1000,\n      \"ItemBatcher\": {\n        \"MaxItemsPerBatch\": 500\n      },\n      \"Next\": \"Notifier\"\n    }\n\n```\nThe file can have up to 100M records or 10GB in size. The file type supported are .csv and JSON.\n\n\n***Use cases***\n1. Payroll processing for millions of associates with records stored in .csv or JSON format.\n2. Migrate data from one data store to another using a data export from S3.\n\nTo test out single source pattern in action, see the [Distributed Map state gif generator](https:\u002F\u002Fserverlessland.com\u002Fworkflows\u002Fdistributed-map-state?ref=search).\n\n**Multi file pattern**\n\n In the multi file pattern, Distributed Map accepts S3 file prefix, collects metadata about files, and distributes the metadata to child workflow for further processing. With each iteration, the child workflow runs as many parallel workflows as configured with `MaxConcurrency` attribute. The results of the processing are aggregated and stored in an S3 bucket for further processing.  \n\n![](\u002Fassets\u002Fexternal\u002Fservice\u002Fstep-functions\u002Fpaved-path\u002Fdistributed-data-processing-with-step-functions\u002Fassets\u002Fmultifile.png)\n\nConsider an example where sensors in manufacturing units send operational metrics every 5 minutes. You have as many files as the number of machines in the 5 minute period. At every 5 minute, you run am AWS Step Functions workflow that process all the files, identify anomalies, and take immediate action. \n\n```\n    \"DistributedMap\": {\n      \"Type\": \"Map\",\n      \"ItemReader\": {\n        \"Resource\": \"arn:aws:states:::s3:listObjectsV2\",\n        \"Parameters\": {\n          \"Bucket\": \"$.sensor_data_bucket\"\n          \"Prefix.$\": \"$.fivemin_data_prefix\"\n        }\n      }...\n```\nThe resource provided in this pattern uses the `listObjectsV2` API whereas the single file pattern uses `getObject`. You write a Lambda function to run the calculation to detect anomalies.\n```\n        \"States\": {\n          \"Lambda Invoke\": {\n            \"Type\": \"Task\",\n            \"Resource\": \"arn:aws:states:::lambda:invoke\",\n            \"OutputPath\": \"$.Payload\",\n            \"Parameters\": {\n              \"Payload.$\": \"$\",\n              \"FunctionName\": \"arn:aws:lambda:us-east-1:1234567890:function:process-sensordata--C1x7u12ryFZ2\"\n            }....\n        }\n \n```\nThe Lambda function in the child workflow receives one or more file metadata read from the configured bucket\u002Fprefix location.\n\n```\n{\n  \"Items\": [\n    {\n      \"Etag\": \"\\\"6653728b15bff8bd5d33ebd492500273\\\"\",\n      \"Key\": \"98430099999_sensor1.xml\",\n      \"LastModified\": 1675979708,\n      \"Size\": 8821,\n      \"StorageClass\": \"STANDARD\"\n    },\n    {\n      \"Etag\": \"\\\"ee80f36288e29747af43d081dc9d05a0\\\"\",\n      \"Key\": \"98431099999_sensor2.xml\",\n      \"LastModified\": 1675979708,\n      \"Size\": 9391,\n      \"StorageClass\": \"STANDARD\"\n    }\n    \n }\n```\n\n\n***Use cases***\n* Scan thousands of files for security vulnerabilities .\n* Run an ML model inferencing to predict financial risk scores for your customers with input data stored in individual files in S3.\n* Backfill data transformation for past dates when you onboard a new customer.\n\n\nTo test out the multi file pattern, see the [Large-Scale Parallelization with Distributed Map workshop](https:\u002F\u002Fcatalog.workshops.aws\u002Fstepfunctions\u002Fen-US\u002Fuse-cases\u002Fdata-processing\u002Fdistributed-map). This is a step-by-step example of how to use a Step Functions workflow in Distributed mode to process a list of S3 objects in an S3 bucket.\n\n**Nested single source pattern**\n\nIn a nested single source pattern, Distributed Map is inside another Distributed Map. The outer map reads metadata of multiple files from an S3 bucket, and sends batches of metadata to the inner map. Inner map reads the individual file records and runs child workflows to process them. The data can be aggregated at each Distributed Map for further processing.\n\n![](\u002Fassets\u002Fexternal\u002Fservice\u002Fstep-functions\u002Fpaved-path\u002Fdistributed-data-processing-with-step-functions\u002Fassets\u002Fnested.png)\n\nImagine you are getting end of day sales records from point of sales systems across your restaurants. Each individual files has 10s of 1000s of sales data for a single restaurant. You need to clean, verify, process payments, and generate a report. \n```\n    \"OuterMap to iterate sales across restaurants\": {\n      \"Type\": \"Map\",\n      \"ItemProcessor\": {\n        \"ProcessorConfig\": {\n          \"Mode\": \"DISTRIBUTED\",\n          \"ExecutionType\": \"STANDARD\"\n        },\n        \"StartAt\": \"InnerMap to iterate sales records\",\n        \"States\": {\n          \"InnerMap to iterate sales records\": {\n            \"Type\": \"Map\",\n            \"ItemProcessor\": {\n              \"ProcessorConfig\": {\n                \"Mode\": \"DISTRIBUTED\",\n                \"ExecutionType\": \"EXPRESS\"\n              },...\n```\nThe outer map iterates through each S3 objects located at the prefix using S3 `listObject`, sends the file metadata to the child workflow which is a Distributed Map itself.\n```\n      \"ItemReader\": {\n        \"Resource\": \"arn:aws:states:::s3:listObjectsV2\",\n        \"Parameters\": {\n          \"Bucket\": \"endofday_sales\",\n          \"Prefix.$\": \"$.day\"\n        }\n```\nThe inner map reads a single S3 object from the outer map, iterates on the content which is sales records, and sends it to the Lambda function in the child workflow.\n```\n            \"ItemReader\": {\n              \"ReaderConfig\": {\n                \"InputType\": \"CSV\",\n                \"CSVHeaderLocation\": \"FIRST_ROW\"\n              },\n              \"Resource\": \"arn:aws:states:::s3:getObject\",\n              \"Parameters\": {\n                \"Bucket\": \"endofday_sales\",\n                \"Key.$\": \"$.Key\"\n              }\n            }\n```\n\n\n***Use cases***\n* End of day financial transactions from multiple locations that need to be processed, verified for fraud, reviewed manually by agent if needed.\n* Process thousands of invoices each consisting of 100s of sales records.\n",[23,48,72,85,98],{"title":24,"collapsible":25,"isCollapsed":26,"content":27},"Introduction",false,true,[28],{"title":29,"order":30,"youtubeVideoID":31,"learningGoals":32,"resources":34,"time":44,"path":45,"id":46,"link":47},"Introduction to Distributed Map for Serverless Data Processing",1,"SG6_oy72hh4",[33],"Understand the benefits of distributed data processing with Serverless.",[35,39,43],{"link":36,"text":37,"type":38},"https:\u002F\u002Faws.amazon.com\u002Fstep-functions\u002F","Learn more about AWS Step Functions","Text",{"link":40,"text":41,"type":42},"https:\u002F\u002Fserverlessland.com\u002Fsearch?search=Distributed+Map","Distributed Map resources on Serverless Land","Samples",{"link":11,"text":16,"type":9},"3 min","introduction","introduction.md","\u002Fcontent\u002Fservice\u002Fstep-functions\u002Fpaved-path\u002Fdistributed-data-processing-with-step-functions\u002Fintroduction",{"title":49,"collapsible":25,"isCollapsed":26,"content":50},"Getting Started",[51],{"title":52,"order":53,"youtubeVideoID":31,"learningGoals":54,"callout":56,"resources":57,"time":68,"path":69,"id":70,"link":71},"Step Functions - Distributed Map",2,[55],"Introduction to Distributed Map.",{"title":9,"description":10,"link":11},[58,62,66],{"link":59,"text":60,"type":61},"https:\u002F\u002Fdocs.aws.amazon.com\u002Fstep-functions\u002Flatest\u002Fdg\u002Fconcepts-asl-use-map-state-distributed.html","Distributed Map concepts","text",{"link":63,"text":64,"type":65},"https:\u002F\u002Fcatalog.workshops.aws\u002Fstepfunctions\u002Fen-US\u002Fuse-cases\u002Fdata-processing\u002Fdistributed-map","Workshop on Distributed Map","workshop",{"link":40,"text":41,"type":67},"samples","2 min","getting-started","getting-started.md","\u002Fcontent\u002Fservice\u002Fstep-functions\u002Fpaved-path\u002Fdistributed-data-processing-with-step-functions\u002Fgetting-started",{"title":73,"collapsible":25,"isCollapsed":25,"content":74},"Patterns and Usecases",[75],{"title":6,"order":7,"callout":76,"learningGoals":77,"resources":78,"time":81,"path":82,"id":83,"link":84},{"title":9,"description":10,"link":11},[13],[79,80],{"link":11,"text":16,"type":9},{"link":18,"text":19,"type":20},"5 min","patterns-usecases","patterns-usecases.md","\u002Fcontent\u002Fservice\u002Fstep-functions\u002Fpaved-path\u002Fdistributed-data-processing-with-step-functions\u002Fpatterns-usecases",{"title":86,"collapsible":25,"isCollapsed":26,"content":87},"Performance",[88],{"title":86,"order":89,"learningGoals":90,"callout":92,"time":81,"path":95,"id":96,"link":97},4,[91],"Learn different performance optimization techniques",{"title":9,"description":93,"link":94},"Learn different optimization techniques hands-on using the workshop","https:\u002F\u002Fcatalog.us-east-1.prod.workshops.aws\u002Fworkshops\u002F2a22e604-2f2e-4d7b-85a8-33b38c999234\u002Fen-US\u002Fadvanced\u002Foptimization","performance","performance.md","\u002Fcontent\u002Fservice\u002Fstep-functions\u002Fpaved-path\u002Fdistributed-data-processing-with-step-functions\u002Fperformance",{"title":99,"collapsible":25,"isCollapsed":26,"content":100},"Observability",[101],{"title":99,"order":102,"learningGoals":103,"prerequisite":105,"callout":107,"resources":110,"time":68,"path":117,"id":118,"link":119},5,[104],"Learn different techniques and configurations to observe workflows",[106],"Understanding AWS Step Functions",{"title":9,"description":108,"link":109},"Step by step instruction on AWS Step Functions observability","https:\u002F\u002Fcatalog.workshops.aws\u002Fstepfunctions\u002Fen-US\u002Foperations\u002Fobservability",[111,114,116],{"link":112,"text":113,"type":61},"https:\u002F\u002Fdocs.aws.amazon.com\u002Fstep-functions\u002Flatest\u002Fdg\u002Fmonitoring-logging.html","Logging and Monitoring for AWS Step Functions",{"link":40,"text":41,"type":115},"sample",{"link":11,"text":16,"type":9},"observability","observability.md","\u002Fcontent\u002Fservice\u002Fstep-functions\u002Fpaved-path\u002Fdistributed-data-processing-with-step-functions\u002Fobservability","LESSON",[122,123],"content\u002Fcontributors\u002Fuma-ramadoss.json","content\u002Fcontributors\u002Fadam-wagner.json","Distributed data processing with Step Functions",1792665274008]