[{"data":1,"prerenderedAt":229},["ShallowReactive",2],{"content-svc-dynamodb-guides-using-dynamodbto-build-webapps-10-dynamodb-migrate":3},{"markdown":4,"frontMatterAttributes":5,"bodyRaw":22,"menu":23,"menuType":225,"isCollapseableMenu":26,"nextDocItem":165,"previousDocItem":147,"contributorPaths":226,"contentName":228,"slug":164,"readingTime":80},"\u003Ch1>How to Migrate Data to DynamoDB?\u003C\u002Fh1>\n\u003Cp>Once you have the infrastructure ready and your application coded, if you are performing a migration, it's time to start converting and migrating your data. The following section will provide you with a set of tips to help you build a migration plan.\u003C\u002Fp>\n\u003Ch2>AWS Database Migration Service (AWS DMS)\u003C\u002Fh2>\n\u003Cp>The recommended option for migrating your data is to use \u003Ca href=\"https:\u002F\u002Faws.amazon.com\u002Fdms\u002F\">AWS Database Migration Service (DMS)\u003C\u002Fa>. This service is designed to facilitate database migrations and can be a valuable tool for your migration process.\u003C\u002Fp>\n\u003Cp>However, if you are using the single table design approach, AWS DMS might not be the best fit, as it creates separate tables for each table in the source database. If you are migrating from MongoDB and still want to explore using AWS DMS, I have included a detailed blog post in the resources that explains how to perform a live migration from a MongoDB cluster to Amazon DynamoDB.\u003C\u002Fp>\n\u003Ch2>Custom Script\u003C\u002Fh2>\n\u003Cp>Due to the unique structure of your application table using the single table design, you will need to create your own customizable script for migration.\u003C\u002Fp>\n\u003Cp>Your script will consist of two parts: retrieving the data from the original database and saving the items into the DynamoDB table.\u003C\u002Fp>\n\u003Ch3>Retrieving the data from the source\u003C\u002Fh3>\n\u003Cp>Most databases allow you to export data in formats such as JSON or CSV. If that is the case, you can store those exported items in Amazon S3 and then upload them to DynamoDB.\u003C\u002Fp>\n\u003Cp>For databases with custom export formats, you may need to build a script that performs the data extraction for you.\u003C\u002Fp>\n\u003Cp>The result of this step is having one file per entity in Amazon S3, and potentially some additional files for relationships if you are migrating from a relational database.\u003C\u002Fp>\n\u003Ch3>Importing the data to DynamoDB\u003C\u002Fh3>\n\u003Cp>You have a couple of options for importing the data:\u003C\u002Fp>\n\u003Col>\n\u003Cli>Create a table file in Amazon S3 and import it into DynamoDB.\u003C\u002Fli>\n\u003Cli>Process the items one by one and upload them to DynamoDB.\u003C\u002Fli>\n\u003C\u002Fol>\n\u003Cp>DynamoDB provides a feature that allows you to import data from Amazon S3 into a table, handling capacity management and data import for you. With this feature, you can import a file stored in Amazon S3, formatted like the DynamoDB table, into DynamoDB. However, note that this feature requires creating a new table; you cannot import data into an existing table.\u003C\u002Fp>\n\u003Cp>To build the table file for direct import into DynamoDB or to iterate over the dump files from the database, you will need to create a process to handle that. You can leverage \u003Ca href=\"https:\u002F\u002Faws.amazon.com\u002Fstep-functions\">AWS Step Functions\u003C\u002Fa> distributed map, which is a state type in Step Functions that iterates over items in a JSON or CSV file and executes a set of states. The maximum parallelization with Distributed map is 10,000.\u003C\u002Fp>\n",{"title":6,"order":7,"youtubeVideoID":8,"resources":9},"How to Migrate Data to DynamoDB?",11,"qEQqL3KiOOc",[10,14,18],{"link":11,"text":12,"type":13},"https:\u002F\u002Fdocs.aws.amazon.com\u002Famazondynamodb\u002Flatest\u002Fdeveloperguide\u002FS3DataImport.HowItWorks.html","DynamoDB data import from Amazon S3: how it works","docs",{"link":15,"text":16,"type":17},"https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Fdatabase\u002Fperforming-a-live-migration-from-a-mongodb-cluster-to-amazon-dynamodb\u002F","Performing a Live Migration from a MongoDB Cluster to Amazon DynamoDB","blog",{"link":19,"type":20,"text":21},"https:\u002F\u002Fyoutu.be\u002F_6ym3IRrRxo","video","🆕 Step functions Distributed map - Run 10k parallel serverless executions!","# How to Migrate Data to DynamoDB?\n\nOnce you have the infrastructure ready and your application coded, if you are performing a migration, it's time to start converting and migrating your data. The following section will provide you with a set of tips to help you build a migration plan.\n\n## AWS Database Migration Service (AWS DMS)\n\nThe recommended option for migrating your data is to use [AWS Database Migration Service (DMS)](https:\u002F\u002Faws.amazon.com\u002Fdms\u002F). This service is designed to facilitate database migrations and can be a valuable tool for your migration process.\n\nHowever, if you are using the single table design approach, AWS DMS might not be the best fit, as it creates separate tables for each table in the source database. If you are migrating from MongoDB and still want to explore using AWS DMS, I have included a detailed blog post in the resources that explains how to perform a live migration from a MongoDB cluster to Amazon DynamoDB.\n\n## Custom Script\n\nDue to the unique structure of your application table using the single table design, you will need to create your own customizable script for migration.\n\nYour script will consist of two parts: retrieving the data from the original database and saving the items into the DynamoDB table.\n\n### Retrieving the data from the source\n\nMost databases allow you to export data in formats such as JSON or CSV. If that is the case, you can store those exported items in Amazon S3 and then upload them to DynamoDB.\n\nFor databases with custom export formats, you may need to build a script that performs the data extraction for you.\n\nThe result of this step is having one file per entity in Amazon S3, and potentially some additional files for relationships if you are migrating from a relational database.\n\n### Importing the data to DynamoDB\n\nYou have a couple of options for importing the data:\n\n1. Create a table file in Amazon S3 and import it into DynamoDB.\n2. Process the items one by one and upload them to DynamoDB.\n\nDynamoDB provides a feature that allows you to import data from Amazon S3 into a table, handling capacity management and data import for you. With this feature, you can import a file stored in Amazon S3, formatted like the DynamoDB table, into DynamoDB. However, note that this feature requires creating a new table; you cannot import data into an existing table.\n\nTo build the table file for direct import into DynamoDB or to iterate over the dump files from the database, you will need to create a process to handle that. You can leverage [AWS Step Functions](https:\u002F\u002Faws.amazon.com\u002Fstep-functions) distributed map, which is a state type in Step Functions that iterates over items in a JSON or CSV file and executes a set of states. The maximum parallelization with Distributed map is 10,000.\n",[24,35,65,99,133,179,217],{"title":25,"collapsible":26,"isCollapsed":27,"content":28},"Introduction",false,true,[29],{"title":25,"order":30,"time":31,"path":32,"id":33,"link":34},1,"2 min","introduction","introduction.md","\u002Fcontent\u002Fservice\u002Fdynamodb\u002Fguides\u002Fusing-dynamodbto-build-webapps\u002Fintroduction",{"title":36,"collapsible":26,"isCollapsed":27,"content":37},"1. Hosting Your Application with App Runner",[38,50],{"title":39,"order":40,"youtubeVideoID":41,"resources":42,"time":31,"path":47,"id":48,"link":49},"What is AWS App Runner?",2,"z-RywG7Xfk8",[43],{"link":44,"text":45,"type":46},"https:\u002F\u002Faws.amazon.com\u002FApp Runner\u002F","AWS App Runner","service","01-apprunnerintro","01-apprunnerintro.md","\u002Fcontent\u002Fservice\u002Fdynamodb\u002Fguides\u002Fusing-dynamodbto-build-webapps\u002F01-apprunnerintro",{"title":51,"order":52,"youtubeVideoID":53,"resources":54,"time":31,"path":62,"id":63,"link":64},"AWS AppRunner Demo",3,"jOwQdXbmHC4",[55,59],{"link":56,"text":57,"type":58},"https:\u002F\u002Fgithub.com\u002Fmavi888\u002Fnodeexpress-backend-app","NodeJS application code","code",{"link":60,"type":58,"text":61},"https:\u002F\u002Fgithub.com\u002Fmavi888\u002Fcdk-migrated-app-infra\u002Ftree\u002Fapprunner","AWS CDK infrastructure code","02-apprunnerdemo","02-apprunnerdemo.md","\u002Fcontent\u002Fservice\u002Fdynamodb\u002Fguides\u002Fusing-dynamodbto-build-webapps\u002F02-apprunnerdemo",{"title":66,"collapsible":26,"isCollapsed":27,"content":67},"2.Understanding DynamoDB Basics",[68,84],{"title":69,"order":70,"youtubeVideoID":71,"resources":72,"time":80,"path":81,"id":82,"link":83},"Amazon DynamoDB 101",4,"tkj2r7kFcpM",[73,76],{"link":74,"text":75,"type":46},"https:\u002F\u002Faws.amazon.com\u002Fdynamodb\u002F","Amazon DynamoDB service page",{"link":77,"type":78,"text":79},"https:\u002F\u002Faws.amazon.com\u002Fdynamodb\u002Fpricing\u002F","pricing","Amazon DynamoDB pricing","3 min","03-dynamodb-intro","03-dynamodb-intro.md","\u002Fcontent\u002Fservice\u002Fdynamodb\u002Fguides\u002Fusing-dynamodbto-build-webapps\u002F03-dynamodb-intro",{"title":85,"order":86,"youtubeVideoID":87,"resources":88,"time":31,"path":96,"id":97,"link":98},"Amazon DynamoDB under the hood",5,"ZhXqNcbR4n0",[89,93],{"link":90,"text":91,"type":92},"https:\u002F\u002Fwww.usenix.org\u002Fconference\u002Fatc22\u002Fpresentation\u002Felhemali","Amazon DynamoDB: A Scalable, Predictably Performant, and Fully Managed NoSQL Database Service","article",{"link":94,"type":92,"text":95},"https:\u002F\u002Fwww.theregister.com\u002F2022\u002F11\u002F14\u002Fmeet_the_database_that_supports\u002F","Meet the database that supports Amazon.com","04-dynamodb-underthehood","04-dynamodb-underthehood.md","\u002Fcontent\u002Fservice\u002Fdynamodb\u002Fguides\u002Fusing-dynamodbto-build-webapps\u002F04-dynamodb-underthehood",{"title":100,"collapsible":26,"isCollapsed":27,"content":101},"3.Designing Your DynamoDB Table",[102,113,124],{"title":103,"order":104,"youtubeVideoID":105,"resources":106,"time":80,"path":110,"id":111,"link":112},"Is DynamoDB the right database for you?",6,"TpeP3Cmz5PE",[107],{"link":108,"text":109,"type":92},"https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Fdatabase\u002Fhow-to-determine-if-amazon-dynamodb-is-appropriate-for-your-needs-and-then-plan-your-migration\u002F","How to determine if Amazon DynamoDB is appropriate for your needs, and then plan your migration","05-dynamo-rightforyou","05-dynamo-rightforyou.md","\u002Fcontent\u002Fservice\u002Fdynamodb\u002Fguides\u002Fusing-dynamodbto-build-webapps\u002F05-dynamo-rightforyou",{"title":114,"order":115,"youtubeVideoID":116,"resources":117,"time":31,"path":121,"id":122,"link":123},"Introduction to Single Table Design",7,"lWCch8GEK4E",[118],{"link":119,"text":120,"type":20},"https:\u002F\u002Fyoutu.be\u002FQ6-qWdsa8a4","7 Common DynamoDB Patterns for Modeling and Building an App with Alex De Brie","06-dynamodb-singletable","06-dynamodb-singletable.md","\u002Fcontent\u002Fservice\u002Fdynamodb\u002Fguides\u002Fusing-dynamodbto-build-webapps\u002F06-dynamodb-singletable",{"title":125,"order":126,"youtubeVideoID":127,"resources":128,"time":31,"path":130,"id":131,"link":132},"Design a DynamoDB table with Single Table Design",8,"JLZOI8patlw",[129],{"link":119,"text":120,"type":20},"07-dynamodb-singletable-modeling","07-dynamodb-singletable-modeling.md","\u002Fcontent\u002Fservice\u002Fdynamodb\u002Fguides\u002Fusing-dynamodbto-build-webapps\u002F07-dynamodb-singletable-modeling",{"title":134,"collapsible":26,"isCollapsed":26,"content":135},"4.Implementing DynamoDB in Your Application",[136,147,157,165],{"title":137,"order":138,"youtubeVideoID":139,"resources":140,"time":31,"path":144,"id":145,"link":146},"Provision Amazon DynamoDB table and indexes with AWS CDK",9,"4o1KzB2AAk4",[141],{"link":142,"text":143,"type":58},"https:\u002F\u002Fgithub.com\u002Fmavi888\u002Fcdk-migrated-app-infra\u002Ftree\u002FdynamoDB\u002Flib","AWS CDK code","08-dynamo-cdkinfra","08-dynamo-cdkinfra.md","\u002Fcontent\u002Fservice\u002Fdynamodb\u002Fguides\u002Fusing-dynamodbto-build-webapps\u002F08-dynamo-cdkinfra",{"title":148,"order":149,"youtubeVideoID":150,"resources":151,"time":80,"path":154,"id":155,"link":156},"Code your application",10,"9ftDmRMEfDM",[152],{"link":56,"text":153,"type":58},"Get the code","09-dynamodb-app","09-dynamodb-app.md","\u002Fcontent\u002Fservice\u002Fdynamodb\u002Fguides\u002Fusing-dynamodbto-build-webapps\u002F09-dynamodb-app",{"title":6,"order":7,"youtubeVideoID":8,"resources":158,"time":80,"path":162,"id":163,"link":164},[159,160,161],{"link":11,"text":12,"type":13},{"link":15,"text":16,"type":17},{"link":19,"type":20,"text":21},"10-dynamodb-migrate","10-dynamodb-migrate.md","\u002Fcontent\u002Fservice\u002Fdynamodb\u002Fguides\u002Fusing-dynamodbto-build-webapps\u002F10-dynamodb-migrate",{"title":166,"order":167,"youtubeVideoID":41,"resources":168,"time":31,"path":176,"id":177,"link":178},"Deploy and Load Test",12,[169,170,173],{"link":142,"text":153,"type":58},{"link":171,"type":20,"text":172},"https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=B0pDPSQcAf4","Load test your serverless applications | Run Artillery in a Lambda function",{"link":174,"type":20,"text":175},"https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=0VNKHIcQ5wk","Building Amazon CloudWatch dashboards with AWS CDK","11-dynamodb-test","11-dynamodb-test.md","\u002Fcontent\u002Fservice\u002Fdynamodb\u002Fguides\u002Fusing-dynamodbto-build-webapps\u002F11-dynamodb-test",{"title":180,"collapsible":26,"isCollapsed":27,"content":181},"5.Adding Event-Driven Characteristics",[182,190,204],{"title":183,"order":184,"youtubeVideoID":185,"time":186,"path":187,"id":188,"link":189},"DynamoDB Streams",13,"M0GJzjsw5So","1 min","12-dynamo-streams","12-dynamo-streams.md","\u002Fcontent\u002Fservice\u002Fdynamodb\u002Fguides\u002Fusing-dynamodbto-build-webapps\u002F12-dynamo-streams",{"title":191,"order":192,"youtubeVideoID":193,"resources":194,"time":186,"path":201,"id":202,"link":203},"Acting on Stream Events",14,"RL0l2Wz3deQ",[195,198],{"link":196,"text":197,"type":58},"https:\u002F\u002Fgithub.com\u002Fmavi888\u002Fcdk-migrated-app-infra","Code Repository",{"link":199,"type":20,"text":200},"https:\u002F\u002Fyoutu.be\u002F8Yv_V4havbc","Unlock the Power of EventBridge Pipes: Integrate Across Platforms with Ease!","13-dynamodb-ebpipes","13-dynamodb-ebpipes.md","\u002Fcontent\u002Fservice\u002Fdynamodb\u002Fguides\u002Fusing-dynamodbto-build-webapps\u002F13-dynamodb-ebpipes",{"title":205,"order":206,"youtubeVideoID":207,"resources":208,"time":186,"path":214,"id":215,"link":216},"Filtering Subscription Messages",15,"36iMOJQUAuE",[209,212,213],{"link":210,"text":211,"type":17},"https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Fcompute\u002Fintroducing-payload-based-message-filtering-for-amazon-sns\u002F","Introducing payload-based message filtering for Amazon SNS",{"link":196,"text":197,"type":58},{"link":199,"type":20,"text":200},"14-dynamodb-sns","14-dynamodb-sns.md","\u002Fcontent\u002Fservice\u002Fdynamodb\u002Fguides\u002Fusing-dynamodbto-build-webapps\u002F14-dynamodb-sns",{"title":218,"collapsible":26,"isCollapsed":27,"content":219},"Conclusion",[220],{"title":218,"order":221,"time":186,"path":222,"id":223,"link":224},16,"conclusion","conclusion.md","\u002Fcontent\u002Fservice\u002Fdynamodb\u002Fguides\u002Fusing-dynamodbto-build-webapps\u002Fconclusion","LIST",[227],"content\u002Fcontributors\u002Fmarcia-villalba.json","Using DynamoDB to build WebApps",1792665265822]