5 tips for migrating data from MongoDB to Amazon DocumentDB
Trying to migrate data from your existing MongoDB instance into an Amazon DocumentDB cluster? Here’s how you should go about it.
Jun 08, 2023 • 4 Minute Read
Trying to migrate data from your existing MongoDB instance into an Amazon DocumentDB cluster? It can be a very complex process, but it doesn't need to be. In this post, we share five tips on how to make this process simpler, and reduce the impact on your existing production application.
1. Identify Your Sources
Before beginning any migration, it's important you gather information about the sources of information you'll be migrating to the Amazon DocumentDB cluster.
Make sure to catalog specific information about the sources such as the application owner and the version of MongoDB. If you've got a service level agreement (SLA) that applies to the application, capture this as well.
Having all of this information will ensure you're prepared for the next steps in your migration.
2. Use a VPN or AWS Direct Connect
In the simplest case, the MongoDB service is running with the same Virtual Private Cloud (VPC) as the DocumentDB cluster. If this isn't the case, you'll need to use a virtual private network (VPN) or the AWS Direct Connect service to migrate data from your on-premise MongoDB cluster into the Amazon DocumentDB cluster.
Because of this restriction, it's important to note where the source databases are located when identifying sources. Make sure to include any necessary credentials to access the source as well.
3. Use mongodump and mongorestore for simple migrations
Simple migrations of smaller collections can be done using the mongodump and mongorestore tools. These tools work with a binary format that ensures the smallest possible file size while retaining all data. They can be useful to migrate development and QA environments with more relaxed uptime service level agreements, or for a proof of concept migration to allow application testing in the new environment.
You can also utilize the mongoexport and mongoimport tools, however these commands utilize a text-based JSON or CSV format which is slower than the binary format used by mongodump and mongorestore.
To improve migration performance and reduce the impact on the production database, be sure to use hidden replica set members. These members still receive operation log updates and will vote in elections for primary, but they do not process requests from clients. This frees up resources for the migration process, and ensures it doesn't interfere with the production traffic.
If possible, create the hidden members in the same VPC as the Amazon DocumentDB cluster.
5. Use DMS for online migrations to reduce downtime
Nobody likes downtime. To minimize the impact on availability, use the Database Migration Service (DMS) to migrate the data.
DMS lets you continuously migrate updates between the source and target databases, keeping them in sync. This allows the existing application to continue updates to the MongoDB database during the migration process.
Once the two databases are up to date, you can switch the application to the DocumentDB cluster during a maintenance window. This minimizes the amount of downtime caused by the migration.
Summary
Let's face it, migrating a production application to a new database is a daunting task. But by taking the time to identify your sources - and making sure you have the right connection to these sources - you can reduce the chances of you tearing your hair out.
Performing a proof-of-concept migration using the mongodump and mongorestore tools can allow you to perform application testing that uncovers issues sooner. And when you're ready for that final migration, make sure to use the database migration service to pull data from hidden replica set members. This helps keep the impact on the existing production environment to a minimum
Before you migrate, brush up on DocumentDB & AWS
Before you move your data across, it can be helpful to make sure you know everything you can about Amazon DocumentDB. One of the best ways to do that is to get some hands-on experience in a real, risk-free live environment.
A Cloud Guru offers a Getting Hands on with Amazon DocumentDB course which covers how to set up, secure and scale MongoDB-compatible databases in depth. The course offers expert video advice, dozens of hands on labs, and exams you can take to test your knowledge.
Once you've completed the course, you can be confident of not only succeeding with your data migration, but working with DocumentDB well into the future.
