EBS Snapshot Lifecycle

Amazon Elastic Block Storage (EBS) snapshots are a great addition to your backup and disaster recovery plans and
Amazon Data Lifecycle Manager (DLM) can make the process easy.

Accelerate your career!

Get started with ACG and transform your career with courses and real hands-on labs in AWS, Microsoft Azure, Google Cloud, and beyond

Amazon EBS (Elastic Block Storage) snapshots are a great addition to your backup and disaster recovery plans. They are commonly used by organizations small and large for just this purpose. Part of  any good disaster recovery plan is reviewing and testing it regularly. Your review should include cleaning up old snapshots because as your environment changes and grows over time you may find that you have accumulated a large collection of snapshots. While snapshots are considered incremental and the associated storage costs per snapshot tend to be low, this is not always the case. For instance, you may be required to have frequent large volume snapshots with various retention times on them. Over time this could potentially add up to a lot of storage to pay for. In addition, manually managing all those snapshots could be quite cumbersome. This is where Amazon Data Lifecycle Manager (DLM) can come to the rescue.

Amazon Data Lifecycle Manager provides a simple and automated way for you to manage your EBS snapshot backups. Using this service you will be able to automate the creation, retention and deletion of your EBS snapshots. Some benefits of using this include:

• Protect valuable data by enforcing a regular backup schedule.
• Create standardized AMIs that can be refreshed at regular intervals.
• Retain backups as required by auditors or internal compliance.
• Reduce storage costs by deleting outdated backups.
• Create disaster recovery backup policies that back up data to isolated accounts

You have a compliance directive that requires you to create a new snapshot of an EBS volume. The snapshot needs to happen every hour, and delete the snapshots that are more than 2 days old. This could be a repetitive task, but with a little help from DLM we can quickly set this up to be automated.

For our example we will assume that the volume already exists and we are setting up a new Lifecycle policy. From our EC2 dashboard we want to go to Volumes in the navigation pane and select our volume. 

DLM works off of tags, so the first thing we will want to do is set up a special tag for this volume (and potential future volumes) that the lifecycle policy will use for managing the snapshots.

1. Click the volume ID, then the ‘Tags’ tab. From here we want to click ‘Manage tags’, and then add a tag. 
2. I am going use Key:enable-snapshot Value:yes and then save this.
3. Now that our volume is tagged, click on the LifeCycle Manager link in the navigation pane.
4. From there select EBS snapshot from the dropdown and click Next step.
5. In this case we are targeting a volume so make sure that is selected, then we want to use our newly added tag for the Target resource. Enter the tags and click add.
6. Provide a description for this policy. 
7. We will leave the IAM role as default, although you can create custom roles for this. Also, we won’t add any tags to the snapshots for this example, but it is something that you can do if you need or want to.
8. Next, verify that the policy is set to be enabled. And click next.

Now we need to create the schedule for the policy. We know that it needs to run every hour and that they should be retained for 2 days. On the scheduling screen we can fill this in. Because we are only creating this one schedule I am just going to leave the name as the default.  One thing to note is that starting time is in UTC, and the policy will run within an hour of that starting time. We can skip the rest of the advanced settings, and move onto reviewing the policy. Once you verify everything looks ok, click Create policy. 

This takes us back to the Lifecycle Manager view and we can see our new policy. After waiting for the start time of the schedule to pass, we can check on the Snapshots tab and see we have a new snapshot.  In 2 days this will be removed per the retention policy we setup.

Just like that we have setup an automated snapshot creation and deletion policy to help with our compliance. You should not have to worry about an ever growing number of snapshots anymore as they are now managed by the Data Lifecycle Manager. With all technology there is a chance for failure, luckily you can use CloudWatch to monitor for such events and address any issues that may arise.

Want to learn more about AWS? Check out my latest course, Running Linux Servers on AWS. With a large part of the AWS cloud being built on Linux, it may not be all that surprising that a lot of the virtual machines that companies and individuals run within AWS are also using Linux. Check out this course where we will delve into the world of running Linux systems on the AWS platform.

Want to keep up with all things AWS? Subscribe to A Cloud Guru on YouTube for weekly Amazon news and AWS announcements. You can also like ACG on Facebook, follow us on Twitter, or join the conversation on Discord!