I have a fair amount of files and media living on my Synology DS411 array, which is my designated file server for the lab and home, and wanted to back it up off-site for cheap. After reviewing my options, I felt like Amazon Glacier was the best bang for the buck and probably the safest bet when it comes to “will the vendor be around for a while.”
If you’re new to Glacier, it is an archive service provided by Amazon with a primary focus on ingesting data frequently (doing backups) but rarely doing restores. Their business model is set up so that it is cheap to back up data and more costly to restore data. That’s OK for me – I consider Glacier to be my security blanket to combat array failures, major outages, or a full site loss (knock on wood) where I need to restore the entire backup set and not just a file or two.
Although I’ve read a fair number of KBs, videos, and guides on how to set this up from Amazon, I felt like the identity management part is a bit goofy for a first timer (like me). And so here’s what I did to get the backups flowing.
Amazon Management Console Setup
I already have an account with Amazon – doesn’t everyone? – and so I started here. I needed to empower my Amazon account to do things in the Amazon Web Services (AWS) Management Console. Visit the AWS page to either sign in or sign up. Once that’s done, find the Identity and Access Management (IAM) page to begin setting up your account.
AWS has a handy little wizard that walks you through the five things they suggest doing for IAM. You’ll see it when you hit the IAM Dashboard. When you begin, there will not be any green check marks.
But what does this all mean? In essence, you’ll be asked to delete the root access keys – which give full admin over the entire AWS console – and trade them in for specific user accounts with their own keys. This gives you greater control granularity over the services and the ability to generate keys for a service account. It’s similar to the idea of not adding your AD service accounts to the Domain Administrators group.
Once that’s done, you’re asked to use multi-factor authentication (MFA), sometimes referred to as two-factor authentication (2FA), to help secure your account from malicious folks. I tend to use 2FA or MFA on almost everything, but this is ultimately an optional step. The Google Authenticator app in the Google Play store is great for this.
At this point you’ve secured your AWS Management Console; it’s time to build out a user and group for your backups.
Creating a Synology Backup User in IAM
Navigate to the Users screen within IAM and click the Create New Users button. I ended up making an account called Synology because it’s simple and obvious. Make sure to check the box to generate an access key for each user.
Once you click the Create button you’ll have the opportunity to write down the Access Key ID and Secret Access Key. Copy these somewhere safe for later or use the Download Credentials button to save them to a CSV file.
[symple_box color=”red” fade_in=”false” float=”center” text_align=”left” width=””]Hint: Don’t just backup your keys to AWS or else you won’t be able to get to them in a disaster scenario.[/symple_box]
If you later forget or lose the Secret Access Key, you’ll have to generate a new one. You can do this by editing the user details, scrolling down to the Access Credentials section, and clicking the Manage Access Keys button. From here you can delete old keys and generate new ones.
Replacing the Access Keys in your Synology backup task is not possible. You’ll have to issue a Retrieve task (which I cover towards the end of this post). This process asks you for a set of keys and then pulls in a backup set from Glacier. You can then delete the backup and vaults and start with a fresh vault. This is a bit goofy; it would be nicer if you could just edit the keys on the backup task.
Creating a Glacier Group in IAM
The Synology user currently has no authority to do anything. Let’s fix that. Head over to the Group screen within IAM and click Create New Group. I name my groups based on their permissions; as such, my group is named Glacier-Full because it gives members full access to the Glacier service.
I then choose the Amazon Glacier Full Access policy template.
If this is too heavy handed for you, there’s an option to use a policy wizard. In the example below, I removed any ability to delete archives, vaults, or vault notifications, and set the ARN to an existing vault. I did not opt to go this route, but you certainly can if you wish.
The final step is to review and accept the permissions.
Add The Synology User to the Glacier-Full Group
From the Groups page, select the new Glacier-Full group. There should be a warning under the list of Users (which is empty) stating that the group does not contain any users.
Click the Add Users to Group button and add the Synology user to the group.
At this point you have a valid service account with the required keys to back up to AWS Glacier. Let’s move on to the Synology configuration.
Install the Synology Glacier Backup App
Log into your Synology box and navigate to the Package Center. From there, choose Backup applications and select Glacier Backup. Install it and optionally select the Auto-update feature (I do).
You’ll now have a Glacier Backup icon on your Synology dashboard. Click it!
You can now start backing things up from the Synology box to AWS Glacier. Let’s get that done.
Creating a Synology Backup Job
Navigate to Backup, then Action > Create to open the backup wizard.
The first step is to name the task. Synology also wants you to accept the fact that this costs money; Amazon is going to charge you for this service.
The next step is to enter your keys and region. Remember those Access and Secret keys I told you to write down or save earlier? Go get those and enter them here. You’ll also want to select a region that makes sense based on your geography and price; I chose US West (Oregon). Amazon provides the costs for their service based on geography on this pricing page. Most of the US based services are the same price – California is a bit more expensive. If your data is sensitive you’ll likely want to enable the transfer encryption option. Which reminds me – you can read more about data at rest encryption on the Glacier FAQ page.
At this point, the Synology is going to issue a connection test to make sure your keys work. If you’re able to get to the next screen, you’re good to go. Otherwise, you either have an issue with your network connection, the keys, or your user does not have access to Glacier. Review your settings in IAM to make sure they match the steps we went through earlier in this post.
The third step is to identify what to back up. Simple enough. I chose my Shared folder, which contains my documents and media files. Get as granular as you want.
Finally, figure out how often you want to back things up, and if you want to shoot over a backup immediately. I use a daily schedule set to 3 AM.
The first backup will take a while. Make sure that you see data flowing over to Glacier. Once completed, the Overview screen will show a completed backup task. Here’s an example of a manual backup that I ran:
Congratulations, your stuff is in the cloud.
Examining Glacier Vaults
You might also want to see your stuff in the magical cloud. Head over to the AWS Console and select Glacier. You’ll see a pair of vaults generated by the Synology. Vaults don’t refresh their size or number of archives very often; don’t worry if yours look empty after a backup. This is normal. Check back the next day and you should see that the Glacier dashboard is updated.
I have about 3.38 GB (3.15 GiB) of data backed up. This costs about $0.16 a month. If I need to restore the data, it will be about a $1. I consider this money well spent.
Here’s my first bill from AWS. It ended up being $0.23 because of a test vault delete and some other requests to help build this post, and I also grew the backup vault to about 4 GB of total data. I back up daily at 3AM.
I’ve also created a generic billing alarm using the CloudWatch service to trigger an email if my total spend with AWS exceeds $10 in one month. I don’t really care about any charges below that dollar amount. You can configure the alarm for whatever you feel makes sense, and will receive an email if your spend hits that value, or create a specific alarm for just Amazon Glacier.
[symple_box color=”red” fade_in=”false” float=”left” text_align=”left” width=””]Update: You can now configure Data Retrieval Policies to further control spend on data restoration. Check out my blog post entitled Using Data Retrieval Policies to Control AWS Glacier Restore Costs.[/symple_box]
Retrieving Backup Sets
If you lost your backup task or are using a brand new Synology array, you’ll need to pull the backup sets out of your Glacier Vaults. To do this, issue a Retrieve task from the Glacier app by going to Backup > Action > Retrieve task. You’ll then see one or more new tasks appear with a status of Retrieving… for a long while as your vaults are inspected. This action requires having the Access and Secret Keys to view your Vaults.
Once completed, you can restore data from any of the retrieved backup tasks or delete the files and vaults from Glacier. You won’t be able to edit the backup set and create a schedule; that option is grayed out.
Thoughts
Amazon Glacier is a pretty cheap way to get your data off premises without breaking the bank. It’s also relatively simple to setup and integrate into your Synology array. I’ve been using the service for about 2 months as of this posting and haven’t had to so much as look at the backups: they just work.
Restoration is all or nothing at this point – I can only restore the entire backup which would conflict with my existing data. In DSM 5.1 the folks at Synology are going to offer an Explorer feature to do more granular restores. For now, I’ve tested restoring a dummy vault successfully, but do not see an easy way to do a test restore of my production data without hitting conflicts with my existing data. I could always just snag Damian Karlson’s PowerShell scripts for Glacier or one of the many Glacier file explorers to do more fun stuff with my Vaults. I’ll save that adventure for another post.
