After my previous post about a complete code for automated RDS cross-region backup copy, some issues and new feature requests have been raised on GitHub (thanks for that!) - and with your help, support for encrypted RDS instances and Aurora Clusters have now been added!
Encryption
You can enable a built-in encryption-at-rest in most RDS instances. RDS will use a KMS key to encrypt your data and all snapshots made from that database. Since KMS keys are region-specific, when copying such snapshots to another region, they have to be re-encrypted using a key that’s located in that another region. Luckily, AWS deals with it for us, we just have to provide the information which key we’d like to use.
With support for such snapshots now added, when creating the CloudFormation stack, you will be asked for an ARN of a KMS key in your target region. This key will be used to re-encrypt the snapshots during the copy operation. At this time, you can only provide one key, which will be used for all snapshots for all databases that the code deals with, which should generally be fine for most use-cases, where having snapshots in another region encrypted with the same key is not a problem.
Aurora support
Another requested feature was support for Aurora clusters. Due to the way Aurora operates, the instances that run the database create a single “cluster”. All operations made on Aurora are made on the cluster, not on specific instances within the cluster. This may be a bit confusing, especially since AWS Console actually shows all the actions under " Instances" tab, instead of “Clusters” - but in the AWS API behind all of this, those operations are actually relating to the cluster as a whole.
The operations (describing a cluster, copying a snapshot, etc) are essentially the same as for non-Aurora instances,
except having a “cluster” in operation and parameter names. For example, getting list of snapshots for a non-Aurora RDS
instance is describe_db_snapshots, for Aurora - describe_db_cluster_snapshots. The output is also very similar,
except the list of snapshots is returned under DBClusterSnapshots key instead of DBSnapshots. With that in mind,
copying Aurora snapshots is in itself very similar to copying non-Aurora snapshots, it’s mostly just about choosing the
right names.
Unfortunately, the actual process of triggering the copy process is not as simple. With non-Aurora instances, the Lambda code was triggered by an RDS event notification when a snapshot was finished. Aurora clusters not only have their own set of operations, they also have a completely separate set of events they generate, and, as it turns out, they do not have an event for automated snapshots. There is only an event for manual ones. Because of that, a whole other approach to copying those snapshots needed to be put in place. Once you enable Aurora support when creating your CloudFormation stack (set “Use for Aurora cluster” parameter to “yes”), a daily schedule with CloudWatch Events will be put in place. Once a day, CloudWatch Events will trigger the Lambda, which will go through all most-recent snapshots of our Aurora clusters and copy them over to another region. This should generally be fine for most use-cases, but be aware that this specific Lambda execution will take longer than those for a single non-Aurora instances.
One thing to be aware of: the CloudWatch Event is scheduled with a rate of once a day - the exact time of the day will be chosen by AWS, and it may happen to be before your backup window for Aurora clusters. If that happens, when the Lambda executes on the day, the automated snapshot of your cluster will not have been made yet for that day, so the previous day snapshot will be copied. If that happens (and is a big issue for you), you can change the schedule of the event to a cron-like time and set it to run at a specific time of a day, after all your backup windows for Aurora clusters.
GitHub repo where you can find the latest code: https://github.com/pbudzon/aws-maintenance
