Migrating RDS from EC2 to VPC
Forward: One of the frustrating things about working with a cloud provider like Amazon Web Services, is that each service has to have a name. Each name is, of course, too long to use in everyday conversation, so they’re all initialized and acronymed until someone new to the platform has no idea what any of the letters mean. I apologize in advance and I promise I’ll try to explain all of the TLAs (Three Letter Acronyms) as clearly as possible.
##The Problem
In early 2013, I started using Amazon RDS (Relational Database Service) to manage old MySQL data for work. (RDS lets you pick a database engine (Oracle, MySQL, Postgres, SQLServer), gives you great flexibility with storage and scalability and manages backups and failovers transparently.) After a few months, I migrated the rest of our data and things have been great.
Recently (sometime in 2013), AWS made their Virtual Private Cloud platform the default for all new accounts, but this was after our RDS instance was created. At the Re:invent conference, AWS made it very clear that their VPC platform was the way forward and that anything new should be using it.
So here I am, brain the size of a planet ready to start migrating other services to AWS, but our primary database is not in a Virtual Private Cloud and I shudder at the idea of having to redo everything in a few years when AWS decides that support for standalone instances goes away.
##The Solution
The easiest solution would simply be to launch a read-replica that resides in a VPC and then promote it to Master, but that is not (as of right now) allowed.
I combined the techniques for replicating to an external source and importing data from an external source to make a painless, zero-downtime migration into a VPC. If you have enough data in your database that mysqldump has the potential to run for hours and hours, this might be useful to you.
First, add a user for replication and remember the username and password. Next, create a read replica of the original database. When it’s up and running, connect to it on the command line and manually stop the replication. (Since you’re not technically a root user, you need to utilize the stored procedures that RDS provides for you.)
> CALL mysql.rds_stop_replication();
Then, run “SHOW SLAVE STATUS\G” and copy the values for Master_Log_File and Read_Master_Log_Pos. Leave the replication halted and return to the RDS Console in your browser. Select the read replica and take a snapshot of it.
At this point, I’m assuming your VPC is already configured. If you haven’t done it yet, you can refer to this article on running RDS on VPC. Make sure your Database Subnet Groups have been created and that you have an Internet Gateway associated with those subnets and a route table that allows traffic. You’ll also need to make rules in your security groups that allow traffic between the VPC and EC2 RDS instances. This is discussed in the article I linked to above.
Once the snapshot is complete, find it under the snapshots tab, select it and hit the big Restore Snapshot button. In the options, make sure to specify your VPC and corresponding Subnet Groups.
When this new instance is running, connect to it on the command line and run the set_external_master stored procedure, using the replication user and password you set up first, along with the master log file information you copied. Now, run
> CALL mysql.rds_start_replication();
and you’re good to go! Use “SHOW SLAVE STATUS\G” and watch the value of seconds_behind_master to see when you’re all caught up.
You’ll need to change your application code to point to the new instance. Please test this first and choose the method of migration that’s appropriate for you. When you’ve successfully migrated to your database in your VPC, use
> CALL mysql.rds_reset_external_master();
and then shut down your original instance along with the temporary read replica you created for making a snapshot. You can probably remove some rules from your Security groups as well.