In this post, we’ll discuss Docker automatic MySQL slave propagation for help with scaling.
In my previous posts on the Docker environment, I covered Percona XtraDB Cluster. Percona XtraDB Cluster can automatically scale by conveniently adding new nodes using the highly automated State Snapshot Transfer. State Snapshot Transfer allows a new node to copy data from an existing node (I still want to see how this is possible with MySQL Group Replication).
This is not the case with regular MySQL Replication. With MySQL Replication, the slave setup still requires manual steps (well, unless you’ve already scripted it for your environment). At least these are “simple” steps (ha!). Percona XtraBackup can setup replication with less work (see this link for details: https://www.percona.com/doc/percona-xtrabackup/2.4/howtos/setting_up_replication.html ), but it still requires poking around and switching between servers.
However, nothing stops us from making it more automatic (similar to SST in Percona XtraDB Cluster), especially with Docker images. Why Docker? Because Docker provides a highly-controlled environment where we can orchestrate how scripts get executed. Severalnines provides a great intro into MySQL with Docker .
There are a few more components for this setup:
- Percona XtraBackup, to copy data without blocking masting
- Streaming backup functionality from Percona XtraBackup to directly copy data from one container to another
- Percona Server, with BACKUP LOCK https://www.percona.com/doc/percona-server/5.6/management/backup_locks.html (using BACKUP LOCK is less intrusive than FLASH TABLES WITH READ LOCK)
- Docker Network – with Docker Network it is easy to find a container by “name” (“replicaset_master” for example, thanks Kevin for the hint in comments )
Before jumping to my solution, I should point to some work in this area by Joyent: https://www.joyent.com/blog/dbaas-simplicity-no-lock-in .
I propose my image https://hub.docker.com/r/perconalab/ps-master-slave/ , with sources on GitHub https://github.com/percona/percona-docker/tree/master/percona-server-master-slave .
First, we need to start a master node:
dockerrun -d -p 3306:3306 --net=replicaset_net --name=replicaset_master -e MYSQL_ROOT_PASSWORD=Theistareyk perconalab/ps-master-slave --innodb-buffer-pool-size=2G
I assume that we’ve created the network replicaset_net already, either bridge or overlay .
You can create a slave by pointing to the master container:
dockerrun -d -p 3306 --net=replicaset_net --name=replicaset_slave1 -e MYSQL_ROOT_PASSWORD=Theistareyk -e MASTER_HOST=replicaset_master perconalab/ps-master-slave --innodb-buffer-pool-size=2G
The started node will automatically connect to MASTER_HOST , copy the data and perform all the steps needed to start the slave.
You can even copy data from a running slave, instead of the master, like this:
dockerrun -d -p 3306 --net=replicaset_net --name=replicaset_slave2 -e MYSQL_ROOT_PASSWORD=Theistareyk -e MASTER_HOST=replicaset_master -e SLAVE_HOST=replicaset_slave1 perconalab/ps-master-slave --innodb-buffer-pool-size=2G
This node will copy data from SLAVE_HOST , but then will point itself to MASTER_HOST .
Docker Network lets you use container names "replicaset_master" and "replicaset_slave1" instead of IP addresses, which is very convenient.
As the result of above, we have one master and two slaves running. We can start as many slave nodes as needed.
Please remember, this is more proof-of-concept than “production ready” images, but it gives a good direction for implementation.