Skip to content

Instantly share code, notes, and snippets.

@miklos-szel

miklos-szel/mha

Created July 21, 2016 14:40
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save miklos-szel/ec0483a53b9ee8d33540279c3fefbec3 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save miklos-szel/ec0483a53b9ee8d33540279c3fefbec3 to your computer and use it in GitHub Desktop.
Download ZIP
MHA online failover sample output
Raw
mha
Thu Jul 21 14:34:16 2016 - [info] MHA::MasterRotate version 0.56.
Thu Jul 21 14:34:16 2016 - [info] Starting online master switch..
Thu Jul 21 14:34:16 2016 - [info]
Thu Jul 21 14:34:16 2016 - [info] * Phase 1: Configuration Check Phase..
Thu Jul 21 14:34:16 2016 - [info]
Thu Jul 21 14:34:16 2016 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
Thu Jul 21 14:34:16 2016 - [info] Reading application default configuration from /etc/mha_mukkannnn.cnf..
Thu Jul 21 14:34:16 2016 - [info] Reading server configuration from /etc/mha_mukkannnn.cnf..
Thu Jul 21 14:34:16 2016 - [info] GTID failover mode = 0
Thu Jul 21 14:34:16 2016 - [info] Current Alive Master: 10.77.24.236(10.77.24.236:3306)
Thu Jul 21 14:34:16 2016 - [info] Alive Slaves:
Thu Jul 21 14:34:16 2016 - [info] 10.77.24.111(10.77.24.111:3306) Version=5.6.31-77.0-log (oldest major version between slaves) log-bin:enabled
Thu Jul 21 14:34:16 2016 - [info] Replicating from 10.77.24.236(10.77.24.236:3306)
It is better to execute FLUSH NO_WRITE_TO_BINLOG TABLES on the master before switching. Is it ok to execute on 10.77.24.236(10.77.24.236:3306)? (YES/no): yes
Thu Jul 21 14:34:20 2016 - [info] Executing FLUSH NO_WRITE_TO_BINLOG TABLES. This may take long time..
Thu Jul 21 14:34:20 2016 - [info] ok.
Thu Jul 21 14:34:20 2016 - [info] Checking MHA is not monitoring or doing failover..
Thu Jul 21 14:34:20 2016 - [info] Checking replication health on 10.77.24.111..
Thu Jul 21 14:34:20 2016 - [info] ok.
Thu Jul 21 14:34:20 2016 - [info] 10.77.24.111 can be new master.
Thu Jul 21 14:34:20 2016 - [info]
From:
10.77.24.236(10.77.24.236:3306) (current master)
+--10.77.24.111(10.77.24.111:3306)
To:
10.77.24.111(10.77.24.111:3306) (new master)
+--10.77.24.236(10.77.24.236:3306)
Starting master switch from 10.77.24.236(10.77.24.236:3306) to 10.77.24.111(10.77.24.111:3306)? (yes/NO): yes
Thu Jul 21 14:34:35 2016 - [info] Checking whether 10.77.24.111(10.77.24.111:3306) is ok for the new master..
Thu Jul 21 14:34:35 2016 - [info] ok.
Thu Jul 21 14:34:35 2016 - [info] 10.77.24.236(10.77.24.236:3306): SHOW SLAVE STATUS returned empty result. To check replication filtering rules, temporarily executing CHANGE MASTER to a dummy host.
Thu Jul 21 14:34:35 2016 - [info] 10.77.24.236(10.77.24.236:3306): Resetting slave pointing to the dummy host.
Thu Jul 21 14:34:35 2016 - [info] ** Phase 1: Configuration Check Phase completed.
Thu Jul 21 14:34:35 2016 - [info]
Thu Jul 21 14:34:35 2016 - [info] * Phase 2: Rejecting updates Phase..
Thu Jul 21 14:34:35 2016 - [info]
Thu Jul 21 14:34:35 2016 - [info] Executing master ip online change script to disable write on the current master:
Thu Jul 21 14:34:35 2016 - [info] /usr/local/sbin/master_ip_online_change.sh --command=stop --orig_master_host=10.77.24.236 --orig_master_ip=10.77.24.236 --orig_master_port=3306 --orig_master_user='mha' --orig_master_password='xxx' --new_master_host=10.77.24.111 --new_master_ip=10.77.24.111 --new_master_port=3306 --new_master_user='mha' --new_master_password='xxx' --orig_master_ssh_user=root --new_master_ssh_user=root --orig_master_is_new_slave
===> setting read_only=ON on current master
Warning: Using a password on the command line interface can be insecure.
Thu Jul 21 14:34:36 2016 - [info] ok.
Thu Jul 21 14:34:36 2016 - [info] Locking all tables on the orig master to reject updates from everybody (including root):
Thu Jul 21 14:34:36 2016 - [info] Executing FLUSH TABLES WITH READ LOCK..
Thu Jul 21 14:34:36 2016 - [info] ok.
Thu Jul 21 14:34:36 2016 - [info] Orig master binlog:pos is mysql-bin.000001:120.
Thu Jul 21 14:34:36 2016 - [info] Waiting to execute all relay logs on 10.77.24.111(10.77.24.111:3306)..
Thu Jul 21 14:34:36 2016 - [info] master_pos_wait(mysql-bin.000001:120) completed on 10.77.24.111(10.77.24.111:3306). Executed 0 events.
Thu Jul 21 14:34:36 2016 - [info] done.
Thu Jul 21 14:34:36 2016 - [info] Getting new master's binlog name and position..
Thu Jul 21 14:34:36 2016 - [info] mysql-bin.000002:120
Thu Jul 21 14:34:36 2016 - [info] All other slaves should start replication from here. Statement should be: CHANGE MASTER TO MASTER_HOST='10.77.24.111', MASTER_PORT=3306, MASTER_LOG_FILE='mysql-bin.000002', MASTER_LOG_POS=120, MASTER_USER='rdsrepladmin', MASTER_PASSWORD='xxx';
Thu Jul 21 14:34:36 2016 - [info] Executing master ip online change script to allow write on the new master:
Thu Jul 21 14:34:36 2016 - [info] /usr/local/sbin/master_ip_online_change.sh --command=start --orig_master_host=10.77.24.236 --orig_master_ip=10.77.24.236 --orig_master_port=3306 --orig_master_user='mha' --orig_master_password='xxx' --new_master_host=10.77.24.111 --new_master_ip=10.77.24.111 --new_master_port=3306 --new_master_user='mha' --new_master_password='xxx' --orig_master_ssh_user=root --new_master_ssh_user=root --orig_master_is_new_slave
Thu Jul 21 14:34:36 2016 - [info] ok.
Thu Jul 21 14:34:36 2016 - [info] Setting read_only=0 on 10.77.24.111(10.77.24.111:3306)..
Thu Jul 21 14:34:36 2016 - [info] ok.
Thu Jul 21 14:34:36 2016 - [info]
Thu Jul 21 14:34:36 2016 - [info] * Switching slaves in parallel..
Thu Jul 21 14:34:36 2016 - [info]
Thu Jul 21 14:34:36 2016 - [info] Unlocking all tables on the orig master:
Thu Jul 21 14:34:36 2016 - [info] Executing UNLOCK TABLES..
Thu Jul 21 14:34:36 2016 - [info] ok.
Thu Jul 21 14:34:36 2016 - [info] Starting orig master as a new slave..
Thu Jul 21 14:34:36 2016 - [info] Resetting slave 10.77.24.236(10.77.24.236:3306) and starting replication from the new master 10.77.24.111(10.77.24.111:3306)..
Thu Jul 21 14:34:36 2016 - [info] Executed CHANGE MASTER.
Thu Jul 21 14:34:36 2016 - [info] Slave started.
Thu Jul 21 14:34:36 2016 - [info] All new slave servers switched successfully.
Thu Jul 21 14:34:36 2016 - [info]
Thu Jul 21 14:34:36 2016 - [info] * Phase 5: New master cleanup phase..
Thu Jul 21 14:34:36 2016 - [info]
Thu Jul 21 14:34:36 2016 - [info] 10.77.24.111: Resetting slave info succeeded.
Thu Jul 21 14:34:36 2016 - [info] Switching master to 10.77.24.111(10.77.24.111:3306) completed successfully.
-------------------------------
Current value: 10.77.24.111 -> read_only = 0
-------------------------------
Current value: 10.77.24.236 -> read_only = 1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment