Very interesting and nice presentation. Thank you for the detailed explanation.

Would like to know, when can we expect Multiple-Maxscale servers for deployment? Having single MaxScale is a single point of failure for critical applications. Similar to master-slave for MariaDB database, it’s better to have the same concept for MaxScale as well.

If MaxScale server is down, then there is no point of having master with many slaves. It will defeat the purpose.

If you recommend any work-around for this? Your suggestion is greatly appreciated. Thank you very much in advance.

Thanks for the write up and suggestions.

Few clarification and few comments on failover
(1) MaxScale does not trigger failover it self , but has mechanism to invoke the failover script based on configured events
(2) MaxScale differentiates between having lost communication with master (lost_master event) and having communication with master but the database being down on master (master_down event).
(3) MaxScale can be configured to trigger the external script only on master_down event and not lost_master event.
(4) MaxScale does not automatically invoke MariaDB Replication Manager – MaxScale can be configured to invoke any other external script – could be MHA or could be custom developed by any other user. Some of our users do not like automatic failover, but just notification of master failure – for such users the script could be simply a curl script to send email notification of failure.
(5) Yes, MariaDB Replication Manager script is for MariaDB based database server using MariaDB GTID.
(6) Since MaxScale has knowledge of before failure topology – I agree improvements can be made to MariaDB Replication Manager to use this information rather then rediscovering and not having the pre-failure topology information
(7) While MaxScale and scripts are decoupled, MaxScale does not automatically assume that the failover has been resolved upon triggering of script – Maxscale still continues to monitor the topology and detects when a new master has been promoted through its monitoring.
(8) MaxScale has configurable monitor_interval as well as backend_connect_timeout that can be used to control flapping – though another period can be added specifically for triggering failover script to minimize flapping further
(9) Single MaxScale can handle multiple topologies.
(10) We will take your input on shell script and what you have done in orchestrator, as well as tighter integration between MaxScale and the failover script/tool

On High Availability of MaxScale itself:
We are exploring, Multiple-MaxScale servers to load-balance on our roadmap – and will consider all your points on leader election, group communication, taking over from failed MaxScale

Thanks again for detailed blog about your experience and observation on what can be done with MaxScale to address operational scenarios.

Thank you. Indeed you are not the first to raise this point, and I understand it. At this time I don’t just rely on MySQL at the backend, I rely on the relational model (so sometimes complex GROUP BY queries etc.). So migration to etcd will be non-trivial.

But there are thoughts in that direction; I don’t expect this to come up in the near future unless someone else steps up to do it.

very interesting presentation, thank you. I’ll make sure to try Orchestrator in the near future!

However, something that puts me off is the reliance on a mysql datastore. I understand the temptation to use mysql for everything, but wouldn’t a distributed datastore such as consul or etcd be more simple and efficient? I just don’t want to setup a galera cluster for that. Plus consul/etcd also have leader election, so you might find that interesting.