In a week’s time I’ll be speaking at Percona Live Amsterdam. I will be presenting:
Managing and Visualizing your replication topologies with Orchestrator
23 September 4:20PM
This talk will present orchestrator, on which I’ve been working for the last year and a half, originally at Outbrain and now at Booking.com.
I will show off what orchestrator can do to manage your replication topologies. From visualization, through topology refactoring to automated crash recoveries, orchestrator today plays a key role at Booking.com infrastructure, at scale (oh I love using these words).
You can expect an outrageous demo, a visual walkthrough, some command line examples, and a lot on the logic and mechanisms behind orchestrator. I will present the difficult problems orchestrator covers.
orchestrator is free and open source, and is built to be as generic as possible; it is known to be used by multiple well known companies these days, so please join the party.
With that, I conclude with the almighty motto:
Hi,
I understand that the Orchestrator can handle the master failures and change topology but does it also facilitates the transfer of this change in topology to the Application so that it can point the DML to Master and Read only to Slaves?
I know it is not easy to do everything in the same software, could you please explain how do you handle read/write split at booking.com ?
Hi,
Indeed; this part is external to orchestrator and executed by orchestrator based on what you tell it to do in configuration.
This would be any shell script, executable or whatever. Orchestrator lets you pass any imporatn information to such scripts, in the form of {failedHost}, {successorHost}, {…others…}
It is up to those executable to do the right thing. Because everyone uses different methods of master discovery (and we use different methods at Booking.com), I left it outside orchestrator.
This of course means orchestrator is not a complete failover solution. It’s responsibilities are healing the topology and calling the tools to complete the master discovery.
We are using different methods for server discovery. From VIPs and CNAMES to Zookeeper rosters. Slaves are part of pools which are managed by rosters. Masters typically recognized by DNA, A record or CNAMEs; puppet ensures masters are outside any pool. We are also looking into proxy solutions.
The obvious question: Why is Orchestrator better than MHA?
Hi Rick,
It is not better as the two do not compare.
Orchestrator is more than a failover mechanism. It is first a discovery and refactoring solution.
With regard to failovers orchestrator is a complete solution for intermediate masters failvoers — which is a big problem
for us at Booking.com; MHA does not solve that.
Orchestrator is a non-complete master failover solution (and MHA is). It reaches the point of healing your topology, then relays the job to someone else to finish up. It does not assume you’re using VIPs; it does not care what type of master discovery you use.
Orchestrator heals the topology based on state, as opposed to based on configuration. It knows your datacenters, understand sreplication rules etc. and solves the problem taking all the above into consideration.
It solves GTID, Pseudo-GTID or normal binlog file:pos scenarios.
It understands binlog servers and does master failover for “pure” binlog server or hybrid binlog server topologies.
But this comment is getting too long. May I suggest I have the honor of you attending my talk?
I’m likely to show up at your talk.
Does it also deal with MariaDB’s flavor of GTIDs?