April 18, 2015
The MySQL Community Awards initiative is an effort to acknowledge and thank individuals and corporates for their contributions to the MySQL ecosystem. It is a from-the-community, by-the-community and for-the-community effort. The committee is composed of an independent group of community members of different orientation and opinion, themselves past winners or known contributors to the community.
The 2015 community awards were presented on April 15rd, 2015, during the community event at the Percona Live conference. The winners are:
MySQL Community Awards: Community Contributor of the year 2015
- Daniël van Eeden
Daniël has done great work on MySQL security and has continued to fantastically support MySQL User Group.NL. He has also logged a lot of bugs (and submitted patches), across all sorts of different MySQL products and has done a great deal to help improve the quality of MySQL.Daniël consistently provides extremely good feedback, on a wide range of features and products, from MySQL server security, through InnoDB, partitioning, and even on other products such MySQL Enterprise Backup and MySQL Enterprise Monitor. His bugs are always reported with a high quality, and many times he even includes a contribution to fix those bugs.
- Justin Swanhart
Justin has worked tirelessly for the past few years on some amazing projects of his own design, Shard-Query and Flexviews. Cross shard aggregation is an extremely complex thing to get right, and Shard-Query takes an interesting approach at this. Flexviews provides a materialized view framework, which is something that MySQL lacks to many people's annoyance. Additionally Justin has also built some performance_schema related tools, reported many MySQL bugs, and has been a public speaker about MySQL in "can do" style.
- Morgan Tocker
In his day job, Morgan is Community Manager at Oracle. While some of his community interaction has been because of his job, he has gone far and beyond his corporate responsibilities. He is one of the most prolific writers on the MySQL Planet, he has been the most public face of MySQL, and he is always asking for feedback and showing a sincere concern for the Open Source community.For example, Morgan’s community polls on what defaults should be changed in MySQL 5.7 put some of the MySQL product decision making directly into the hands of the community. He is a key player on keeping the community and the MySQL developers at Oracle in touch with each other.
MySQL Community Awards: Application of the year 2015
Continue Reading »
March 31, 2015
In two weeks time I will be presenting Pseudo GTID and Easy Replication Topology Management at Percona Live. From the time I submitted the proposal a LOT has been developed, experimented, deployed and used with both Pseudo GTID and with orchestrator. In my talk I will:
- Suggest that you skip the "to GTID or not to GTID" question and go for the lightweight Pseudo GTID
- Show how Pseudo GTID is used in production to recover from various replication failures and server crashes
- Do an outrageous demonstration
- Tell you about 50,000 successful experiments and tests done in production
- Show off orchestrator and its support for Pseudo GTID, including automated crash analysis and recovery mechanism.
I will further show how the orchestrator tooling makes for a less restrictive, more performant, less locking, non-intrusive, trusted and lightweight replication topology management solution. Continue Reading »
February 6, 2015
The 2015 MySQL Community Awards event will take place, as usual, in Santa Clara, during the Percona Live MySQL Conference & Expo, April 2015.
The MySQL Community Awards is a community based initiative. The idea is to publicly recognize contributors to the MySQL ecosystem. The entire process of discussing, voting and awarding is controlled by an independent group of community members, typically based of past winners or their representatives, as well as known contributors.
It is a self-appointed, self-declared, self-making-up-the-rules-as-it-goes committee. It is also very aware of the importance of the community; a no-nonsense, non-political, adhering to tradition, self criticizing committee.
The Call for Nominations is open. We are seeking the community’s assistance in nominating candidates in the following categories: Continue Reading »
January 29, 2015
This coming Sunday I'll be presenting Pseudo GTID and easy replication management at FOSDEM, Brussels.
There's been a lot of development on Pseudo GTID these last few weeks. In this talk I'll show you how you can use Pseudo GTID instead of "normal" GTID to easily repoint your slaves, recover from intermediate master failure, promote slaves to masters as well as emply crash safe replication without crash safe replication.
Moreover, I will show how you can achieve all the above with less constraints than GTID, and for bulk operations -- with less overhead and in shorter time. You will also see that Pseudo GTID is a non intrusive solution which does not require you to change anything in your topologies.
Moral: I'll try and convince you to drop your plans for using GTID in favor of Pseudo GTID.
We will be employing Pseudo GTID as the basis for high availability and failover at Booking.com on many topologies, and as a safety mechanism in other topologies where we will employ Binlog servers.
January 26, 2015
For purposes of auditing anything that goes on our servers we're looking to parse the binary logs of all servers (masters), as with "Anemomaster". With Row Based Replication this is problematic since pt-query-digest does not support parsing RBR binary logs (true for 2.2.12, latest at this time).
I've written a simple script that translates RBR logs to SBR-like logs, with a little bit of cheating. My interest is that pt-query-digest is able to capture and count the queries, nothing else. By doing some minimal text manipulation on the binary log I'm able to now feed it to pt-query-digest which seems to be happy.
The script of course does not parse the binary log directly; furthermore, it requires the binary log to be extracted via:
mysqlbinlog --verbose --base64-output=DECODE-ROWS your-mysql-binlog-filemame.000001
The above adds the interpretation of the RBR entires in the form of (unconventional) statements, commented, and strips out the cryptic RBR text. All that is left is to do a little manipulation on entry headers and uncomment the interpreted queries.
The script can be found in my gist repositories. Current version is as follows: Continue Reading »
December 18, 2014
Orchestrator 1.2.9 GA has been released. Noteworthy:
- Added "ReadOnly" (true/false) configuration param. You can have orchestrator completely read-only
- Added "AuthenticationMethod": "multi": works like BasicAuth (your normal HTTP user+password) only it also accepts the special user called "readonly", which, surprise, can only view and not modify
- Centralized/serialized most backend database writes (with hundreds/thousands monitored servers it was possible or probable that high concurrency led to too-many-connections openned on the backend database).
- Fixed evil evil bug that would skip some checks if binary logs were not enabled
- Better hostname resolve (now also asking MySQL server to resolve hostname; resolving is cached)
- Pseudo-GTID (read here, here, here) support now considered stable (apart from being tested it has already been put to practice multiple times in production at Outbrain, in different planned and unplanned crash scenarios)
I continue developing orchestrator as free and open source at my new employer, Booking.com.
November 14, 2014
I'm excited to be joining Booking.com at the Netherlands in a couple weeks
I'm looking forward to be working with a great team and friendly people! I hope to contribute from my experience and of course be challenged by difficult problems.
Booking.com is a supporter of open source in multiple aspects, and I am looking forward to continue working with open source solutions as well as releasing open source code.
I am leaving my work at Outbrain feeling grateful for the opportunity of working at this wonderful company! I am awed and humbled by the amazing teams I've worked with, whose level of knowledge and insights I can only aspire to match. Thank you in particular to the Infrastructure team, of which I was proud to be part of.
Outbrain allowed me and others (and in fact encouraged and supported) to develop as much open source as we saw fit. This is not a minor thing: when you orient your code towards open source, you need to make generalizations which are not always providing direct benefit to the company, and which consume precious time. I hold Outbrain in the highest respect for their support for open source.
November 10, 2014
Orchestrator release 1.2.7-beta now supports semi-automatic slave promotion to master upon master death, via Pseudo GTID.
When the master is dead, orchestrator automatically picks the most up-to-date slaves and marks them as "Master candidates". It allows a /api/make-master call on such a slave (S), in which case it uses Pseudo GTID to enslave its siblings, and set S as read-only = 0. All we need to do is click the "Make master" button. Continue Reading »
October 31, 2014
And not for long!
The Call for Papers for Percona Live MySQL Conference and Expo, to be held at Santa Clara in April 2015, is open. The dead line for submissions is Nov. 16th; that's just around the corner.
As with previous years, we will hold a 4 day conference, the first being a tutorials day and three days for sessions, BoF and lightning talks, as well as community events. The committee is expecting to review at about 250-300 submissions, out of which it will pick at about 100 talks to schedule or reserve.
We will be using these tracks:
- High Availability
- Performance Optimization
- Replication and Backup
- MySQL in the Cloud
- MySQL and NoSQL
- MySQL Case Studies
- What’s New in MySQL
This year we will roughly pre-define the desired number of sessions we wish to have per track. This is not set in stone and everything is fluid. Yet, this will give us better guidelines at choosing and pursuing content for this conference.
Submitting a proposal
We encourage all members of the community to submit their tutorial/session/BoF proposals as soon as possible. Please register/login at the conference home page.
The guidelines for submitting a proposal are generally unchanged; please review past recommendations: , , , . To add to all these:
- Do note that we are likely to only review a proposal just once. Please submit only after you have finalized your draft.
- Make a reasonable length of proposal. We believe 250 - 300 words are quite enough for a good proposal. Please don't write an essay, and remember that you proposal is what gets printed on the schedule, and what is read by the conference attendees when choosing the next talk to go to.
- Write a descent Bio.
Continue Reading »
October 27, 2014
Orchestrator 1.2.1-beta supports Pseudo GTID (read announcement): a means to refactor the replication topology and connect slaves even without direct relationship; even across failed servers. This post illustrates two such scenarios and shows the visual way of mathcing/re-synching slaves.
Of course, orchestrator is not just a GUI tool; anything done with drag-and-drop is also done via web API (in fact, the drag-and-drop invoke the web API) as well as via command line. I'm mentioning this as this is the grounds for failover automation planned for the future.
Scenario 1: the master unexpectedly dies
The master crashes and cannot be contacted. All slaves are stopped as effect, but each in a different position. Some managed to salvage relay logs just before the master dies, some didn't. In our scenario, all three slaves are at least caught up with the relay log (that is, whatever they managed to pull through the network, they already managed to execute). So they're otherwise sitting idle waiting for something to happen. Well, something's about to happen.
Note the green "Safe mode" button to the right. This means operation is through calculation of binary log files & positions with relation to one's master. But the master is now dead, so let's switch to adventurous mode; in this mode we can drag and drop slaves onto instances normally forbidden. At this stage the web interface allows us to drop a slave onto its sibling or any of its ancestors (including its very own parent, which is a means of reconnecting a slave with its parent). Anyhow:
We notice that orchestrator is already kind enough to say which slave is best candidate to be the new master (127.0.0.1:22990): this is the slave (or one of the slaves) with most up-to-date data. So we choose to take another server and make it a slave of 127.0.0.1:22990: Continue Reading »