mysqlconf – code.openark.org http://shlomi-noach.github.io/blog/ Blog by Shlomi Noach Mon, 16 Apr 2012 09:00:17 +0000 en-US hourly 1 https://wordpress.org/?v=5.3.3 32412571 It’s *that time* of the year https://shlomi-noach.github.io/blog/mysql/its-that-time-of-the-year https://shlomi-noach.github.io/blog/mysql/its-that-time-of-the-year#comments Mon, 16 Apr 2012 09:00:17 +0000 https://shlomi-noach.github.io/blog/?p=4824 Even without attending the Percona Live conference in Santa Clara, you could tell something big was going on.

One way of measuring it was by looking at the flow of announcements. Here’s a brief list, and apologies if I’ve missed anyone:

All within the first days of the conference.

What this means, over the surface

I read a post by someone who was ranting about Oracle making so many announcements just as the conference began. He obviously suspected there was no coincidence. I got the impression he was looking at it the wrong way: as if Oracle’s announcements came to discourage the relevance of the conference.

I beg the opposite.

Obviously no one is insinuating the timing is coincidental. This does not mean, though, that by announcing new features companies try to undermine the conference. On the contrary: it’s part of the celebration. The days of the conference are full of excitement. People are meeting, sharing experiences. It’s a great opportunity to throw in a few more goodies and let everyone enjoy themselves.

No new development can make a conference’s talk obsolete, as was insinuated by another’s post. We all know it takes time for new released to become widespread. So it just adds up to the excitement that we not only have great fun now, but we are expected to enjoy new features to be stable by next year.

What this means, under the hood

To make my point even more interesting, consider that it takes a huge amount of energy to have a release, or a set of features to be released at a specific date. You won’t hold out for a stable release for 4 months. You won’t rush a premature release by 3 months.

It follows that many companies were planning these releases months ahead. Hold on. they were planning these releases months ahead to match the dates of the Percona Live conference. I don’t look at this as undermining the conference: I see this as showing confidence in the conference. The conference will be great, so our announcements will play well!

Even more under the hood

There is really nothing special about it. You see this happening in other conferences as well. LinuxCon is full of announcements. MySQL’s case is actually better. While Linuxcon suffers from premature announcements of new patches, with keynotes and sessions describing those patches, patches that are quickly discarded a few months later, we do happen to work with stable projects and products. No one is immune from the forces of economy, but we usually enjoy reliable announcements.

And, an interesting phenomena is created: we get a release cycle.

Everyone is eager to announce something at the conference; so we get to expect releases on the conference. With Oracle throwing another conference this fall, we can expect even more announcements. Not unlike Ubuntu’s release cycle – April & October, Tick Tock, Tick Tock, it’s time for a release.

For all these I congratulate Percona on a job well done!

]]>
https://shlomi-noach.github.io/blog/mysql/its-that-time-of-the-year/feed 2 4824
common_schema talk at Percona Live https://shlomi-noach.github.io/blog/mysql/common_schema-talk-at-percona-live https://shlomi-noach.github.io/blog/mysql/common_schema-talk-at-percona-live#respond Sun, 08 Apr 2012 16:57:15 +0000 https://shlomi-noach.github.io/blog/?p=4809 Are you attending PerconaLive?

Allow me to suggest you attend the Common Schema: a framework for MySQL server administration session on April 12, 14:00 – 14:50 @ Ballroom F.

This talk is by none other than Roland Bouman. Roland co-authored parts of common_schema, and is a great speaker.

I have a personal interest, of course, being the author of most of the components in common_schema. I would like to convert you to a supporter of this project. I know a few very smart people who think this project is an important tool. I would like more people to get to know it. Eventually, I would like developers and DBAs alike to consider it an inseparable part of any MySQL installation.

Then I shall have world domination, Bwa ha ha!

PS,

Have fun, I will unfortunately not attend myself this year. Having been on the program committee, I can tell it’s going to be a great conference!

]]>
https://shlomi-noach.github.io/blog/mysql/common_schema-talk-at-percona-live/feed 0 4809
Call for Nominations for 2012 MySQL Community Awards https://shlomi-noach.github.io/blog/mysql/call-for-nominations-for-2012-mysql-community-awards https://shlomi-noach.github.io/blog/mysql/call-for-nominations-for-2012-mysql-community-awards#comments Mon, 13 Feb 2012 12:13:18 +0000 https://shlomi-noach.github.io/blog/?p=4715 This post complements Henrik’s Call for Nominations for 2012 MySQL Community Awards.

Recap: we keep the tradition of awarding MySQL community members for their notable contributions to the MySQL ecosystem.

Previously, the awards were given by MySQL AB/Sun. Later on they were given by the community itself, as will follow this year, when the awards are presented during the Percona Live MySQL Conference & Expo in Santa Clara, this April.

Henrik describes in details the three categories: community contributor, application, corporate contributer -of the year.

A bit more about the categories

To add to Henrik’s description of the categories, keep in mind the following:

  • Community member would be a person. That person could be working by himself/herself, or can be part of some corporate. It does not matter. What matters is the person’s contribution.
  • Application: some code or project which either enhance/complement MySQL (e.g. Replication/HA solution) or uses MySQL. In the latter case, it is important that MySQL role’s in the application is significant. For example, some popular site built with some CMS using MySQL may not qualify, if it could run just the same with PostgreSQL or other databases, or if the owners are not actually aware or at all mindful about the specific database technology they are using.
  • Corporate: we’re still figuring this one out. The general idea is to acknowledge a corporate which, some way or the other, did a good thing to the MySQL ecosystem or the community.

A bit more about the nomination and voting process

Anyone can nominate anyone! Please send your nominations by mail to mysql.awards [at] gmail.com. Please specify who/which your nominee is, in which category, and why. Kindly include disclosure of relationship with nominee (this does not disqualify, we just want to have everything in the open), if applicable.

At this stage we are just collecting as many names as we can, so please keep these coming!

When all names are collected we open a discussion between voting committee members, followed by a poll on each category.

Traditionally, there have been three winners in each category. This is no longer the case, and has not been as such in the previous two years. Now, there are no strict rules to that; we go with the flow. If there just aren’t enough votes for some nominee, we may choose not to award said nominee. So, if two nominees get a high number, say 5 votes each, from the community, and the next one only gets one vote, then we will probably only award the two. This is roughly what happened last year.

I say “there are no rules” because we follow our common sense and the insights provided by the various members of the panel.

A bit more about the people involved

The voting panel is comprised of winners from previous two years, in addition to the conference’s chairman and chairwoman. I think the list of people involved is diverse and balanced.

Henrik and myself are secretaries, which means we manage the discussion board and the voting process and participate in the discussions, without voting ourselves. We kinda just took the position to ourselves, it looks great on our resume. We can later apply for secretary jobs anywhere we like.

So as you can understand, this whole thing is just a community effort to keep up a good tradition. There are no written standards or guidelines, just the common sense, goodwill and etiquette of the people involved.

]]>
https://shlomi-noach.github.io/blog/mysql/call-for-nominations-for-2012-mysql-community-awards/feed 3 4715
Percona Live: MySQL Conference And Expo 2012 – a note on proposals https://shlomi-noach.github.io/blog/mysql/percona-live-mysql-conference-and-expo-2012-a-note-on-proposals https://shlomi-noach.github.io/blog/mysql/percona-live-mysql-conference-and-expo-2012-a-note-on-proposals#comments Tue, 29 Nov 2011 11:08:05 +0000 https://shlomi-noach.github.io/blog/?p=4485 As a member of the conference committee I review the session and tutorial proposals for the Percona Live MySQL conference in Santa Clara, 2012.

The sessions are expected to be technical, and I’m happy the proposals follow this guideline. I use Giuseppe‘s and Baron‘s general guidelines for submitting a proposal. I wish to humbly add a couple notes myself.

Be fairly brief

Explain your session/tutorial as clearly as you can. The reader should be able to get the general impression of the session from two or three paragraphs. Some can make a point in two sentences; for most it would take somewhat more.

If you’re going to talk about some database feature, for example, please do not write the manual for that feature. That’s for the session itself. Just explain how you’re going to discuss the feature, and why it should be of interest (e.g. what the benefits of this feature are, the risks or pitfalls, the ingenious C code behind it or the quirks of the operating system involved).

Clarify

It’s important for me to understand two things when reading a proposal, which establish the grounds for better evaluating the proposal:

  • Who the target audience is (newbies, developers, DBAs, Linux internal experts etc.)
  • To what depth are you going to deliver the content you describe.

That is not to say you should explicitly state “This session is for MySQL DBAs”, but the attendee should be able to easily decide whether your session appeals to his type of work or expertise. I, myself, have happened upon sessions that were completely different from what I expected. To illustrate, I give two examples, while not disclosing the exact details:

  • A session which was about locking in database. I got the impression it was about ways to avoid locking, issues with mutexes etc. It turned out to be a discussion between the presenter and a few member of the audience about the specific code internals, lines 667-684 in the lock_module.cc file, and the recently reported bug. To me it was more like the weekly rnd meeting of some company. I couldn’t understand anything of the entire talk.
  • A session promising insight on working out great scale-out with some product: I was expecting to hear of the “DOs and DON’Ts”, or of great configuration and implementation tricks on the subject. However, it turned out to be more of a general talk on “how we used the product in our company and found it to work great”.

The two sessions above were perfectly valid, and had their place in the conference. But were poorly described in the two respects I mentioned.

A great submission, in my opinion, is one where attendees get what the expect, and don’t shyly leave the conference room 15 minutes into the talk.

Submit a proposal here.

]]>
https://shlomi-noach.github.io/blog/mysql/percona-live-mysql-conference-and-expo-2012-a-note-on-proposals/feed 2 4485
Impressions from MySQL conf 2011, part IV https://shlomi-noach.github.io/blog/mysql/impressions-from-mysql-conf-2011-part-iv https://shlomi-noach.github.io/blog/mysql/impressions-from-mysql-conf-2011-part-iv#comments Tue, 19 Apr 2011 12:08:40 +0000 https://shlomi-noach.github.io/blog/?p=3474 This post concludes my impressions from some of the talks I’ve been to.

Thursday

I opened this day’s sessions with a smile.

Antony Curtis and Arjen Lentz have authored the OQGraph storage engine a while back. I had every intention to try it out, but never got round to it. Which is why I was happy to find this session in the conference. OQGraph is a specialized engine, and comes to solve hierarchal or otherwise graph-related queries, such as: “who are the descendants of a given node”, “find a route from a to b”, etc. MySQL does not support the RECURSIVE syntax as Oracle does, and does not provide out-of-the-box solution for that.

The syntax is just excellent: you just write something like SELECT * FROM my_graph WHERE from_node = ’empusa fasciata’ AND to_node = ‘homo sapiens’ to find a route. Otherwise just use WHERE from_node = ‘Heathrow Central’ to find all outgoing links. So this is just plain old SQL, no new syntax involved.

I rounded corners. It is also possible (and required) to specify an algorithm. Do you want Djekstra? BFS? You specify it in the query. The result of a route query is a rowset, where each row is a step in the route, along with its sequence within the route. So you can do your ORDER BY, LIMIT etc. I find that syntax-wise, OQGraph is very intuitive!

Implementation-wise, there are two versions: 2 (stable) and 3 (testing), which are very different. In version 2 you must create and populate an OQGraph table. You must populate it with from-to;weight values. The entire table is stored in memory a-la MEMORY engine. In version 3 the OQGraph is not directly populated. Instead, you use the CONNECTION table property to identify a real table where the data resides, along with the names of the relevant columns. So your data can reside within your normal InnoDB table, and your queries will follow your normal isolation-level rules.

Graph search implementation is done via the boost() library. This means that information must be loaded in memory. Tests show that up to a million or a few million edges are as much as OQGraph can take. So right now it may not fit your requirements is you need larger datasets.

Antony seemed like he was enjoying every minute of writing OQGraph, was very aware of current limitations, and offered great ideas on improvement. The version 3 design is a huge improvement over version 2, and I can’t wait to see how it evolves!

An interesting view into Facebook’s attempt to locate & define data drifts within MySQL replication. Apparently, Facebook guys realized there were some inconsistencies between masters and slaves. They embarked on a mission to find out what exactly was inconsistent, and why.

Daniel Peek described their method of detecting changes. In similar manner to Maatkit’s mk-table-checksum, they use hash codes over chunks of rows. To be more specific: in order to detect the differences (if any) between master & slaves on a given table, they iterate said table in chunks, such that first line of any chunk overlaps with last line of previous chunk. This is easily done when there’s unique key, less so when there isn’t. They copy rows to a utility table a chunk at a time (e.g. 1,000 rows at a time), and take checksum. Their script then compared checksum on master & slaves. If it equals, they conclude (at very high probability) that tested chunk is identical. Otherwise they take action to detect the exact row that is changed.

When the two chunks do not have the same number of rows, then there’s at the very least a missing or extra row in either master or slave. Otherwise, there’s a change in column’s data. Apparently, nearly all data drifts are changes in column data, not missing or extra rows.

To make a long story short, they realized most of their data drift related to TIMESTAMPs. After realizing some timezone settings were incorrect, they were still left with 0.00056% (if my memory serves me right) worth of data drift. How often do they do this test? Once, is the reply. What is the meaning of 0.00056%? We don’t know, is the answer.

And, although we are left with unanswered questions: Why does that happen? How can we detect such changes in reasonable time? What should we do once we realize there is a drift? Whom shall we trust? We have gone to length to recognize a way to detect such drifts.

The conference had to end with a bang. In the last three years (hopefully with some sleep), Justin Swanhart has been working on what seems to me like an utterly (positively) crazy project: Flexviews: a materialized views solution for MySQL.

MySQL’s native views are immaterialized. There is no data in them. When you access the view, you directly query the tables beneath. Justin’s code allows you to create a materialized view (implemented as an actual MySQL table), which can utilize almost any type of SELECT you would want to use. It can only SELECT tables (including other materialized views).

How does this work? You don’t directly create your materialized view. You get a set of stored routines with which you create the view. Each provides with a simple step in the creation of the view (e.g. JOIN to this table, return this column, etc.). You don’t have to worry too much about this, since Justin provides with a PHP script which translates a SQL query to the required invocation sequence of said routines.

OK, view is created. What then? Flexviews reads binary logs (must be in ROW binlog format), and decides whether logged action manipulates tables used by any materialized table. If so, it logs that action into a special log table.

What have we got so far? SQL parser which generates sequence of stored routines; sequence of stored routines which creates a table, while storing structure; Binlog reader, capable of recognizing relevant events, associating them with relevant materialized views; and, most importantly, some brains which can figure out just how said event affects data. Think about it: say our query did some aggregation, with COUNT(*) or SUM(column). To be able to update the aggregated result based on the fact a few rows have been added/deleted/updated sounds to me like very hard work.

Flexviews do not get updated immediately, but rather on demand. It’s all in the logs, it’s just a matter of when to apply the logs. We had a very nice & simple demonstration of this.

I don’t mean to be too superlative, but this was a very impressive session, and myself, as well as other people, all had an occasional “wow” slip during and concluding this talk.

Throughout the week

  • Planet MySQL

During conference, Planet MySQL was swamped with various announcements. Little were technical announcements, like “X.X has been released”, and most were purely marketing announcements. This is to be expected; everyone wants to make a living. I think it is quite all right that for 4 days a year, the planet is mostly marketing stuff. I personally mostly ignore the planet during these days.

]]>
https://shlomi-noach.github.io/blog/mysql/impressions-from-mysql-conf-2011-part-iv/feed 3 3474
Impressions from MySQL conf 2011, part III: BoF Replication https://shlomi-noach.github.io/blog/mysql/impressions-from-mysql-conf-2011-part-iii-bof-replication https://shlomi-noach.github.io/blog/mysql/impressions-from-mysql-conf-2011-part-iii-bof-replication#comments Tue, 19 Apr 2011 04:11:14 +0000 https://shlomi-noach.github.io/blog/?p=3528 This post continues my impressions from some of the talks I’ve been to. I’ll dedicate this post to a single session.

Wednesday

  • BoF: Replication, Lars Thalmann, Mat Keep (Oracle)

Lars Thallman presented the MySQL 5.5 replication features, as well as the expected 5.6 features. Among other features, one could notice parallel replication, binlog checksums, sub-second resolution and more. There was an open discussion about these features, asking for comments; looking for new ideas and suggestion from the audience.

I can’t possibly cover it all. I’ll note two discussion I participated in, and which have interested me. This also serves for noting down to myself my ideas and thoughts.

Parallel replication

As the trend goes, parallel replication will be implemented by having a thread per schema. My own contribution to the discussion is that with such paradigm, the slave may be at any state inconsistent with the master. That is, at any point in time it can be in such state that has never existed in any point in time on the master. This is because it is possible that one or more of the replication threads executes much faster than one or more of the others.

We have discussed the meaning of STOP SLAVE. I have suggested that STOP SLAVE works in such way that it stops in a consistent slave. Lars suggested there could be normal STOP SLAVE and STOP SLAVE CONSISTENT.

It has been noted that we may not care about the slave being inconsistent, as the prerequisite for parallel replication is that the distinct schemata are independent of one another. After some thought, I disagree. Today parallel replication may be implemented on different schemata; tomorrow – based on finer parameters. I think it’s best to prepare for the inevitable future. Besides, STOP SLAVE INCONSISTENT is such an easy feature to implement!

Slave analysis

I also had a long time feature request. While writing management tools, I realized it is difficult to learn the topology of a replication environment by simple interrogation. If you connect to a master, you can either issue SHOW PROCESSLIST to detect the slaves, or SHOW SLAVE HOSTS.

The former does not tell you the port on which the slaves are listening. The latter requires that you configure report_host and report_port properly.

What I would like to see is a SHOW SLAVES STATUS command (or similar INFORMATION_SCHEMA table) on the master. Such that it lists connected slaves, one per row, with the following details:

  • Slave_host (this is easy, already exists in SHOW PROCESSLIST)
  • Slave_port (slave should send this kind of information to master upon first connect)
  • Slave_master_log_file (last log file requested by slave)
  • Slave_master_log_pos (last log pos requested by slave)
  • Slave_seconds_behind_master (slave should report this upon each request)
  • Slave_server_id (slave should send this upon connect)

By reading Slave_host, Slave_port & Slave_server_id, it makes it easy for us to build replication topology. It’s a single operation to detect a master’s slaves. It takes recursive steps to build entire topology.

By reading the Slave_master_log_file, Slave_master_log_pos, we can easily determine that binary logs up to that point have already been written to the slave’s relay log. This makes it easier to decide how to purge master logs. Slave_seconds_behind_master completes the replication picture.

]]>
https://shlomi-noach.github.io/blog/mysql/impressions-from-mysql-conf-2011-part-iii-bof-replication/feed 4 3528
Impressions from MySQL conf 2011, part II https://shlomi-noach.github.io/blog/mysql/impressions-from-mysql-conf-2011-part-ii https://shlomi-noach.github.io/blog/mysql/impressions-from-mysql-conf-2011-part-ii#respond Sun, 17 Apr 2011 12:02:35 +0000 https://shlomi-noach.github.io/blog/?p=3506 This post continues my impressions from some of the talks I’ve been to.

Wednesday

Grant McAlister described the Amazon RDS offer, which provides with a pre-installed MySQL servers, and supports auto management of replication and high availability. He desribed asynchronous vs. synchronous replication, logical (i.e log shipping & replaying) vs. physical replication.

Amazon implement physical replication by shipping data pages to a secondary, standby server, located at a different availability zone. A transaction does not complete before pages are shipped to, and acknowledged by standby machine. The standby machine writes data pages in parallel. This is similar in concept to DRBD. RDS uses InnoDB, which promises data integrity in case of power/network failure.

The fail over process, in case active master has crashed, involves blocking access to the active master, starting MySQL on standby master (promoted to be active), while changing elastic IP for master to point to promoted master. McAlister said this process takes a few minutes. Live demo resulted at about 4 minutes.

Actually, the recovery time is affected by the InnoDB transaction log. In replying my question, McAlister explained that the InnoDB trasaction log is not under the user’s control. It is managed by Amazon. You can’t just set it to 512MB.

Otherwise RDS also offers normal, asynchronous replication, so as to allow for read scale out.

I later on spotted Grant McAlister in the expo hall, and had the chance to ask a few more questions. Here is a very brief summary of a few Q&A items:

Q: Today, EBS performance is unexpected, since other users may be using the disk. Will you allow RDS customers to have dedicated EBS volumes?

A: We’re working towards making this work.

Q: In many cases I’ve seen, EBS is very slow storage. People do software RAID over several EBS volumes to boost performance, but then loose the ability to do consistent snapshots. Any solution for that?

A: We get so much of that! We’re working to change this as well, in time.

Q: Does an RDS user get access to the DB machine?

A: No. Though you do get the API for monitoring it.

Q: How much control do I get over the MySQL configuration file?

A: You have control over most of the settings; a few are under Amazon’s control (e.g. control over innodb transaction log file size)

Q: How far have you went into changing MySQL code?

A: A little. Not far enough that compatibility breaks. People need compatibility. We also need to be quickly able to provide with new versions.

Q: If I need to upgrade MySQL version, does RDS provide with the mechanism to first upgrade slave, then do fail-over?

A: No. We just upgrade a server (i.e. master) in place.

Q: Any integration of collateral tools?

A: No. You have access to the database; you are free to run your own tools.

I hope my memory serves me correctly in recalling the above conversation.

Baron presented with some of Aspersa’s tools: ioprofile, diskstats, summary, mysql-summary, collect, stalk & sift.

The presentation was accompanied with demos, wither using live data or prepared, real world data. Not surprisingly, Baron came up with real customer cases, where the “standard” tools gave little help in way of analyzing the problem (e.g. iostat not separating read and write counters). Apparently this is a bash/sec/awk based toolkit, with bash unit tests. Cool!

Some of the tools are self explanatory; others require deeper understanding of either OS internals or of the patterns one should be looking for. Lot’s to study.

Baron writes tools based on extensive experience of consulting & performance optimization work; not only of his own, but of Percona’s entire team. When he says “this tool saves a lot of analysis time” or “with this tool we were able to analyze a problem we were unable to, or found it very hard to, solve with existing tools”, you’d better check it out. In fact, I noticed the audience laptop usage during this talk was very low. There was a lot of attention in the crowd.

This session described the replication topology and failover design on one (undisclosed) of Google’s products, using MySQL. Using XXX shards of MySQL, that is, each with XX servers in replication topology (we are asked to do the math ourselves).

Ian & Eric described the logic behind their sharding, scale out & failover process, and the sad reality. Apparently shards are implemented on many levels: multiple MySQL instances on single machine, multiples schemata in each MySQL instances, sharded data within schema. For scale-out, they prefer adding new machines, then distributing MySQL instances between them. There is nothing too complicated about this, once we are past syncing masters & slaves.

Speaking of which, their preferred way of taking snapshots is MySQL shutdown, then full file system copy. They do not want to bother with licenses. With XX servers, it should be fairly easy to take one down for backup/copy.

Otherwise they may be splitting schemata between servers, or, at the worst, managing the shards. An interesting note was that Google is known to use crappy hardware, which makes for a lot of hardware failures. Apparently, their solution needs to work a lot.

A very nice notion they presented is the empty shard. They have an empty shard, with complete replication topology, on which version upgrades or otherwise tests are done. Very nice! (Forgive me if this is plain obvious to you).

The logic part detailed the expected “which slave to promote”, “how to modify the hierarchy” etc. procedures.

Besides being very interesting in itself, the session also posed the interesting notion, that most everyone (and in particular the BIG companies), use plain old MySQL replication for both scale out & high availability. I think this is an important notion.

I’ve heard a lot about Gwen Shapira before, and this is the first time I heard her speak. I enjoyed her session. Gwen presented a vendor-neutral session on sharding. There were a lot of “how not to shard” notes. These were based either on her very early experience, before the word “sharding” came to be known, as well as recent experiences. The stories and examples were instructive and amusing (lot’s of “war stories”). She presented with general guidelines on proper sharding, but, as the title suggests, presented no silver bullet. Actually, she presented with counter cases where a non-desirable sharding solution for one company, may yet be viable for another.

I went to see my own talk that day :). I was honored to have good audience and I’m thankful for their attention. I gained good participation, and a couple feature requests/bug reports.

]]>
https://shlomi-noach.github.io/blog/mysql/impressions-from-mysql-conf-2011-part-ii/feed 0 3506
Impressions from MySQL conf 2011, Part I https://shlomi-noach.github.io/blog/mysql/impressions-from-mysql-conf-2011-part-i https://shlomi-noach.github.io/blog/mysql/impressions-from-mysql-conf-2011-part-i#comments Fri, 15 Apr 2011 02:10:31 +0000 https://shlomi-noach.github.io/blog/?p=3485 Having the conference behind now, I’m reviewing some of my impressions and of sessions I attended.

The people

To begin with, this conference was a big success for me, in many respects. The sessions were great (more on that later), but of course, meeting with new people and with familiar people, was the more important part.

I live in Israel, which makes travel to the US very long and expensive. Apparently not many MySQL community members in my neighborhood, so I don’t ever get to meet the faces. The conference makes that possible. I did not participate in all community events, as I had scheduled calls with little girls who miss their father. And I was very much under jet lag. And I have more excuses on demand.

But I did get to meet known faces; people I only knew by name; unfamiliar people who were familiar with my work (fun!); and otherwise just (ex-)strangers.

The sessions

There was a variety of sessions to choose from. Many times, I had to pick one out of two or three sessions I was interested in, running at the same time. Not all sessions appeal to one in the same way, but looking back, I find there were a lot of GOOD sessions I attended. I mostly like sessions that are very technical; preferably drilling into details of algorithms & implementation.

I wish to review some of the sessions I attended. I can’t possibly review them all, so my apologies to those I left out.

Monday

This is actually the first time I learned about the internal details of Tungsten. Robert, CEO of Continuent & original developer of Tungsten, did an excellent job in reviewing the various aspects of Tungsten. But rather than being a promotional talk, this was a purely technical tutorial.

So it was interesting to understand the difficulties in building Tungsten, at least on the MySQL part. For example, the Tungsten replicator must do its own filtering of events based on server id, since, although a replicating slave knows how to ignore circular queries issues by itself, there is no client API to do so; you can’t “send over” the server id along with a query and expect the server to ignore it.

As seems to be the current and emerging paradigm, parallel replication is possible by paralleling queries from different schemata. But apparently Continuent seem to want to change that. That is, there’s nothing that strictly forces that this must be the paradigm; it’s just the current implementation.

Giuseppe was in charge of live demos, and has shown stuff like slave disconnect, fail over, multi-master replication etc. Edward Archibald later presented a cluster-like solution based on Tungsten and a replication aware connection proxy.

There was plenty of time for asking questions. Interesting technical issues were discussed and answered. It was also stated very clearly what Tungsten does not do for you, or when it may actually be the gun with which you shoot yourself in the foot. Good talk!

Tuesday

Roland is now passionate about finding an easier query language for web applications to use. Rather than SQL, he suggests using MQL (created by Freebase). MQL uses JSON (-like) syntax to describe both queries (requests) and result sets (responses). I won’t go into the details since Roland himself laid them out on his blog.

As usual with Roland, his session is well thought of. He describes an elegant, appealing syntax. In my opinion, MQL is an intuitive language. It is easily comprehensible by both DBAs and developers. Possibly by my mother.

However Roland goes further and offers his MQL-to-SQL, an open source implementation of a translation tool between the two languages. In his presentation, he demonstrated some simple and some not-so-simple queries, generated by MSQL, translated to SQL, the results of which translated back to MQL.

MQL is not an all-purpose solution. You don’t do analytics with MQL, but it does seem to answer some problems that other ORMs (e.g. Django) are, to the best of my knowledge, unable to perform in convenient manner (that is, without reverting to some SQL-like syntax).

Special thanks

Thanks to Giuseppe Maxia, when on MySQL Replication Advanced Techniques he presented some of the openark kit tools, and, having located me in the crowd, invited me to speak.

To be continued

I need to prepare for my flight. Will probably publish next part(s) once back home. Hope to see you all next year!

]]>
https://shlomi-noach.github.io/blog/mysql/impressions-from-mysql-conf-2011-part-i/feed 5 3485
Slides for “openark-kit: MySQL utilities for everyday use” https://shlomi-noach.github.io/blog/mysql/slides-for-openark-kit-mysql-utilities-for-everyday-use https://shlomi-noach.github.io/blog/mysql/slides-for-openark-kit-mysql-utilities-for-everyday-use#comments Wed, 13 Apr 2011 22:16:20 +0000 https://shlomi-noach.github.io/blog/?p=3480 Today I have delivered my talk, openark-kit: MySQL utilities for everyday use, at the O’REILLY  MySQL Conference 2011.

The slides are uploaded to the O’Reilly site, and I’m attaching them here as well. Feel free to download the PDF: openark-kit-mysqlconf11.pdf

I wish to thank all who attended my talk!

 

]]>
https://shlomi-noach.github.io/blog/mysql/slides-for-openark-kit-mysql-utilities-for-everyday-use/feed 3 3480
Would you be my friend on mysqlconf? (tempting offer inside) https://shlomi-noach.github.io/blog/mysql/would-you-be-my-friend-on-mysqlconf-tempting-offer-inside https://shlomi-noach.github.io/blog/mysql/would-you-be-my-friend-on-mysqlconf-tempting-offer-inside#comments Wed, 09 Mar 2011 06:22:37 +0000 https://shlomi-noach.github.io/blog/?p=3262 I’m still throwing papers to the trash and starting all over, fixing, rewriting and improving my talk at mysqlconf 2011, where I will be presenting openark-kit: MySQL utilities for everyday use.

However I’ve got something up my sleeve: a benefit many can enjoy, that’ll make me a respectful, popular and sought after speaker. While others may try and lure you with such earthly temptations as a 20% off discount, I am in a position to offer you a more spiritual gift: my friendship!

See, if you become my friend, I can offer you a 25% discount on the MySQL conference. Yes, that’s 5% more than my competitors! The only thing I ask in return is that you be my friend (hey, it’s called “friends of speaker”). Not like a FB virtual friend, but a real friendship! One where you can buy me beer or dinner!

If you agree to such humane terms, I will be in the position to let you know that all you have to do is fill in mys11fsd in your registration form.

No, wait! I let it slip! Rewrite: You should fill in mys11fsd [will only tell you this password after your commitment to a beer] in your registration form.

Oh no, not again!

Don’t use mys11fsd without talking to me first… You’re not supposed to… Oh, my beer!

Argghhh!

]]>
https://shlomi-noach.github.io/blog/mysql/would-you-be-my-friend-on-mysqlconf-tempting-offer-inside/feed 4 3262