If you're unfamiliar with mycheckpoint, well, the one minute sales pitch is: it's a free and open source monitoring tool for MySQL, which is extremely easy to install and execute, and which includes custom queries, alerts (via emails), and out of the box HTTP server and charting.

This is mostly a maintenance release, with some long-time requested features, and of course solved bugs. Here are a few highlights:

• Supports MariaDB and MySQL 5.6 (issues with new variables, space padded variables, text-valued variables)
• Supports alerts via function invocation on monitored host (so not only checking alerts via aggregated data like 'Seconds_behind_master' but also by SELECT my_sanity_check_function() on monitored instance). See alerts.
• Supports single-running-instance via "--single" command line argument
• Supports strict sql_mode, including ONLY_FULL_GROUP_BY, overcoming bug #69310.
  
• Supports sending of pending email HTML report
• Better re-deployment process
• Better recognizing of SIGNED/UNSIGNED values
• Some other improvements in charting, etc.

mycheckpoint is released under the BSD license.

Downloads are available from the project's page.

oak-online-alter-table Changes / bug fixes include:

• Support for keyword-named columns
• Use of FORCE INDEX due to lack of MySQL's ability for figure out the chunking key at all times
• --sleep-ratio option added; allows for sleep time proportional to execution time (as opposed to constant sleep time with --sleep)
• Support for chunk-retry (in case of deadlock) via --max-lock-retries)
• Fixed order of cleanup
• Fixed bug with verbose messages with non-integer chunking key
• Fixed bug with single-row tables (people, no need for this tool for single row tables )
• Friendly verbose messages to remind you what's being executed

oak-chunk-update changes includes:

• Verbosing query comment if exists (friendly printing of what's being executed)

(Do check out QueryScript's split(); it's a simple, server side solution which works almost same way as oak-chunk-update)

More issues and changes not listed here.

Download

openark-kit is released under the new BSD license, and is freely available.

• Download latest openark-kit revision (#196).
• Browse openark-kit documentation.

Note: I was using what seems to be the "old" method of compiling; quoting Leif Walsh:

... We are looking at deprecating that method of building (MariaDB source plus binary fractal tree handlerton).  It only really needed to be that complex when we were closed source.

I also tried the "new" method of compiling, which I couldn't work out.

Here's how it goes: TokuDB is newly released as open source. As such, it got a lot of attention, many downloads and I hope it will succeed.

However as stable as the product may be, it's new to open source, which means anyone compiling it from source is an early adopter (at least for the compilation process).

Installation process

This is an unorthodox, and actually weird process. See section 6 on the Tokutek docs. In order to compile the project you must download:

• The source code tar.gz
• And the binary (?!) tar.gz
• And the binary checksum
• And the Tokutek patches
• And the patches checksum

You extract the source tarball. But instead of doing the standard "./configure && make && sudo make install" you need to copy a shell script called tokudb.build.bash one directory level up, and run it from there.

tokudb.build.bash lists gcc47 and g++47 on rows 3, 4. Modify "gcc47" to "gcc", modify "g++47" to  "g++". I'm assuming you don't have a binary called gcc47. Why would you?

Dependencies

You will need CMake >= 2.8

This means Ubuntu LTS 10.04 users are unable to compile out of the box; will need to manually install later version of CMake.

Also needed is zlib1g-dev, rpmbuild.

While compiling

I ran out of disk space. What? I was using a 10G partition I use for my compilations. Looking at "df -h" I get that:

• The source tarball is extracted (I did it)
• The binary tarball is also extracted (someone has to explain this for me)
• And inside the source directory we have:

bash$ df -h
...
1484    build.RelWithDebInfo.rpms
5540    build.RelWithDebInfo

At about 7GB (and counting) of build... stuff?.

UPDATE: just ran out on disk space again. Is this an incremental thing? Like every time my compilation fails and I recompile some files are not cleaned up? If so, put them on /tmp! OK, moving everything to a 300GB partition and starting all over.

More while compiling

I got errors on missing libraries. Like I was missing libssl, rpmbuild. This is what the "configure" script is for -- to test for dependencies. It's really a bummer to have to recompile 4-5 times (and it's a long compilation), only to find out there's another missing package.

After compiling

What is the result of the compilation? Not a "make install" prepared binary. The result is a MySQL-binary package. Se need to extract and put on /usr/local/somewhere etc.

Conclusions

The compilation process is unexpected and non-standard. The output is unexpected.

The correct way of doing this is a "./configure && make && sudo make install". I don't understand the need for a binary package while compiling from source. Isn't this the chicken and the egg?

A source distribution is no different from a binary distribution. You must have a testing environment to verify the source distribution actually works. This test environment is typically a bare-new-RedHat or a bare-new-Ubuntu etc. The machines at Tokutek are already installed with needed packages. Not so on my compilation machine. I suggest that apt-gets and yum installs for dependencies are added to the source distribution testing. This is the only reliable way for you guys at Tokutek to know that clients will actually be able to install via source.

Question #1

Brian Aker (HP) asks Simone Brunozzi (Amazon) what the underlying technology for DynamoDB is. Simone says can't disclose. Brian says: "it's MySQL!!". Simone says: "can't disclose". Brian insists: "it's MySQL!!"

Seriously? I will be very much surprised to learn that DynamoDB uses MySQL; it doesn't make sense to me. Why would Brian Aker say that though? Did he just mean to tease Simone or is there something I just don't get?

(Yes, Brian?)

Question #2

Matt Aslett speaks about adoption of MySQL & variants, and expected adoption in next years, Mentions MariaDB, Percona Server, SkySQL. KEeps saying how the SkySQL server gets more traction.

What does he mean? There's no SkySQL fork; does he mean SkySQL cloud offer? Or just SkySQL support services, typically for MariaDB variant? But in that case, SkySQL is out of context. What's going on?

Question #3

Matt Aslett presents quite pessimistic prediction for MySQL. Reduced popularity in next years. Relatively good news for MontyProgram/MariaDB; otherwise a lot of switch-off to PostgreSQL, poor adoption for Continuent, low ratios of "evaluation" to "adoption" of technologies; really quite depressing. Later mentions at about 200+ questionnaires.

I don't have a special interest here as I don't work for any mentioned company; other than my general desire to see the ecosystem flourishing.

Are 200+ people enough to both give a faithful picture of current MySQL usage and adoption? Are they enough for prediction 1 year into the future? 4 years into the future?

In Israel, with less than 8M population, election surveys usually look at 500+ people. To me it doesn't sound a lot, but statistics is not my strong skill. However those picked for the survey have to be a diverse population, distributed in similar ratio to overall population (so Jews, Muslims, Christians, Orthodox, income level, geographic location, etc.).

Does the same happen with those 200+ questionnaires in the 451 research? The reason I ask is this: at the end of keynote Matt says: "if you want your voice to be heard, if you think differently, contact me, and I'll add you to our survey". Does this mean anyone stepping up is included? Great, so a hypothetical company called GalleriaDB would encourage its 50 employees to enlist, thereby completely shifting the balance.

Who are those 200+ people in the survey? World wide known experts? Your regular DBA? Your remote DBA consultant? Your web developer? Do they represent the overall MySQL ecosystem population? Do their insights into the future collide with those of everyone?

Please discuss below

common_schema booth in Percona Live 2013 (someone left their soup on our table)

Allen Kinnard

Yay! I got a booth!

I confess it was mostly deserted. My hero Allen Kinnard, whom I've never met before, was kind enough to volunteer to occupy the booth.
So between him and me, both of course also looking to visit other booths and talk to people, the common_schema booth was only moderately attended.

Well, this was the DotOrg Pavilion: free booths for free & open source projects (e.g. OpenStack, PhpMyAdmin, etc.).
We both did our best to explain common_schema to the visitors.

The booth was actually titled "common_schema & openark-kit". However I don't recall that anyone asked me about openark-kit. Most were just interested in what common_schema was. I get that openark-kit is well known by now to many.

We did not have so many visitors, which played well with our occasional absence. But this was a last moment arrangement. In future events I may try to get things on top and have an army of volunteers to help me out (and I hope that by next time common_schema is widely used).

Now for the criminal department: practically all other DotOrg booths came prepared, with printed material, giveaway stuff and the like. We were like way beyond "the minimalist". We were "the poor and the pitiful". So I asked Kortney if she could arrange me some decorations (pulling some strings - yeah!). We were thinking a bowl of candies or something. Eventually she brought a few dozen candies (no bowl) and laid them nicely on the table; very nice touch! Good mixture of colors, nice wave design.

And guess what? Visitors actually thought they may take one of our candies! Oh, I schooled them.
But on Wednesday, when we arrived to the booth, a horrid sight stroke us: someone stole (and probably ate) 80% of our candies!
Well, I hope you've got bellyache, and next time you should know there will be surveillance, candy-thief!

My talk was well attended, and I was fortunate to have a warm and engaged audience. Thank you to all those who attended, and thank you for those who provided feedback! Was happy to be able to present my work a great group of people.

You can find my slide either on Percona Live's website, or on Slideshare, as well as embedded right here.

I set two aims to my talk:

1. To have the audience know how to download and install common_schema (Check!)
2. Have everyone in the audience find one tool from common_schema that will make their day better. Quick raise of hands at the end of presentation: Check! All hands are up!

As I've noted following a question, rdebug is still in alpha. It modifies one's routines in such way that does not alter the routine's behavior, except:

• Affecting the value returned by ROW_COUNT() (I have no immediate solution to that)
• Also affecting result of LAST_INSERT_ID() (I expect to have this solved).

Thank you again; give common_schema a try. Best ways to support: submit bug reports, ideas, and above all: spread the word. common_schema for world domination!

Woo! Hard to sum up four excellent days. Shall I review the great talks I've been to? The keynotes? The well organized events?

You know what, skip it. There was ONE thing that overshadowed everything. It was the ONE thing for me that was the pure essence of the conference and its greatest joy:

Meeting and talking to a great many great people!

I was fortunate to meet up with so many people; none that I planned; things just went in such way that I engaged in so many conversations with so many people. I found myself talking about hamsters, peacocks, living in the village, living in the city, working from home, commute, relocation, working with your spouse, life in Israel, life in Argentina, prisons in the US, having many children, gun control, politics (heaven forbid!), fruit, vegetables, breakfasts, open source, community, buying (non-expensive) presents to your kids, new ventures, zen, philosophy, capitalism, socialism, books, being who you are, weddings, Java, scripting, NoSQL, how to contribute to an open source project, ... the list goes on.

I was "adopted" by the PalominoDB team at Pedro's, crash-partied on Pythian fellows, talked technical (or non technical) with Tokutek guys, meeting up with Oracle people (finally I get the faces behind the names!); the companies do not matter, I'm just throwing in names. The people are awesome! Representing their companies on the technical side, and being purely interesting people on the personal side, I met with men and women from all over the community. Apologies: impossible to list all nor account for!

It's great to have a place and time where we all meet together.

The bottom line? I am fortunate to have my current profession: I enjoy my work, and the people I meet are fantastic and of the highest quality!

Thanks all with whom I've met, and for all of those with whom I haven't had the chance to speak: see you next time!

• common_schema: DBA's framework for MySQL: an introduction to common_schema, my evolving server side solutions project. This will be a revised version of the talk I gave at Percona Live London; I have felt some weaknesses during that talk, which I've thrown out, letting room for cool stuff. I will discuss common_schema's various views, interesting and useful routines, the power of QueryScript, and a brief intro to the newcomer rdebug, debugger and debugging API for MySQL. If you're not familiar with common_schema, it's a good time to pick up on what I (being most biased) consider to be your smart assistant to MySQL maintenance and administration!
• The query which is the peak of my career: this is a 6 minute lightning talk. You're bound to attend if you're at the community reception (which you are), so I don't need to do promotional. You already payed the ticket and the doors will be locked. No escapees.

As far as I'm concerned the conference can be closed down the moment I provide these two talks, and we can all go to the beach.

Wait, no, I will also be at the DotOrg Pavillion at the Exhibit Hall, where I present common_schema and openark-kit. Come by to hear more about these!

In other words: let me take you through the steps for debugging your stored routines on your own server. We will step into, step over, step out, modify variables, set a breakpoint, run to breakpoint...

Command line geeks, this one's for you. GUI lovers, this is actually an API; I am hoping for someone wrap it up with a plugin for your favorite GUI editor.

Requirements:

• Install common_schema 2.0 or above (at this time of writing 2.0.0-alpha is released).
• Get sample data & routine file rdebug_demo_sql.txt
• mysql> SOURCE rdebug_demo.sql_.txt
  • You should now have a table called color_count in the test database, along with two routines: review_colors() and review_single_color().
• Open two sessions. We call them the debugger session and the worker session. The worker session will execute the routine; the debugger session will control it.

Walk-through: preparation

Walk this thing with me. We will alternate between the debugger and the worker.

1. worker session: get connection ID.

mysql [worker]> select CONNECTION_ID();
+-----------------+
| CONNECTION_ID() |
+-----------------+
|            1234 |
+-----------------+

I'll use 1234, you will use whatever connection ID your worker has.

2. debugger session: "compile" routine with debug info (this injects code into your routines).

mysql [debugger]> use common_schema;
mysql [debugger]> call rdebug_compile_routine('test', 'review_colors', true);
mysql [debugger]> call rdebug_compile_routine('test', 'review_single_color', true);

If you like, review the routines after compilation as follows:

mysql [debugger]> call rdebug_show_routine('test', 'review_colors');
+---------------------------------------------------------------------------------+
| `test`.`review_colors` breakpoints                                              |
+---------------------------------------------------------------------------------+
| begin                                                                           |
|   declare done bool default false;                                              |
|   declare current_color varchar(32) default null;                               |
|   declare current_count int unsigned;                                           |
|   declare color_cursor cursor for                                               |
|             select color_name, count from test.color_count order by color_name; |
|   declare continue handler for not found set done := true;                      |
|                                                                                 |
|   [:94]open color_cursor;                                                       |
|   [:100]cursor_loop: while not done do                                          |
|     [:112]fetch color_cursor into current_color, current_count;                 |
|     [:125]if done then                                                          |
|       [:132]leave cursor_loop;                                                  |
|     [:138]end if;                                                               |
|                                                                                 |
|     [:145]call review_single_color(current_color);                              |
|   [:154]end while;                                                              |
|   [:160]close color_cursor;                                                     |
| [:165]end                                                                       |
+---------------------------------------------------------------------------------+

mysql [debugger]> call rdebug_show_routine('test', 'review_single_color');
+----------------------------------------------------------------+
| `test`.`review_single_color` breakpoints                       |
+----------------------------------------------------------------+
| begin                                                          |
|   [:4]set @review_message := concat(color_name, ' is pretty'); |
|   [:20]select @review_message;                                 |
| [:25]end                                                       |
+----------------------------------------------------------------+

The above shows the routine code with symbolic breakpoint IDs.

Walk-through - start debugging

3. debugger session: Start a debug session, attach to worker session using its connection ID:

mysql [debugger]> call rdebug_start(1234);

Replace 1234 with your own worker's connection ID as read above.

Let's set verbose mode on; more fun on command line, less typing.

mysql [debugger]> call rdebug_set_verbose(true);

And step into it!

mysql [debugger]> call rdebug_step_into();

This should hang the debugger. Why? Because it's stepping into, and is expecting the worker to actually do something.

4. worker session: execute routine

mysql [worker]> call test.review_colors();

Walk-through - debug

The debugger session should immediately follow with the following (all by entry_time should be identical to your output):

+-------------+----------------+---------------+--------------+---------------------+
| stack_level | routine_schema | routine_name  | statement_id | entry_time          |
+-------------+----------------+---------------+--------------+---------------------+
|           1 | test           | review_colors |           94 | 2013-04-08 15:41:28 |
+-------------+----------------+---------------+--------------+---------------------+

+----------------+---------------+---------------+---------------+----------------+
| routine_schema | routine_name  | variable_name | variable_type | variable_value |
+----------------+---------------+---------------+---------------+----------------+
| test           | review_colors | current_color | local         | NULL           |
| test           | review_colors | current_count | local         | NULL           |
| test           | review_colors | done          | local         | 0              |
+----------------+---------------+---------------+---------------+----------------+

+----------------+---------------+--------------+-------------------+
| routine_schema | routine_name  | statement_id | statement         |
+----------------+---------------+--------------+-------------------+
| test           | review_colors |           94 | open color_cursor |
+----------------+---------------+--------------+-------------------+

That's the result of setting verbose mode. From here, if you've ever debugged code, the way is clear:

5. debugger session: Step into a few more times:

mysql [debugger]> call rdebug_step_into();
mysql [debugger]> call rdebug_step_into();
mysql [debugger]> call rdebug_step_into();
...

Until the stack shows that you have entered the second routine: review_single_color():

+-------------+----------------+---------------------+--------------+---------------------+
| stack_level | routine_schema | routine_name        | statement_id | entry_time          |
+-------------+----------------+---------------------+--------------+---------------------+
|           1 | test           | review_colors       |          145 | 2013-04-08 15:41:28 |
|           2 | test           | review_single_color |           20 | 2013-04-08 15:45:23 |
+-------------+----------------+---------------------+--------------+---------------------+

+----------------+---------------------+-----------------+---------------+-----------------+
| routine_schema | routine_name        | variable_name   | variable_type | variable_value  |
+----------------+---------------------+-----------------+---------------+-----------------+
| test           | review_single_color | @review_message | user_defined  | green is pretty |
| test           | review_single_color | color_name      | param         | green           |
+----------------+---------------------+-----------------+---------------+-----------------+

+----------------+---------------------+--------------+------------------------+
| routine_schema | routine_name        | statement_id | statement              |
+----------------+---------------------+--------------+------------------------+
| test           | review_single_color |           20 | select @review_message |
+----------------+---------------------+--------------+------------------------+

You can further call rdebug_step_out() to leave this routine, rdebug_step_over() to avoid re-entry...

6. debugger session: modify variables

Assuming you are inside the review_single_color() routine, would you like to modify a variable?

mysql [debugger]> call rdebug_set_variable('color_name', 'A flower');

Step over a few more times till the worker produces:

+--------------------+
| @review_message    |
+--------------------+
| A flower is pretty |
+--------------------+

Continue playing with rdebug_step_into(), rdebug_step_over(), rdebug_step_out().

7. debugger session: setting a breakpoint

Based on the output of rdebug_show_routine('test', 'review_colors'), above, we now choose to set a non-conditional breakpoint, just before the statement call review_single_color(current_color). That makes breakpoint ID 145.

mysql [debugger]> call rdebug_set_breakpoint('test', 'review_colors', 145, NULL, true);

8. debugger session: running up to a breakpoint

Now, let's allow the worker to run until it reaches this breakpoint:

mysql [debugger]> call rdebug_run();
+-------------+----------------+---------------+--------------+---------------------+
| stack_level | routine_schema | routine_name  | statement_id | entry_time          |
+-------------+----------------+---------------+--------------+---------------------+
|           1 | test           | review_colors |          145 | 2013-04-08 15:41:28 |
+-------------+----------------+---------------+--------------+---------------------+

+----------------+---------------+---------------+---------------+----------------+
| routine_schema | routine_name  | variable_name | variable_type | variable_value |
+----------------+---------------+---------------+---------------+----------------+
| test           | review_colors | current_color | local         | white          |
| test           | review_colors | current_count | local         | 10             |
| test           | review_colors | done          | local         | 0              |
+----------------+---------------+---------------+---------------+----------------+

+----------------+---------------+--------------+-----------------------------------------+
| routine_schema | routine_name  | statement_id | statement                               |
+----------------+---------------+--------------+-----------------------------------------+
| test           | review_colors |          145 | call review_single_color(current_color) |
+----------------+---------------+--------------+-----------------------------------------+

Run the above a few times: we always get back to the same statement. That is, until there's nothing more to do and the routine leaves.

Walk-through - stopping and cleanup

8. debugger session: Stop the debugging session:

mysql [debugger]> call rdebug_stop();

You can start again via rdebug_start(). If, however, you're no longer interested in debugging, you should remove debugging code from your routines:

mysql [debugger]> call rdebug_compile_routine('test', 'review_colors', false);
mysql [debugger]> call rdebug_compile_routine('test', 'review_single_color', false);

Conclusion

This is most there is to it. Read the API for a complete list of functionality

With a different license in mind for rdebug, common_schema changes license to GPL (2 or above).

common_schema 2.0 is ready for download. All things rdebug, it is alpha -- otherwise it's a stable release.

rdebug

I'm very happy to release this alpha version of rdebug, and urge everyone to try it out.

The idea is to have an open, free, server side debugger and debugging API for MySQL stored routines. To elaborate:

• It's server side by that it's implemented by stored routines. Not by a connector; not an emulator; not a GUI tool hack. The entire functionality lies within common_schema, a schema in your server.
• It's a debugger: you can debug your own stored routines (with limitations)
• It's a debugging API: there's a distinct specification and a set of calls which makes for a debugging process
• It's open since the source code is yours to browse.
• It's free as in free beer.
• It's free as it makes you independent of a specific debugger. It provides an API that anyone can use. You can run the API yourself from the command line; or plugins for your favorite GUI editor can be developed to use this API.

On a separate blog post I will take you to a rdebug test drive.

As always, nothing is released before extensive documentation is in place.

I'd love to get input on this.

GPL

common_schema < 2.0 was released under the BSD license, which is less restrictive. I was pondering the BSD license for a couple years now, and with the arrival of rdebug have decided to switch to GPL. I've been through this thinking of change of license in other projects of mine; am generally agreeing that best not to change licensing throughout lifetime. I actually do see GPL as promoting open source software better than BSD, and with all the issues around GPL this actually means something to me. I write open source; I love people using it; I love people extending it; I want to be re-released as open source, or I want better control of the code.

So this turns out to be something that is important to me, and just before common_schema takes the world in storm (212 downloads today, 212,000,000 tomorrow), I want to have this settled. If no storm comes, well, I'll have up to 212 people banging on my door (I provide free coffee).

Text routines

Two text routines are added:

• replace_sections(): replace a text given a from-to combination, and a replacement string, which could include a back-reference. For example:

mysql> select replace_sections('The <b>quick</b> brown <b>fox</b>', 
        '<b>', '</b>', 
        '<span>\\0</span>') as result;
+-----------------------------------------------+
| result                                        |
+-----------------------------------------------+
| The <span>quick</span> brown <span>fox</span> |
+-----------------------------------------------+

• hexcode_text(): a convenience routine which shows a beautified hex-code of a given text. I get to need it when using UNICODE characters which are hard to detect, are visually identical to other characters, or are copied from MS Word.

mysql> call hexcode_text('the quick brown fox jumps over the lazy dog');
+-----------------------------------------------------------------------------------+
| >0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f   0 1 2 3 4 5 6 7 8 9 a b c d e f |
+-----------------------------------------------------------------------------------+
| 74 68 65 20 71 75 69 63 6B 20 62 72 6F 77 6E 20   t h e   q u i c k   b r o w n   |
| 66 6F 78 20 6A 75 6D 70 73 20 6F 76 65 72 20 74   f o x   j u m p s   o v e r   t |
| 68 65 20 6C 61 7A 79 20 64 6F 67                  h e   l a z y   d o g           |
+-----------------------------------------------------------------------------------+

Get it

Download common_schema here.