Comments on: oak-hook-general-log: streaming general log

By: mensajes claro

mensajes claro — Sat, 02 Oct 2010 04:11:29 +0000

Gadb Your comment was so usefull for me “From there you can create alerts or do take any number of actions on its contents.”

By: shlomi

shlomi — Tue, 23 Mar 2010 05:21:48 +0000

HA! 😀

By: Henrik Ingo

Henrik Ingo — Mon, 22 Mar 2010 21:16:52 +0000

Heh, it seems this wishlist item was implemented in record speed 🙂

http://karlssonondatabases.blogspot.com/2010/03/mysql-audit-plugin-api.html

By: shlomi

shlomi — Mon, 22 Mar 2010 08:04:11 +0000

@gba
Thanks. For this case my tool would not be needed. Just use the normal general log file.

@Henrik,
I agree. What I would like to see is a hooking mechanism for all logs. So that one is able to hook up a streaming listener on the slow log, or a socket listener, or a file listener, or… you get my point.
The current status is too hard coded. Either there’s a log file or a log table or both. But if there’s log table then it applies to slow log as well. And you can’t stream it…

By: Henrik Ingo

Henrik Ingo — Mon, 22 Mar 2010 07:58:21 +0000

I’ve often thought that the next “destination” for general log should be just a socket. A tool like this could then connect to it and receive the data. No need to write to table or file or anything on disk. If nobody is listening, then the log disappears into cyberspace.

Should be a fun project for someone…

By: gba

gba — Mon, 22 Mar 2010 02:27:36 +0000

You could pipe the data into a file and consume it with Splunk this will index the log data and make it searchable. From there you can create alerts or do take any number of actions on its contents.

e.g.:

$ python src/oak/oak-hook-general-log.py –socket=/tmp/mysql.sock –user=root > /var/log/general.log
$ splunk add monitor /var/log/general.log

By: shlomi

shlomi — Sun, 21 Mar 2010 17:14:40 +0000

BTW,

Is the 65% slowdown measured for CSV tables or MyISAM?

By: shlomi

shlomi — Sun, 21 Mar 2010 13:21:33 +0000

Hi Giuseppe,

Indeed. Nevertheless, on a recent job I did I had the need to monitor the general log. And there’s a few other cases I have in mind.

This is the reason why the tool forces you to timeout, and does not leave the general log open.

regards

By: Giuseppe Maxia

Giuseppe Maxia — Sun, 21 Mar 2010 13:00:33 +0000

Careful with table logs. They take a huge toll on performance. While general log on a file accounts for 12% to 20% speed loss, table logs can slow your server down 65%, which is quite a blow!

Giuseppe