Comments on: Verifying GROUP_CONCAT limit without using variables

By: SQL trick: overcoming GROUP_CONCAT limitation in special cases | code.openark.org

Wed, 21 Jul 2010 13:14:35 +0000

[...] Verifying GROUP_CONCAT limit without using variables [...]

By: shlomi

shlomi — Fri, 11 Jun 2010 04:58:46 +0000

@Justin,

Yes, I was referring to this type of solution on "...Using a stored function can do the trick...".

It *is* somewhat evil

By: Justin Swanhart

Justin Swanhart — Fri, 11 Jun 2010 02:26:03 +0000

Oh, and you don't have to call it in group_concat, it can just be a select from dual.. I was just trying to trick MySQL into somehow evaluating the session var change in the same SQL statement, but that doesn't work.

By: Justin Swanhart

Justin Swanhart — Fri, 11 Jun 2010 02:24:15 +0000

It is kind of (well especially, perhaps) evil, but you could create a stored function that sets the session variables. Then instruct users to select from a view that invokes that SP before they select from any other views.

DELIMITER ;;
DROP FUNCTION IF EXISTS test.set_gc_len;;

CREATE FUNCTION test.set_gc_len (
)
RETURNS TINYTEXT
NO SQL
BEGIN
SET SESSION group_concat_max_len=65535;
RETURN '';
END
;;

DELIMITER ;

CREATE VIEW `test`.`z1` AS select group_concat(`test`.`set_gc_len`() separator ',') AS `group_concat(test.set_gc_len())`,(char_length(group_concat(repeat('0',32768) separator ',')) >= 32768) AS `group_concat_max_len_is_long_enough`;

-- this first call to the view increases the max GC len
mysql> select * from test.z1\G
*************************** 1. row ***************************
group_concat(test.set_gc_len()):
group_concat_max_len_is_long_enough: 0
1 row in set, 1 warning (0.00 sec)

mysql> select * from test.z1\G
*************************** 1. row ***************************
group_concat(test.set_gc_len()):
group_concat_max_len_is_long_enough: 1
1 row in set (0.01 sec)

By: shlomi

shlomi — Thu, 10 Jun 2010 17:20:00 +0000

@Sheeri,
I figured you might have figured it , but there's more to it.
The mycheckpoint monitoring tool works in a special way. The whole point of the tool is to create a sophisticated (if I may say so) views hierarchy; such that for getting monitored data (reports, charts, full blown HTML pages) -- you don't actually use the tool; you just query for the data, e.g.
SELECT html FROM sv_report_html_brief;
or
SELECT report FROM sv_report_human_sample;

So - no tool here. Nowhere to set the session variable. I hastily dropped in the openark-kit. You are correct: I do not have this issue there.

By: Sheeri Cabral

Sheeri Cabral — Thu, 10 Jun 2010 17:11:30 +0000

Shlomi -- I figured it was in those tools. But if it's just the tool checking a few things, then setting it *within* the tool will ensure that the tool always has it set appropriately. I'm talking about setting the *session* variable, not the *global* variable.

By: shlomi

shlomi — Thu, 10 Jun 2010 17:07:27 +0000

Hi Sheeri,
Fair question. Actually, it all comes up by issues I'm having with developing the openark-kit & mycheckpoint, two open source projects I'm working on.
The problem is, I have no idea who will install these, what their MySQL expertise will be; will they have privileges to change these settings, and so on.
I want the tools to handle these cases where the settings are insufficient.

Who knows how the next installation will go? Since the defaults are so low, how can I be sure that the next machine anyone sets up will have a manual setting for group_concat_max_len?

As long as the machines are mine, or under my control, yes - I just simple set up the variables as I please.

By: Sheeri Cabral

Sheeri Cabral — Thu, 10 Jun 2010 15:56:00 +0000

Why not just set the group_concat_max_len variable for the session so that you won't have an issue with the operation no matter what?

By: shlomi

shlomi — Thu, 10 Jun 2010 08:05:23 +0000

Hi Roland,
Right, didn't even mention it...
Only available as of 5.1, while I have a requirement for 5.0 - 5.1.

Thank you for noting this down.

By: Roland Bouman

Roland Bouman — Thu, 10 Jun 2010 08:00:49 +0000

Hi Shlomi,

why not simply use one of the %_VARIABLES views in the information_schema?

mysql> select * from information_schema.global_variables where variable_name = 'group_concat_max_len';
+----------------------+----------------+
| VARIABLE_NAME | VARIABLE_VALUE |
+----------------------+----------------+
| GROUP_CONCAT_MAX_LEN | 1024 |
+----------------------+----------------+
1 row in set (0.00 sec)