Normally, I would simply:

SELECT @@group_concat_max_len

However, I am using views, where session variables are not allowed. Using a stored function can do the trick, but I wanted to avoid stored routines. So here’s a very simple test case: is the current group_concat_max_len long enough or not? I’ll present the long version and the short version.

The long version

SELECT
  CHAR_LENGTH(
    GROUP_CONCAT(
      COLLATION_NAME SEPARATOR ''
    )
  )
FROM
  INFORMATION_SCHEMA.COLLATIONS;

If the result is 1024, we are in a bad shape. I happen to know that the total length of collation names is above 1800, and so it is trimmed down. Another variance of the above query would be:

SELECT
  CHAR_LENGTH(
    GROUP_CONCAT(
      COLLATION_NAME SEPARATOR ''
    )
  ) = SUM(CHAR_LENGTH(COLLATION_NAME))
    AS group_concat_max_len_is_long_enough
FROM
  INFORMATION_SCHEMA.COLLATIONS;

+-------------------------------------+
| group_concat_max_len_is_long_enough |
+-------------------------------------+
|                                   0 |
+-------------------------------------+

The COLLATIONS, CHARACTER_SETS or COLLATION_CHARACTER_SET_APPLICABILITY tables provide with known to exist variables (assuming you did not compile MySQL with particular charsets). It’s possible to CONCAT, UNION or JOIN columns and tables to detect longer than 1800 characters in group_concat_max_len. I admit this is becoming ugly, so let’s move on.

The short version

Don’t want to rely on existing tables? Not sure what values to expect? Look at this:

SELECT CHAR_LENGTH(GROUP_CONCAT(REPEAT('0', 1025))) FROM DUAL

GROUP_CONCAT doesn’t really care about the number of rows. In the above example, I’m using a single row (retrieved from the DUAL virtual table), making sure it is long enough. Type in any number in place of 1025, and you have a metric for your group_concat_max_len.

SELECT
  CHAR_LENGTH(GROUP_CONCAT(REPEAT('0', 32768))) >= 32768 As group_concat_max_len_is_long_enough
FROM
  DUAL;
+-------------------------------------+
| group_concat_max_len_is_long_enough |
+-------------------------------------+
|                                   0 |
+-------------------------------------+

The above makes a computation with REPEAT. One can replace this with a big constant.

Recap: this information can be useful if you’re looking for a good innodb_log_file_size value, such that will not pose too much I/O (smaller values will make for more frequent flushes), not will make for a too long recovery time (larger values mean more transactions to recover upon crash).

It is assumed that the 60 seconds period represents an average system load, not some activity spike period. Edit the sleep time and factors as you will to sample longer or shorter periods.

SELECT
  innodb_os_log_written_per_minute*60
    AS estimated_innodb_os_log_written_per_hour,
  CONCAT(ROUND(innodb_os_log_written_per_minute*60/1024/1024, 1), 'MB')
    AS estimated_innodb_os_log_written_per_hour_mb
FROM
  (SELECT SUM(value) AS innodb_os_log_written_per_minute FROM (
    SELECT -VARIABLE_VALUE AS value
      FROM INFORMATION_SCHEMA.GLOBAL_STATUS
      WHERE VARIABLE_NAME = 'innodb_os_log_written'
    UNION ALL
    SELECT SLEEP(60)
      FROM DUAL
    UNION ALL
    SELECT VARIABLE_VALUE
      FROM INFORMATION_SCHEMA.GLOBAL_STATUS
      WHERE VARIABLE_NAME = 'innodb_os_log_written'
  ) s1
) s2
;

Sample output:

+------------------------------------------+---------------------------------------------+
| estimated_innodb_os_log_written_per_hour | estimated_innodb_os_log_written_per_hour_mb |
+------------------------------------------+---------------------------------------------+
|                                584171520 | 557.1MB                                     |
+------------------------------------------+---------------------------------------------+

I wish to present a SQL trick which does the same, without need for resetting tables. Suppose you have some status table, and you wish to measure the change in status per second, per minute etc. The trick is to query for the value twice in the same query, with some pause in between, and make the difference calculation.

For sake of simplicity, I’ll demonstrate using 5.1′s INFORMATION_SCHEMA.GLOBAL_STATUS. Please refer to INFORMATION_SCHEMA.GLOBAL_STATUS: watch out for some discussion on this.

In our example, we wish to measure the number of questions per second. Getting the number of questions is done with:

SELECT * FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'questions';
+---------------+----------------+
| VARIABLE_NAME | VARIABLE_VALUE |
+---------------+----------------+
| QUESTIONS     | 3619           |
+---------------+----------------+
1 row in set (0.00 sec)

Applying the trick, thus solving the problem:

SELECT SUM(value) AS questions_per_sec FROM (
  SELECT -VARIABLE_VALUE AS value
    FROM INFORMATION_SCHEMA.GLOBAL_STATUS
    WHERE VARIABLE_NAME = 'questions'
  UNION ALL
  SELECT SLEEP(1)
    FROM DUAL
  UNION ALL
  SELECT VARIABLE_VALUE
    FROM INFORMATION_SCHEMA.GLOBAL_STATUS
    WHERE VARIABLE_NAME = 'questions'
) s1;
+-------------------+
| questions_per_sec |
+-------------------+
|               126 |
+-------------------+
1 row in set (1.01 sec)

Make a one minute measurement with SLEEP(60), then divide SUM by 60.

Note on transactional tables

The above trick will not work when reading values from transactional tables, and with isolation level >= REPEATABLE-READ, since, by definition, you must get the same value back while in the same transaction. So this works on MyISAM, MEMORY, functions and otherwise non transactional data sources.

At last, it is possible to ask questions like:

node1> SELECT * FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'innodb_os_log_written';
+-----------------------+----------------+
| VARIABLE_NAME         | VARIABLE_VALUE |
+-----------------------+----------------+
| INNODB_OS_LOG_WRITTEN | 512            |
+-----------------------+----------------+
1 row in set (0.00 sec)

node1> SELECT * FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'questions';
+---------------+----------------+
| VARIABLE_NAME | VARIABLE_VALUE |
+---------------+----------------+
| QUESTIONS     | 28             |
+---------------+----------------+
1 row in set (0.00 sec)

Watch out #1

As with all INFORMATION_SCHEMA tables, to get a single row one needs to materialize the entire table. To ask the above two questions, the table will materialize twice. This means gathering all the information — twice. To get 20 values, we materialize the table 20 times. It not only takes time, but also increases some of the status variables themselves, like questions, select_scan, created_tmp_tables. Ironically, when we used SHOW GLOBAL STATUS and had to parse the results in our application code, we only issued the query once. But with the convenience of INFORMATION_SCHEMA, it’s much easier (and makes more sense!) to query per variable.

Watch out #2

So if we’re to access a handful of status variables, and wish to only materialize the table once, what can we do? An easy solution is to create a MEMORY table which looks just like GLOBAL_STATUS, like this:

node1> CREATE TABLE memory_global_status LIKE INFORMATION_SCHEMA.GLOBAL_STATUS;
Query OK, 0 rows affected (0.00 sec)
node1> INSERT INTO memory_global_status SELECT * FROM INFORMATION_SCHEMA.GLOBAL_STATUS;
Query OK, 291 rows affected (0.01 sec)
Records: 291  Duplicates: 0  Warnings: 0

We can now query the memory_global_status table, having ‘friezed’ the status, for as many times as we wish, with no real cost.

But let’s take a look at:

node1> SHOW TABLE STATUS LIKE 'memory_global_status'\G
*************************** 1. row ***************************
           Name: memory_global_status
         Engine: MEMORY
        Version: 10
     Row_format: Fixed
           Rows: 291
 Avg_row_length: 3268
    Data_length: 1050624
Max_data_length: 16755036
   Index_length: 0
      Data_free: 0
 Auto_increment: NULL
    Create_time: NULL
    Update_time: NULL
     Check_time: NULL
      Collation: utf8_general_ci
       Checksum: NULL
 Create_options:
        Comment:
1 row in set (0.00 sec)

Ouch! How did we get Avg_row_length: 3268, and Data_length: 1050624? That’s quite more then we expected. Well, most of the values in GLOBAL_STATUS are just intgers. But some, just a few, are textual, and so the table definition is:

node1> SHOW CREATE TABLE INFORMATION_SCHEMA.GLOBAL_STATUS \G
*************************** 1. row ***************************
       Table: GLOBAL_STATUS
Create Table: CREATE TEMPORARY TABLE `GLOBAL_STATUS` (
  `VARIABLE_NAME` varchar(64) NOT NULL DEFAULT '',
  `VARIABLE_VALUE` varchar(1024) DEFAULT NULL
) ENGINE=MEMORY DEFAULT CHARSET=utf8
1 row in set (0.00 sec)

A MEMORY tables works with FIXED row format, which means we need to allocate 64 utf8 characters for VARIABLE_NAME, plus 1024 utf8 characters for VARIABLE_VALUE. This makes for: (1+64*3) + 2+(1024*3) = 3267 (the missing byte is to indicate the NULLable values).

I’m not sure why the table definition is as such. VARIABLE_NAME can be safely declared as ascii, and, as far as I can see, so can VARIABLE_VALUE. There are a few ON/OFF values (I’ve expressed my opinion and concerns on these here and here; why not just use 0/1?). SSL_CIPHER seems like the only variable which can get long enough to justify the 1024 characters.

If you don’t mind about truncating those text values, or don’t mind about text values at all (we usually care about the counters), you can altogether disregard them when SELECTing from GLOBAL_STATUS. One can also add a HASH index on the VARIABLE_NAME parameter to avoid using full table scans upon reading each value.

I will present queries for:

• Checking on database engines and size
• Locating duplicate and redundant indexes
• Checking on character sets for columns and tables, looking for variances
• Checking on processes and long queries (only with MySQL 5.1)

Dimensions

The following query returns the total size per engine per database. For example, it is common that in a given database, all tables are InnoDB. But once in a while, and even though default-engine is set to InnoDB, someone creates a MyISAM table. This may break transactional behavior, or may cause a mysqldump --single-transaction to be ineffective.

See aggregated size per schema per engine:

SELECT TABLE_SCHEMA, ENGINE, COUNT(*) AS count_tables,
  SUM(DATA_LENGTH+INDEX_LENGTH) AS size,
SUM(INDEX_LENGTH) AS index_size FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_SCHEMA NOT IN ('mysql', 'INFORMATION_SCHEMA')
  AND ENGINE IS NOT NULL GROUP BY TABLE_SCHEMA, ENGINE

Result example:

+--------------+--------+--------------+----------+------------+
| TABLE_SCHEMA | ENGINE | count_tables | size     | index_size |
+--------------+--------+--------------+----------+------------+
| test         | InnoDB |            3 | 12140544 |          0 |
| world        | InnoDB |            1 |  4734976 |          0 |
| world        | MyISAM |            5 | 10665303 |    4457472 |
+--------------+--------+--------------+----------+------------+

I may not have intended to, but it seems I have both MyISAM and InnoDB tables in the world database.

The index_size may be important with MyISAM when estimating the desired key_buffer_size.

See per table size (almost exactly as presented in INFORMATION_SCHEMA):

SELECT TABLE_SCHEMA, TABLE_NAME, ENGINE,
 SUM(DATA_LENGTH+INDEX_LENGTH) AS size,
SUM(INDEX_LENGTH) AS index_size FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_SCHEMA NOT IN ('mysql', 'INFORMATION_SCHEMA')
 AND ENGINE IS NOT NULL GROUP BY TABLE_SCHEMA, TABLE_NAME

Indexes

We will now turn to check for duplicate or redundant indexes. We begin by presenting the following table:

mysql> show create table City \G
*************************** 1. row ***************************
       Table: City
Create Table: CREATE TABLE `City` (
  `ID` int(11) NOT NULL auto_increment,
  `Name` char(35) character set utf8 NOT NULL default '',
  `CountryCode` char(3) NOT NULL default '',
  `District` char(20) NOT NULL default '',
  `Population` int(11) NOT NULL default '0',
  PRIMARY KEY  (`ID`),
  UNIQUE KEY `ID` (`ID`),
  KEY `Population` (`Population`),
  KEY `Population_2` (`Population`,`CountryCode`)
) ENGINE=MyISAM AUTO_INCREMENT=4080 DEFAULT CHARSET=latin1

We can see that the Population_2 index covers the Population index, so the latter is redundant and should be removed. We also see that the ID index is redundant, since there is a PRIMARY KEY on ID, which is in itself a unique key. How can we test such cases by querying the INFORMATION_SCHEMA? Turns out we can do that using the STATISTICS table.

[Update: thanks to Roland Bouman's comments. The following queries only consider BTREE indexes, and do not verify FULLTEXT or HASH indexes]

See if some index is a prefix of another (in which case it is redundant):

SELECT * FROM (
  SELECT TABLE_SCHEMA, TABLE_NAME, INDEX_NAME,
    GROUP_CONCAT(COLUMN_NAME ORDER BY SEQ_IN_INDEX) AS columns
  FROM `information_schema`.`STATISTICS`
  WHERE TABLE_SCHEMA NOT IN ('mysql', 'INFORMATION_SCHEMA')
    AND NON_UNIQUE = 1 AND INDEX_TYPE='BTREE' 
  GROUP BY TABLE_SCHEMA, TABLE_NAME, INDEX_NAME
) AS i1 INNER JOIN (
  SELECT TABLE_SCHEMA, TABLE_NAME, INDEX_NAME,
    GROUP_CONCAT(COLUMN_NAME ORDER BY SEQ_IN_INDEX) AS columns
  FROM `information_schema`.`STATISTICS`
  WHERE INDEX_TYPE='BTREE' 
  GROUP BY TABLE_SCHEMA, TABLE_NAME, INDEX_NAME
) AS i2
USING (TABLE_SCHEMA, TABLE_NAME)
WHERE i1.columns != i2.columns AND LOCATE(CONCAT(i1.columns, ','), i2.columns) = 1

The above query lists pairs of indexes in which one of them is a true prefix of the other. I’m using GROUP_CONCAT(COLUMN_NAME ORDER BY SEQ_IN_INDEX) to aggregate columns per index, by order of appearance in that index.
The query only considers cases when the prefix (the “shorter”) index is non-unique. Else wise there is no redundancy, as the uniqueness of the index imposes a constraint which is not achieved by the “longer” index.

Result example:

+--------------+------------+------------+------------+--------------+------------------------+
| TABLE_SCHEMA | TABLE_NAME | INDEX_NAME | columns    | INDEX_NAME   | columns                |
+--------------+------------+------------+------------+--------------+------------------------+
| world        | City       | Population | Population | Population_2 | Population,CountryCode |
+--------------+------------+------------+------------+--------------+------------------------+

See if any two indexes are identical:

SELECT * FROM (
  SELECT TABLE_SCHEMA, TABLE_NAME, INDEX_NAME,
    GROUP_CONCAT(COLUMN_NAME ORDER BY SEQ_IN_INDEX) AS columns, NON_UNIQUE
  FROM `information_schema`.`STATISTICS`
  WHERE TABLE_SCHEMA NOT IN ('mysql', 'INFORMATION_SCHEMA')
  AND INDEX_TYPE='BTREE'
  GROUP BY TABLE_SCHEMA, TABLE_NAME, INDEX_NAME
) AS i1 INNER JOIN (
  SELECT TABLE_SCHEMA, TABLE_NAME, INDEX_NAME,
    GROUP_CONCAT(COLUMN_NAME ORDER BY SEQ_IN_INDEX) AS columns, NON_UNIQUE
  FROM `information_schema`.`STATISTICS`
  WHERE INDEX_TYPE='BTREE'
  GROUP BY TABLE_SCHEMA, TABLE_NAME, INDEX_NAME
) AS i2
USING (TABLE_SCHEMA, TABLE_NAME)
WHERE i1.columns = i2.columns AND i1.NON_UNIQUE = i2.NON_UNIQUE
  AND i1.INDEX_NAME < i2.INDEX_NAME

The above checks for unique or non-unique indexes alike. It checks for indexes with identical columns list (and in the same order, of course). Any two indexes having the same list of columns imply a redundancy. If both are unique or both are non-unique, either can be removed. If one is unique and the other is not, the non-unique index should be removed.

Result example:

+--------------+------------+------------+---------+------------+------------+---------+------------+
| TABLE_SCHEMA | TABLE_NAME | INDEX_NAME | columns | NON_UNIQUE | INDEX_NAME | columns | NON_UNIQUE |
+--------------+------------+------------+---------+------------+------------+---------+------------+
| world        | City       | PRIMARY    | ID      |          0 | ID         | ID      |          0 |
+--------------+------------+------------+---------+------------+------------+---------+------------+

You may also wish to take a look at the excellent mk-duplicate-key-checker, a maatkit utility by Baron Schwartz.

Character sets

Show the character sets for all tables:

SELECT TABLE_SCHEMA, TABLE_NAME, CHARACTER_SET_NAME, TABLE_COLLATION
FROM INFORMATION_SCHEMA.TABLES
INNER JOIN INFORMATION_SCHEMA.COLLATION_CHARACTER_SET_APPLICABILITY
  ON (TABLES.TABLE_COLLATION = COLLATION_CHARACTER_SET_APPLICABILITY.COLLATION_NAME)
WHERE TABLE_SCHEMA NOT IN ('mysql', 'INFORMATION_SCHEMA')

Surprisingly, the TABLES table does not include the character set for the table, only the collation, so we must join with COLLATION_CHARACTER_SET_APPLICABILITY to get the character set for that collation. Yes, it’s more normalized this way, but INFORMATION_SCHEMA is not too normalized anyway.

See all the textual columns, along with their character sets:

SELECT TABLE_SCHEMA, TABLE_NAME, COLUMN_NAME, CHARACTER_SET_NAME, COLLATION_NAME
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA NOT IN ('mysql', 'INFORMATION_SCHEMA')
  AND CHARACTER_SET_NAME IS NOT NULL
ORDER BY TABLE_SCHEMA, TABLE_NAME, COLUMN_NAME

See those columns for which the character set or collation is different from the table’s character set and collation:

SELECT columns.TABLE_SCHEMA, columns.TABLE_NAME, COLUMN_NAME,
  CHARACTER_SET_NAME AS column_CHARSET,
  COLLATION_NAME AS column_COLLATION,
  table_CHARSET, TABLE_COLLATION
FROM (
  SELECT TABLE_SCHEMA, TABLE_NAME, COLUMN_NAME, CHARACTER_SET_NAME, COLLATION_NAME
  FROM INFORMATION_SCHEMA.COLUMNS
  WHERE TABLE_SCHEMA NOT IN ('mysql', 'INFORMATION_SCHEMA')
    AND CHARACTER_SET_NAME IS NOT NULL
) AS columns INNER JOIN (
  SELECT TABLE_SCHEMA, TABLE_NAME, CHARACTER_SET_NAME AS table_CHARSET, TABLE_COLLATION
  FROM INFORMATION_SCHEMA.TABLES
  INNER JOIN INFORMATION_SCHEMA.COLLATION_CHARACTER_SET_APPLICABILITY
    ON (TABLES.TABLE_COLLATION = COLLATION_CHARACTER_SET_APPLICABILITY.COLLATION_NAME)
) AS tables
ON (columns.TABLE_SCHEMA = tables.TABLE_SCHEMA AND columns.TABLE_NAME = tables.TABLE_NAME)
WHERE (columns.CHARACTER_SET_NAME != table_CHARSET OR columns.COLLATION_NAME != TABLE_COLLATION)
ORDER BY TABLE_SCHEMA, TABLE_NAME, COLUMN_NAME

Result example:

+--------------+------------+-------------+----------------+------------------+---------------+-------------------+
| TABLE_SCHEMA | TABLE_NAME | COLUMN_NAME | column_CHARSET | column_COLLATION | table_CHARSET | TABLE_COLLATION   |
+--------------+------------+-------------+----------------+------------------+---------------+-------------------+
| world        | City       | Name        | utf8           | utf8_general_ci  | latin1        | latin1_swedish_ci |
+--------------+------------+-------------+----------------+------------------+---------------+-------------------

Processes (MySQL 5.1)

With MySQL 5.1 comes a boost to INFORMATION_SCHEMA. Among the new tables we can find the PROCESSLIST table, as well as GLOBAL_VARIABLES and GLOBAL_STATUS. Together with the new Event Scheduler, it seems the sky is the limit.

See which processes are active:

SELECT * FROM information_schema.PROCESSLIST WHERE COMMAND != 'Sleep'

Show slow queries:

SELECT * FROM information_schema.PROCESSLIST WHERE COMMAND != 'Sleep' AND TIME > 4

How many processes per user?

SELECT USER, COUNT(*) FROM information_schema.PROCESSLIST GROUP BY USER

How many processes per host?

SELECT SUBSTR(HOST, 1, LOCATE(':',HOST)-1) AS hostname, COUNT(*)
FROM information_schema.PROCESSLIST GROUP BY hostname

Along with the Event Scheduler, a stored procedure may decide to KILL processes executing for more than 10 minutes, KILL users who have too many connections, perform some logging on connections and more.

Conclusion

I have presented what I think is a set of useful queries. When I approach a new database I use these to get an overall understanding of what’s in it. Finding duplicate indexes can explain a lot about how the designers or developers think the database should behave. Looking at the non-default character sets shows if textual columns have been carefully designed or not. For example, querying for non-default columns characters and getting no results may imply that many textual columns have improper character sets.