code.openark.org » Graphs http://code.openark.org/blog Blog by Shlomi Noach Fri, 12 Mar 2010 13:27:07 +0000 http://wordpress.org/?v=2.9.2 en hourly 1 Static charts vs. interactive charts http://code.openark.org/blog/mysql/static-charts-vs-interactive-charts http://code.openark.org/blog/mysql/static-charts-vs-interactive-charts#comments Tue, 02 Mar 2010 13:28:08 +0000 shlomi http://code.openark.org/blog/?p=2027 I’m having my usual fun with charts. Working on mycheckpoint, I’ve generated monitoring charts using the Google Chars API. But I’ve also had chance to experiment and deploy interactive charts, JavaScript based. In particular, I used and tweaked dygraphs.

I’d like to note some differences in using charts of both kinds. And I think it makes a very big difference.

Static charts

I’ll call any image-based chart by “static chart”. It’s just a static image. Example of such charts are those generated by Google Image Charts (they now also have new, interactive charts), or RRDtool. Show below is an example of a static chart; in this example, generated by Google:

Pros and cons of static charts

Pros

  • Images can be viewed on any graphical platform. Browsers, email clients, cell phones, whatever.
  • Self contained: chart image, legend, scales: all in one image.
  • As such, easy to move around.
  • Are safe to use.

Cons

  • Images are fuzzy. Is the com_replace_psec really 0? Maybe it’s 0.1? A larger value can make lower values hard to tell.
  • Images are inaccurate: the colors can lie. The red and green lines showing are hard to tell apart. The red is painted above the green. Data gets “lost”.
  • They do not zoom (one needs to regenerate larger image)
  • Unless encoded with base64, HTML pages which include images need to link outside.
  • In the particular case of Google Charts, one is limited to 2K length URL. Trust me, it’s a big limitation! (PS, Google now support POST method to allow for up to 16K. But… it’s a POST method…)
  • In the particular case of Google Charts, one must have an internet connection.
  • In the particular case of Google Charts, one must submit data to Google.

Interactive charts

Interactive charts are those which react to your commands. These are either JavaScript or Flash based, mostly. They allow for really nice features. Take the following chart as an example: try and move over with your mouse; or select sections to zoom in.


DML


[graphDiv]


[labelsDiv]

The above chart is generated with dygraphs. Since it is embedded within my Wordpress page, the layout is affected by that of my theme. Take a look at this example page to see similar charts outside this blog site (Internet Explorer users: Maxmimize/minimize button will not work well for now. And, may I suggest Mozilla Firefox?)

Pros and cons of interactive charts

Pros

  • Can present you with exact values. No more doubt about the com_replace_psec values.
  • Can allow for zoom in, zoom out.

Cons

  • Need supporting platform. The above cannot be viewed by non-JavaScript browsers (cell phones, etc.)
  • Browser support is also an issue with JavaScript.
  • Emailing such report will result in mail blocking in many companies: mail filters will not allow for JavaScript code to pass.
  • Charts are not necessarily self-contained, in terms of the chart entity With Flash charts (e.g. Fusion Charts) this works. But in the above, the legend and scales are outside the image. As such, they cannot be just moved around.
  • HTML pages which include such charts can be self contained. The HTML page can include all the JavaScript dependencies, in addition to the chart generating code. Flash based charts cannot be self contained.

Summary

Interactive charts are cool!

I’m now integrating dygraphs into mycheckpoint (How nice it is to work with BSD & MIT licenses!). Though I may later switch to flot, interactive charts will be the next standard charting way in mycheckpoint. I will continue supporting static Google Charts, as follows from the above pros and cons list.

]]> http://code.openark.org/blog/mysql/static-charts-vs-interactive-charts/feed 2 mycheckpoint (rev. 88): mount points monitoring, improved charting, enhanced auto-deploy http://code.openark.org/blog/mysql/mycheckpoint-rev-88-mount-points-monitoring-improved-charting-enhanced-auto-deploy http://code.openark.org/blog/mysql/mycheckpoint-rev-88-mount-points-monitoring-improved-charting-enhanced-auto-deploy#comments Wed, 10 Feb 2010 06:29:47 +0000 shlomi http://code.openark.org/blog/?p=1904 Revision #88 of mycheckpoint is released. In this revision:

  • Disk space monitoring
  • Improved charting
  • Enhanced auto-deploy
  • And more…

Disk space monitoring

mycheckpoint now monitors (on Linux only) three mount points:

  1. The “/” (root) mount point
  2. The datadir mount point
  3. The tmpdir mount point

It may well be the case that two of the above (or perhaps all three of them) share the same mount point. For example, if there isn’t any particular partition for “/tmp“, it is possible that the tmpdir (by default “/tmp“) is on the same mount point as “/“. mycheckpoint does not care.

mycheckpoint monitors and reports the mount point’s used percent, in a similar algorithm df uses.

Disk space monitoring is only possible when monitoring the local machine (i.e. mycheckpoint runs on the same machine as the monitored MySQL server). In the future mycheckpoint may also monitor additional mount points, such as the various logs mount points.

Improved charting

There has been some extensive work to turn the charts into real time-series based. Google charts does not support time series charts; when it will, the required URL length would probably be too long to be used. Some SQL tweaks made it possible to display the charts in correct time-scale even if sampling is taken on non constant interval (or fail to be taken for long periods).

For more examples see the link for HTML brief reports sample, below.

I will write more on SQL Google charts generation in the future.

Enhanced auto-deploy

mycheckpoint will now detect changes to the MySQL version, in addition to changes in mycheckpoint’s version itself. This means there’s no need in ever worrying about upgrades to either one of these components. Just use mycheckpoint to take another sample; it will auto-detect if the MySQL version is different, and start sampling all those new variables introduced in the new version (or stop sampling variables no longer used). It works both for MySQL upgrades and downgrades.

Enhanced localhost detection

To determine whether it is monitoring the local host, mycheckpoint now considers the hostname for the monitored server, and sees if it is either ‘127.0.0.1′, ‘localhost’, or the machine’s hostname or fully qualified hostname.domainname (these last two additions apply for Unix based machines, and have only been tested on Linux so far).

HTML brief reports

Getting a full HTML report is time consuming. I’ve had requests (though not officially submitted through the Issues mechanism) to make it shorter. This is as yet a difficult job. There’s just too much data to aggregate (up to ~180 days of every-5-minute-samples, in a common scenario).

HTML brief reports were introduced in previous versions, and have now been enhanced to include more data. These only present last 24 hours data, and load fast. See HTML brief report sample.

Get it

Downloads are available on Google code’s mycheckpoint page. Documentation can be found on the mycheckpoint home page.

On the press

Not so new by now (it’s two months old), I’m very happy that mycheckpoint has been noted by Jeremy Zawodny in his “My Top Resources of 2009” column on Linux Magazine.

Future plans

Immediate plans for mycheckpoint are:

  • Email alerts notifications; this will allow mycheckpoint to become a real monitoring solution. Following the concept of “SQL oritented monitoring“, these will be SQL based as well.
  • Custom monitoring: allowing user defined queries to be recorded by mycheckpoint; these can then participate in alerts monitoring. This will allow for easy email notifications on program-level errors.
]]> http://code.openark.org/blog/mysql/mycheckpoint-rev-88-mount-points-monitoring-improved-charting-enhanced-auto-deploy/feed 0 mycheckpoint rev. 76: OS monitoring, auto deploy, brief HTML and 24/7 reports http://code.openark.org/blog/mysql/mycheckpoint-rev-76-os-monitoring-auto-deploy-brief-html-and-247-reports http://code.openark.org/blog/mysql/mycheckpoint-rev-76-os-monitoring-auto-deploy-brief-html-and-247-reports#comments Tue, 05 Jan 2010 08:55:14 +0000 shlomi http://code.openark.org/blog/?p=1784 Revision 76 of mycheckpoint comes with quite a few improvements, including:

  • OS monitoring (CPU, load average, memory)
  • Auto-deploy
  • Improved charting
  • Brief HTML reports
  • 24/7 charts

OS Monitoring

When monitoring the local machine, mycheckpoint now monitors CPU utilization, load average, memory and swap space.

This only applies to the Linux operating system; there is currently no plan to work this out for other operating systems.

Examples:

mysql> SELECT os_cpu_utilization_percent FROM sv_report_chart_sample;

mycheckpoint-chart-cpu-sample
mysql> SELECT ts, os_loadavg FROM mycheckpoint.sv_report_sample;
+---------------------+------------+
| 2009-12-27 11:45:01 |       1.78 |
| 2009-12-27 11:50:01 |       2.48 |
| 2009-12-27 11:55:01 |       2.35 |
...
+---------------------+------------+
mysql> SELECT report FROM mycheckpoint.sv_report_human_sample ORDER BY id DESC LIMIT 1 \G
*************************** 1. row ***************************
report:
Report period: 2009-12-27 13:20:01 to 2009-12-27 13:25:01. Period is 5 minutes (0.08 hours)
Uptime: 100.0% (Up: 334 days, 06:37:28 hours)

OS:
 Load average: 1.67
 CPU utilization: 25.2%
 Memory: 7486.4MB used out of 7985.6484MB (Active: 6685.8906MB)
 Swap: 3835.2MB used out of 8189.3750MB
...

Auto-deploy

mycheckpoint now has a version recognition mechanism. There is no need to call mycheckpoint with the “deploy” argument on first install or after upgrade. mycheckpoint will recognize a change of version and will auto-deploy before moving on to monitoring your system.

It is still possible, though, to use “deploy“, in case you just want to make sure an upgrade takes place, without issuing a monitoring action.

Improved charting

Further improvements and bug fixes made to the Google charts, including the implementation of missing values charting.

Brief HTML report

In contrast with the full blown HTML report (see sample), which presents hourly/daily/weekly reports for the many metrics, the new brief report only presents with a few hourly based charts. These include InnoDB performance, DML, OS metrics, and replication status.

To get a brief HTML report, issue:

mysql> SELECT html FROM sv_report_html_brief;

See sample brief HTML report.

24/7 charts

24/7 charts present the various metrics on a 24×7 matrix, which allows for diagnostics of usage throughout the day and week. For example, it makes it easier to see how things slow down on Saturday/Sunday; how load increases on 10:00am every day, etc.

24/7 charts are provided by the sv_report_chart_24_7 view.

DESC sv_report_chart_24_7;
+---------------------------------------+----------+------+-----+---------+-------+
| Field                                 | Type     | Null | Key | Default | Extra |
+---------------------------------------+----------+------+-----+---------+-------+
| innodb_read_hit_percent               | longblob | YES  |     | NULL    |       |
| innodb_buffer_pool_reads_psec         | longblob | YES  |     | NULL    |       |
| innodb_buffer_pool_pages_flushed_psec | longblob | YES  |     | NULL    |       |
| innodb_os_log_written_psec            | longblob | YES  |     | NULL    |       |
| innodb_row_lock_waits_psec            | longblob | YES  |     | NULL    |       |
| mega_bytes_sent_psec                  | longblob | YES  |     | NULL    |       |
| mega_bytes_received_psec              | longblob | YES  |     | NULL    |       |
| key_read_hit_percent                  | longblob | YES  |     | NULL    |       |
| key_write_hit_percent                 | longblob | YES  |     | NULL    |       |
| com_select_psec                       | longblob | YES  |     | NULL    |       |
| com_insert_psec                       | longblob | YES  |     | NULL    |       |
| com_delete_psec                       | longblob | YES  |     | NULL    |       |
| com_update_psec                       | longblob | YES  |     | NULL    |       |
| com_replace_psec                      | longblob | YES  |     | NULL    |       |
| com_set_option_percent                | longblob | YES  |     | NULL    |       |
| com_commit_percent                    | longblob | YES  |     | NULL    |       |
| slow_queries_percent                  | longblob | YES  |     | NULL    |       |
| select_scan_psec                      | longblob | YES  |     | NULL    |       |
| select_full_join_psec                 | longblob | YES  |     | NULL    |       |
| select_range_psec                     | longblob | YES  |     | NULL    |       |
| table_locks_waited_psec               | longblob | YES  |     | NULL    |       |
| opened_tables_psec                    | longblob | YES  |     | NULL    |       |
| created_tmp_tables_psec               | longblob | YES  |     | NULL    |       |
| created_tmp_disk_tables_psec          | longblob | YES  |     | NULL    |       |
| connections_psec                      | longblob | YES  |     | NULL    |       |
| aborted_connects_psec                 | longblob | YES  |     | NULL    |       |
| threads_created_psec                  | longblob | YES  |     | NULL    |       |
| seconds_behind_master                 | longblob | YES  |     | NULL    |       |
| os_loadavg                            | longblob | YES  |     | NULL    |       |
| os_cpu_utilization_percent            | longblob | YES  |     | NULL    |       |
| os_mem_used_mb                        | longblob | YES  |     | NULL    |       |
| os_mem_active_mb                      | longblob | YES  |     | NULL    |       |
| os_swap_used_mb                       | longblob | YES  |     | NULL    |       |
+---------------------------------------+----------+------+-----+---------+-------

Example:

mysql> SELECT com_select_psec, innodb_buffer_pool_pages_flushed_psec FROM mycheckpoint.sv_report_chart_24_7 \G
mycheckpoint-chart-247-sample

Trying mycheckpoint

Future plans

I haven’t got any major immediate issues; planning on user customization of charts and HTML reports. Considering thresholds and alerting for the future.

]]> http://code.openark.org/blog/mysql/mycheckpoint-rev-76-os-monitoring-auto-deploy-brief-html-and-247-reports/feed 2 New and noteworthy in mycheckpoint (rev. 57) http://code.openark.org/blog/mysql/new-and-noteworthy-in-mycheckpoint-rev-57 http://code.openark.org/blog/mysql/new-and-noteworthy-in-mycheckpoint-rev-57#comments Wed, 16 Dec 2009 19:12:07 +0000 shlomi http://code.openark.org/blog/?p=1695 Rev. 57 of mycheckpoint has been released and is available for download.

New and updated in this revision:

Remote host monitoring

It is now possible to monitor one host, while writing into another. Either or both could be remote hosts:

mycheckpoint --host=localhost --monitored-host=192.168.10.178

The above monitors the MySQL server on 192.168.10.178, and writes down to localhost (to be queried later)

mycheckpoint --monitored-host=127.0.0.1 --host=192.168.10.178

The above monitors the MySQL server on 127.0.0.1, and writes down to 192.168.10.178.

As result of the above addition, binary logging for monitoring statements are now enabled by default. The previous solution, in which binary logging were disabled by default and the same schema was utilized by different servers was far from being a clean solution. [Read more on Remote & multiple hosts monitoring]

Improved charting

The google charts reports have been improved significantly. Here’s how:

BeforeAfter
Before on left (or above), After on right (or below)

And

BeforeAfter
Before on left (or above), After on right (or below)

Recap: the charts are not stored anywhere; they are being calculated and generated by SQL queries & views, based on the raw data. For example, to generate the above, I do:

SELECT innodb_estimated_log_mb_written_per_hour FROM sv_report_chart_sample\G
*************************** 1. row ***************************
innodb_estimated_log_mb_written_per_hour: http://chart.apis.google.com/chart?cht=lc&chs=400x200&chts=303030,12&chtt=Latest+24+hours:+Dec+8,+08:20++-++Dec+9,+08:20&chdl=innodb_estimated_log_mb_written_per_hour&chdlp=b&chco=ff8c00&chd=s:OQOTSPPQPSROUQMTRSQTPSURQTQQPSOSRTRRUTRRTSRRRSRSTRRQSURSUVRRXQTVRUTTVUTOYTXSYTTSYUSUTWUPRRPQSQRQTPPSPQMPPQOQMQPMOMOLQORNNPNRNNOPQOOMQQPNRNMNORQMSTQQPTPRUQVQTTVSURUVVVSSSVWQVUSTVSWSSUURWRVRTWdotpQNROPQRPQQMPLRO9PWNRPNNNPXUMLNNPQQPSONPLLNWXZTQROSQMOONQPLJNOOQQKMPSMMPLfPSSSRTUQSORSSRSPNRSQSQ&chxt=x,y&chxr=1,0,2708.0&chxl=0:|09:00||||13:00||||17:00||||21:00||||01:00||||05:00||||&chxs=0,505050,10&chg=4.17,25,1,3,2.78,0&chxp=0,2.78,6.95,11.12,15.29,19.46,23.63,27.80,31.97,36.14,40.31,44.48,48.65,52.82,56.99,61.16,65.33,69.50,73.67,77.84,82.01,86.18,90.35,94.52,98.69

If you thought I was going over the edge in Auto scaling, scaled SQL graphs concluded or SQL pie chart, allow me to assure you: generation of the above charts using SQL & views is SQL blasphemy. While cool, the Google Charts API does not do too much on its own, and expects the caller to do the math for putting in the grids, scales and labels (and though all of them are tightly related, they must all be explicitly specified). I’ll write a later post on this.

Flexible charting

A new table, called charts_api, is introduced. It contains one row, which lists charting configuration.

SELECT * FROM charts_api;
+-------------+--------------+----------------------------------------------------------------+------------------------------------+
| chart_width | chart_height | simple_encoding                                                | service_url                        |
+-------------+--------------+----------------------------------------------------------------+------------------------------------+
|         400 |          200 | ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789 | http://chart.apis.google.com/chart |
+-------------+--------------+----------------------------------------------------------------+------------------------------------+

To change generated charts dimensions, simply update chart_width or chart_height.

To use other chart APIs, update service_url. I’ve tried out Eastwood Charts, a Java-based service, which implements the Google Charts API. There’s still some stuff missing, such as the positioning of labels and grids; and it’s slightly less polished (but configurable). But if you’re concerned about sending requests to Google, it’s a viable alternative.

So, drop the WAR file into your Tomcat / JBoss / other j2ee server, then follow http://127.0.0.1/eastwood/chart?…

Fix to questions vs. queries issues

mycheckpoint now correctly identifies whether to use questions or queries when analyzing DML queries. If you got some 570% for com_update out of total queries and wondered about it, it should now be resolved to normal values.

On upgrading your current installation

  1. Download & install via debian or python installers
  2. Issue mycheckpoint once with “deploy” as argument
  3. Done

Future plans

I’m working on 24×7 charting; minimal operating system monitoring; auto-detection and auto-deploy on version upgrade; and more.

Stay tuned!

]]> http://code.openark.org/blog/mysql/new-and-noteworthy-in-mycheckpoint-rev-57/feed 1 SQL multi line chart http://code.openark.org/blog/mysql/sql-multi-line-chart http://code.openark.org/blog/mysql/sql-multi-line-chart#comments Tue, 03 Nov 2009 10:44:13 +0000 shlomi http://code.openark.org/blog/?p=1400 Time for another charting SQL query. I wish to present a single-query generated multi-line/area chart. I’ll walk through some of the steps towards making this happen. By the end of this post I’ll present some real-data charts, area charts and colored charts.

+---------+-----------------------------------------------------------------------------------+
| y_scale | chart                                                                             |
+---------+-----------------------------------------------------------------------------------+
| 1       | ****---------#######----------------------------------------*******--------###### |
| 0.88    | ----***---###-------##-----------------------------------***-------***---##------ |
| 0.75    | -------**#------------##-------------------------------**-------------*##-------- |
| 0.63    | ------##-*--------------##----------------------------*--------------##**-------- |
| 0.5     | -@@@@@@@@@@@@@@@----------#-----@@@@@@@@@@@@@@@@----**---------@@@@@@@@@@@@@@@@-- |
| 0.38    | ----#-------*--------------##----------------------*---------------#-------*----- |
| 0.25    | --##---------*---------------#--------------------*--------------##---------*---- |
| 0.13    | -#------------**--------------#-----------------**--------------#------------*--- |
| 0       | @---------------*--------------##--------------*---------------#--------------**- |
| -0.12   | -----------------*---------------#------------*--------------##-----------------* |
| -0.25   | ------------------*---------------#---------**--------------#-------------------- |
| -0.37   | -------------------**--------------#-------*---------------#--------------------- |
| -0.5    | ----------------@@@@@@@@@@@@@@@@----##----*-----@@@@@@@@@@@@@@@----------------@@ |
| -0.62   | ----------------------**--------------#-**--------------#------------------------ |
| -0.75   | ------------------------**------------**#-------------##------------------------- |
| -0.87   | --------------------------**-------***---###-------###--------------------------- |
| -1      | ----------------------------*******---------#######------------------------------ |
|         | v:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::v |
|         | 0                                                                               8 |
|         |     # sin(x)                                                                      |
|         |     * cos(x)                                                                      |
|         |     @ sign(tan(x))/2                                                              |
+---------+-----------------------------------------------------------------------------------+

Requirements

We need a generic query, which returns at least these two columns: ordering_column and row_values, The latter being a comma-delimited list of values. For example, the following query will do:

SELECT
 value/10 AS ordering_column,
 CONCAT(SIN(value/10), ',', COS(value/10), ',', SIGN(TAN(value/10))/2) AS row_values,
 @multi_line_chart_values_legend := 'sin(x),cos(x),sign(tan(x))/2' AS legend
FROM tinyint_asc
LIMIT 81;
+-----------------+---------------------------------------------+------------------------------+
| ordering_column | row_values                                  | legend                       |
+-----------------+---------------------------------------------+------------------------------+
|          0.0000 | 0,1,0.0000                                  | sin(x),cos(x),sign(tan(x))/2 |
|          0.1000 | 0.099833416646828,0.99500416527803,0.5000   | sin(x),cos(x),sign(tan(x))/2 |
|          0.2000 | 0.19866933079506,0.98006657784124,0.5000    | sin(x),cos(x),sign(tan(x))/2 |
|          0.3000 | 0.29552020666134,0.95533648912561,0.5000    | sin(x),cos(x),sign(tan(x))/2 |
|          0.4000 | 0.38941834230865,0.92106099400289,0.5000    | sin(x),cos(x),sign(tan(x))/2 |
|          0.5000 | 0.4794255386042,0.87758256189037,0.5000     | sin(x),cos(x),sign(tan(x))/2 |
|          0.6000 | 0.56464247339504,0.82533561490968,0.5000    | sin(x),cos(x),sign(tan(x))/2 |
...

Don’t worry about the @multi_line_chart_values_legend variable; it will be used later on for presenting chart legend.

Since I’ve already presented with how to create line charts in Auto scaling, scaled SQL graphs concluded, we will concentrate on how to present a single chart column, which includes multiple values.

Step 1

Let’s say we wish to present the values 12,5,16. We wish to eventually generate something like:

+------------------+
| unwalked_bar     |
+------------------+
| ----Y------X---Z |
+------------------+

Where X is in position 12, Y in position 5, and Z in position 16.

We begin with having these values concatenated into one string:

SET @values := '12,5,16';
SET @num_values := CHAR_LENGTH(@values)-CHAR_LENGTH(REPLACE(@values,',',''))+1;
SELECT @num_values;
+-------------+
| @num_values |
+-------------+
| 3           |
+-------------+

The query above parses the string and correctly identified there are three tokens.

Step 2

We now move on to tokenizing the text:

SELECT
 *,
 SUBSTRING_INDEX(SUBSTRING_INDEX(@row_values, ',', tinyint_asc.value), ',', -1) AS row_value
FROM
 tinyint_asc,
 (SELECT @row_values := '12,5,16' AS row_values) AS sel_values,
 (SELECT @num_values := CHAR_LENGTH(@row_values)-CHAR_LENGTH(REPLACE(@row_values,',',''))+1 AS num_values) AS sel_num_values
WHERE
 tinyint_asc.value BETWEEN 1 AND @num_values
;
+-------+------------+------------+-----------+
| value | row_values | num_values | row_value |
+-------+------------+------------+-----------+
|     1 | 12,5,16    |          3 | 12        |
|     2 | 12,5,16    |          3 | 5         |
|     3 | 12,5,16    |          3 | 16        |
+-------+------------+------------+-----------+

Using a numbers table (tinyint_asc), we walk the string and tokenize it. We also maintain an indicator per value.

Step 3

We need to recognize the minimum and maximum values:

SELECT
  value AS row_value_indicator,
  row_value,
  @min_row_value := LEAST(IFNULL(@min_row_value, row_value), row_value) AS min_row_value,
  @max_row_value := GREATEST(IFNULL(@max_row_value, row_value), row_value) AS max_row_value
FROM (
  SELECT
  *,
  SUBSTRING_INDEX(SUBSTRING_INDEX(@row_values, ',', tinyint_asc.value), ',', -1)+0 AS row_value
    FROM
      tinyint_asc,
      (SELECT @row_values := '12,5,16' AS row_values) AS sel_values,
      (SELECT @num_values := CHAR_LENGTH(@row_values)-CHAR_LENGTH(REPLACE(@row_values,',',''))+1 AS num_values) AS sel_num_values,
      (SELECT @min_row_value := NULL) AS sel_min_row_value,
      (SELECT @max_row_value := NULL) AS sel_max_row_value
    WHERE
      tinyint_asc.value BETWEEN 1 AND @num_values
  ) sel_row_values
;
+---------------------+-----------+---------------+---------------+
| row_value_indicator | row_value | min_row_value | max_row_value |
+---------------------+-----------+---------------+---------------+
|                   1 |        12 |            12 |            12 |
|                   2 |         5 |             5 |            12 |
|                   3 |        16 |             5 |            16 |
+---------------------+-----------+---------------+---------------+

The last line presents the correct values: 5 and 16 being min, max values respectively.

Step 4

We now iterate from min value to max value, this being the graph range, and, per row value (token in row_values column), we indicate whether there’s a pixel on the graph.

SELECT
  * ,
  IF(tinyint_asc.value = row_value, row_value_indicator, '-') AS display_val
FROM
  tinyint_asc,
  (SELECT
    value AS row_value_indicator,
    row_value,
    @min_row_value := LEAST(IFNULL(@min_row_value, row_value), row_value) AS min_row_value,
    @max_row_value := GREATEST(IFNULL(@max_row_value, row_value), row_value) AS max_row_value
    FROM (
    SELECT
      *,
      SUBSTRING_INDEX(SUBSTRING_INDEX(@row_values, ',', tinyint_asc.value), ',', -1)+0 AS row_value
    FROM
      tinyint_asc,
      (SELECT @row_values := '12,5,16' AS row_values) AS sel_values,
      (SELECT @num_values := CHAR_LENGTH(@row_values)-CHAR_LENGTH(REPLACE(@row_values,',',''))+1 AS num_values) AS sel_num_values,
      (SELECT @min_row_value := NULL) AS sel_min_row_value,
      (SELECT @max_row_value := NULL) AS sel_max_row_value
    WHERE
      tinyint_asc.value BETWEEN 1 AND @num_values
    ) sel_row_values
  ) AS sel_row_values_indicators
WHERE
  tinyint_asc.value BETWEEN 1 AND @max_row_value
;
+-------+---------------------+-----------+---------------+---------------+-------------+
| value | row_value_indicator | row_value | min_row_value | max_row_value | display_val |
+-------+---------------------+-----------+---------------+---------------+-------------+
|     1 |                   1 |        12 |            12 |            12 | -           |
|     1 |                   2 |         5 |             5 |            12 | -           |
|     1 |                   3 |        16 |             5 |            16 | -           |
|     2 |                   1 |        12 |            12 |            12 | -           |
|     2 |                   2 |         5 |             5 |            12 | -           |
|     2 |                   3 |        16 |             5 |            16 | -           |
|     3 |                   1 |        12 |            12 |            12 | -           |
|     3 |                   2 |         5 |             5 |            12 | -           |
|     3 |                   3 |        16 |             5 |            16 | -           |
|     4 |                   1 |        12 |            12 |            12 | -           |
|     4 |                   2 |         5 |             5 |            12 | -           |
|     4 |                   3 |        16 |             5 |            16 | -           |
|     5 |                   1 |        12 |            12 |            12 | -           |
|     5 |                   2 |         5 |             5 |            12 | 2           |
|     5 |                   3 |        16 |             5 |            16 | -           |
|     6 |                   1 |        12 |            12 |            12 | -           |
|     6 |                   2 |         5 |             5 |            12 | -           |
|     6 |                   3 |        16 |             5 |            16 | -           |

This table can get very long.

Step 5

We now group the marks per graph-row value. If there’s no mark, we present with an empty space. If only one mark is present, we display that mark. If two lines collide on that point, we need to decide which mark to use. In the following example, we choose by order of appearance in the row_values column. Another way (used later on) is to choose the higher value.

SELECT
  string_position,
  LEFT(GROUP_CONCAT(bar_string_token ORDER BY bar_string_token DESC separator ''), 1) AS chosen_mark
FROM
  (SELECT
    value AS string_position,
    IF(tinyint_asc.value = row_value, row_value_indicator, ' ') AS bar_string_token
  FROM
    tinyint_asc,
    (SELECT
      value AS row_value_indicator,
      row_value,
      @min_row_value := LEAST(IFNULL(@min_row_value, row_value), row_value) AS min_row_value,
      @max_row_value := GREATEST(IFNULL(@max_row_value, row_value), row_value) AS max_row_value
      FROM (
      SELECT
        *,
        SUBSTRING_INDEX(SUBSTRING_INDEX(@row_values, ',', tinyint_asc.value), ',', -1)+0 AS row_value
      FROM
        tinyint_asc,
        (SELECT @row_values := '12,5,16' AS row_values) AS sel_values,
        (SELECT @num_values := CHAR_LENGTH(@row_values)-CHAR_LENGTH(REPLACE(@row_values,',',''))+1 AS num_values) AS sel_num_values,
        (SELECT @min_row_value := NULL) AS sel_min_row_value,
        (SELECT @max_row_value := NULL) AS sel_max_row_value
      WHERE
        tinyint_asc.value BETWEEN 1 AND @num_values
      ) sel_row_values
    ) AS sel_row_values_indicators
  WHERE
    tinyint_asc.value BETWEEN 1 AND @max_row_value
  ) AS sel_marked_row_values
GROUP BY
  string_position
;
+-----------------+-------------+
| string_position | chosen_mark |
+-----------------+-------------+
|               1 |             |
|               2 |             |
|               3 |             |
|               4 |             |
|               5 | 2           |
|               6 |             |
|               7 |             |
|               8 |             |
|               9 |             |
|              10 |             |
|              11 |             |
|              12 | 1           |
|              13 |             |
|              14 |             |
|              15 |             |
|              16 | 3           |
+-----------------+-------------+

Step 6

It’s now just a matter of using GROUP_CONCAT to turn this into the following horizontal bar:

+------------------+
| unwalked_bar     |
+------------------+
| ----2------1---3 |
+------------------+

(I know I’m mixing spaces and dashes, it’s just for display purposes).

Putting it all together

From this point on, we use the techniques shown on Auto scaling, scaled SQL graphs concluded to generate the complete chart. For the curious, here’s what the complete query looks like. The text in bold is the only thing that needs to change; replace this with your own query.

SELECT
  y_scale,
  horizontal_bar as chart
  FROM
  (
  SELECT
    @multi_line_chart_row_number := @multi_line_chart_row_number+1,
    CASE @multi_line_chart_row_number
      WHEN 1 THEN ROUND(@multi_line_chart_max_value, @multi_line_chart_value_precision)
      WHEN @multi_line_chart_graph_rows THEN ROUND(@multi_line_chart_min_value, @multi_line_chart_value_precision)
      ELSE ROUND(@multi_line_chart_max_value-(@multi_line_chart_max_value-@multi_line_chart_min_value)*(@multi_line_chart_row_number-1)/(@multi_line_chart_graph_rows-1), @multi_line_chart_value_precision)
    END AS y_scale,
    horizontal_bar,
    @multi_line_chart_bar_length := IFNULL(@multi_line_chart_bar_length, CHAR_LENGTH(horizontal_bar))
  FROM
    (SELECT @multi_line_chart_row_number := 0) AS select_row
    INNER JOIN
    (
    SELECT
      GROUP_CONCAT(SUBSTRING(unwalked_bar, tinyint_asc.value, 1) ORDER BY ordering_column SEPARATOR '') AS horizontal_bar
    FROM
      tinyint_asc
    INNER JOIN (
      SELECT
        ordering_column,
        GROUP_CONCAT(bar_string_token ORDER BY string_position SEPARATOR '') AS unwalked_bar
      FROM
        (SELECT
          ordering_column,
          string_position,
          scaled_string_position,
          REPLACE(LEFT(GROUP_CONCAT(bar_string_token ORDER BY bar_string_token DESC SEPARATOR ''), 1), ' ', '-') AS bar_string_token
        FROM
          (SELECT
            ordering_column,
            @multi_line_chart_scaled_string_position := CONVERT((row_value-@multi_line_chart_min_value)*(@multi_line_chart_graph_rows-1)/(@multi_line_chart_max_value-@multi_line_chart_min_value), UNSIGNED) AS scaled_string_position,
            value AS string_position,
            IF(tinyint_asc.value = @multi_line_chart_scaled_string_position+1, SUBSTRING(IF(@multi_line_chart_values_legend IS NULL, @multi_line_chart_graph_fallback_colors, @multi_line_chart_graph_colors), row_value_indicator, 1), ' ') AS bar_string_token
          FROM
            tinyint_asc,
            (SELECT
              ordering_column,
              value AS row_value_indicator,
              row_value
            FROM (
              SELECT
                *,
                @multi_line_chart_min_value := LEAST(IFNULL(@multi_line_chart_min_value, row_value), row_value) AS min_value,
                @multi_line_chart_max_value := GREATEST(IFNULL(@multi_line_chart_max_value, row_value), row_value) AS max_value,
                @multi_line_chart_min_range := LEAST(IFNULL(@multi_line_chart_min_range, ordering_column), ordering_column) AS min_range,
                @multi_line_chart_max_range := GREATEST(IFNULL(@multi_line_chart_max_range, ordering_column), ordering_column) AS max_range
              FROM
                (SELECT
                  *,
                  SUBSTRING_INDEX(SUBSTRING_INDEX(row_values, ',', tinyint_asc.value), ',', -1)+0 AS row_value,
                  @multi_line_chart_num_values := CHAR_LENGTH(row_values)-CHAR_LENGTH(REPLACE(row_values,',',''))+1 AS num_values
                FROM
                  (SELECT @multi_line_chart_values_legend := NULL) AS select_nullify_values_legend,
                  (SELECT @multi_line_chart_bar_length := NULL) AS select_nullify_bar_length,
                  tinyint_asc,
                  (
                    SELECT
                      value/10 AS ordering_column,
                      CONCAT(SIN(value/10), ',', COS(value/10), ',', SIGN(TAN(value/10))/2) AS row_values,
                      @multi_line_chart_values_legend := 'sin(x),cos(x),sign(tan(x))/2'
                    FROM tinyint_asc LIMIT 81
                  ) AS sel_main_values,
                  (SELECT @multi_line_chart_min_value := NULL) AS select_min,
                  (SELECT @multi_line_chart_max_value := NULL) AS select_max,
                  (SELECT @multi_line_chart_min_range := NULL) AS select_min_range,
                  (SELECT @multi_line_chart_max_range := NULL) AS select_max_range,
                  (SELECT @multi_line_chart_graph_colors := '#*@%o+x;m:') AS select_graph_colors,
                  (SELECT @multi_line_chart_graph_fallback_colors := 'abcdefghij') AS select_graph_fallback_colors,
                  (SELECT @multi_line_chart_value_precision := 2) AS select_value_precision,
                  (SELECT @multi_line_chart_graph_rows := 17) AS select_graph_rows
                ) sel_counted_values_main_values
              WHERE
                value BETWEEN 1 AND @multi_line_chart_num_values
              ) sel_row_values
            ) AS sel_row_values_indicators
          WHERE
            tinyint_asc.value BETWEEN 1 AND @multi_line_chart_graph_rows
          ) AS sel_marked_row_values
        GROUP BY
          ordering_column, string_position
        ) AS sel_walked_bar
      GROUP BY
        ordering_column
    ) AS select_vertical
    WHERE
      tinyint_asc.value BETWEEN 1 AND CHAR_LENGTH(unwalked_bar)
    GROUP BY
      tinyint_asc.value
    ORDER BY
      tinyint_asc.value DESC
    ) AS select_horizontal
  ) AS select_horizontal_untitled
UNION ALL
SELECT '', CONCAT('v', REPEAT(':', @multi_line_chart_bar_length-2), 'v')
UNION ALL
SELECT '', CONCAT(@multi_line_chart_min_range, REPEAT(' ', @multi_line_chart_bar_length-CHAR_LENGTH(@multi_line_chart_min_range)-CHAR_LENGTH(@multi_line_chart_max_range)), @multi_line_chart_max_range)
UNION ALL
SELECT
  '', CONCAT('    ', SUBSTRING(@multi_line_chart_graph_colors, value, 1), ' ', SUBSTRING_INDEX(SUBSTRING_INDEX(@multi_line_chart_values_legend, ',', value), ',', -1))
FROM
  tinyint_asc
WHERE
  value BETWEEN 1 AND @multi_line_chart_num_values
  AND @multi_line_chart_values_legend IS NOT NULL
;

Area charts

Sinus functions curves go nicely with ASCII art. But your everyday chart won’t have such nice curves. There is some erratic behavior to, say, your number of queries per hour. It doesn’t display well in the above graph.

Which is why we can choose to present area graphs: not only display the line, but also the area beneath it.

This leads to the problem of hiding: higher values can hide the lower values display. So we need to adjust the above logic and make sure we always bring to front the lower values.

The following example shows the relation between the percentage of slow queries (per hour) and the percentage of full table scans (of total queries, per hour). It’s really fun to see the unmistakeable connection!

+---------+---------------------------------------------------------------+
| y_scale | chart                                                         |
+---------+---------------------------------------------------------------+
| 4       | --------------------------------*--*------------------------- |
| 3       | -----------------------------****-*****------------------*--- |
| 3       | -----------------------------**********-----------------**-*- |
| 3       | ----------*------------------**********--------------*****-** |
| 3       | ------*--**-----------------************-*-----------******** |
| 3       | ------*******---------------****************--------********* |
| 3       | ------*******---------------******************------********* |
| 3       | ------*********-------------******************------********* |
| 3       | -----************--------*-*******************------********* |
| 2       | *----*****************--************************-**-********* |
| 2       | *----******************************************************** |
| 2       | *----******************************************************** |
| 2       | **--********************************************************* |
| 2       | ************************************************************* |
| 2       | ************************************************************* |
| 2       | ************************************************************* |
| 2       | *****************************#******************************* |
| 1       | *****************************##########*****************##*#* |
| 1       | ******#***#*****************###########**************######## |
| 1       | ******#######***************###############*********######### |
| 1       | #*****########**************##################******######### |
| 1       | #****##########*#*******##*###################******######### |
| 1       | #***######################################################### |
| 1       | #***######################################################### |
| 1       | ############################################################# |
|         | v:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::v |
|         | 2009-10-22 20:00:00                       2009-10-25 08:00:00 |
|         |     # slow queries percent                                    |
|         |     * select scan percent                                     |
+---------+---------------------------------------------------------------

Next we look at the number of SELECT, INSERT, UPDATE, DELETE queries per sec – on a per hour basis.

+---------+---------------------------------------------------------------+
| y_scale | chart                                                         |
+---------+---------------------------------------------------------------+
| 135     | --##--------------------------------------------------------- |
| 130     | -###--------------------------------------------------------- |
| 125     | -###----------------------##--------------------------------- |
| 120     | -###-------------------#####--------------------------------- |
| 115     | -###-----------------#######--------------------------------- |
| 110     | -###----------------########----------------------##--------- |
| 104     | -###--------------##########--------------------####--------- |
| 99      | ####*--------#--############------------------######--------- |
| 94      | ####*--------#-#############---------------#########--------- |
| 89      | #####-----##################------------############--------# |
| 84      | ############################-----------#############--------# |
| 79      | ####%########################---####################---###### |
| 74      | ####%######################################################## |
| 69      | ####%*####################################################### |
| 64      | ####%*####################################################### |
| 59      | ##*#%*###############*####################################### |
| 54      | #**#%*#####*#*****###*****##*###########*#####*#####*#####*## |
| 49      | #***%%*#*********************##########****#*********#**##*** |
| 44      | #***%%*************************##**************************** |
| 39      | ****%%******************************************************* |
| 33      | ****%%**********************************************%******** |
| 28      | *%%*%%**********%******%%*%*%*******%***************%******** |
| 23      | %%%%o%**%**%%o%%%%%%%%%%%%%%o******%%*%%%%%%%%%%%%%%%%*****%% |
| 18      | %oooooooo%%oooooooooooooooooo%%oo%oooooooooooooooooooo%o%%ooo |
| 13      | ooooooooo%%oooooooooooooooooo%%%%%%ooooooooooooooooooo%o%%ooo |
|         | v:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::v |
|         | 2009-10-22 20:00:00                       2009-10-25 08:00:00 |
|         |     # com_select per second                                   |
|         |     * com_insert per second                                   |
|         |     o com_update per second                                   |
|         |     % com_delete per second                                   |
+---------+---------------------------------------------------------------+

Here’s the code to do that.

Colored charts

Time to add colors! The previous charts can be easily improved by sending the terminal escape characters to instruct initiating color display (thanks to TheVoo for providing the idea and implementation for that on SQL pie chart).

I recommend using the following pager if you’re on unix-like OS:

pager awk '{sub(/..$/, "")} {sub(/[ ]+$/, "")} !/^[+][-]/ && NR > 3 { print }'

The following chart is similar to the above, but now utilizes colors. It presents the number of popular DML queries per second, on an hourly basis.

Colored multi area chart

Again, comparing slow queries with full table scans:

Slow queries and full scans overlay

Slow queries and full scans overlay

Comparing number of created temporary tables with number of created disk temporary tables:

temp tables vs disk temp tables

temp tables vs disk temp tables

And, once again, the number of popular DML statements, zoomed in.

color_multi_line_chart_dml

Here’s the code for colored charts.

Some of the techinuqes used in my charting series of blogs can be used, pretty much in the same way, in order to generate Google charts. But other techniques can be used, as well.

More on this in future posts.

]]> http://code.openark.org/blog/mysql/sql-multi-line-chart/feed 3 SQL pie chart http://code.openark.org/blog/mysql/sql-pie-chart http://code.openark.org/blog/mysql/sql-pie-chart#comments Wed, 12 Aug 2009 10:49:41 +0000 shlomi http://code.openark.org/blog/?p=1093 My other half says I’m losing it. But I think that as an enthusiast kernel developer she doesn’t have the right to criticize people. (“I like user space better!” – she exclaims upon reading this).

Shown below is a (single query) SQL-generated pie chart. I will walk through the steps towards making this happen, and conclude with what, I hope you’ll agree, are real-world, useful usage samples.

+----------------------------------------------------------------------+
| pie_chart                                                            |
+----------------------------------------------------------------------+
|                                                                      |
|                         ;;;;;;;;;;;;;;;;;;;;;                        |
|                  oooooooo;;;;;;;;;;;;;;;;;;;;;;;;;;;                 |
|             oooooooooooooo;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;            |
|          ooooooooooooooooo                 ;;;;;;;;;;;;#####         |
|        oooooooooooooo                           ;#############       |
|       oooooooooooo                                 ############      |
|      oooooooooooo                                   ############     |
|      ooooooooooo                                     ###########     |
|      oooooooooooo                                   ::::::::::::     |
|       oooooooooooo                                 ::::::::::::      |
|        ooooooooo:::::                           ::::::::::::::       |
|          o::::::::::::::::                 :::::::::::::::::         |
|             :::::::::::::::::::::::::::::::::::::::::::::            |
|                  :::::::::::::::::::::::::::::::::::                 |
|                         :::::::::::::::::::::                        |
|                                                                      |
| ##  red: 1 (10%)                                                     |
| ;;  blue: 2 (20%)                                                    |
| oo  orange: 3 (30%)                                                  |
| ::  white: 4 (40%)                                                   |
+----------------------------------------------------------------------+

Requirements

We need a generic query, which returns at least these two columns: name_column and value_column. For example, the following query will do:

SELECT name AS name_column, value AS value_column FROM sample_values2;
+-------------+--------------+
| name_column | value_column |
+-------------+--------------+
| red         |            1 |
| blue        |            2 |
| orange      |            3 |
| white       |            4 |
+-------------+--------------+

Find sample data in pie_data.sql.

Part 1: expanding the original query

We’re going to need to take the above query’s results and expand them: how much is the ratio from total, per value? As first step, accumulate values:

SELECT
  name_column,
  value_column,
  @accumulating_value := @accumulating_value+value_column AS accumulating_value
FROM (
  SELECT name AS name_column, value AS value_column FROM sample_values2
  ) select_values,
  (SELECT @accumulating_value := 0) select_accumulating_value
;
+-------------+--------------+--------------------+
| name_column | value_column | accumulating_value |
+-------------+--------------+--------------------+
| red         |            1 |                  1 |
| blue        |            2 |                  3 |
| orange      |            3 |                  6 |
| white       |            4 |                 10 |
+-------------+--------------+--------------------+

Next, we calculate ratio of accumulating value, and present it both in [0..1] range, as well as in [0..2*PI] (radians):

SELECT
  name_order,
  name_column,
  value_column,
  accumulating_value,
  accumulating_value/@accumulating_value AS accumulating_value_ratio,
  2*PI()*accumulating_value/@accumulating_value AS accumulating_value_radians
FROM (
  SELECT
    name_column,
    value_column,
    @name_order := @name_order+1 AS name_order,
    @accumulating_value := @accumulating_value+value_column AS accumulating_value,
    @aggregated_name_column := CONCAT(@aggregated_name_column, name_column, ',') AS aggregated_name_column
  FROM (
    SELECT name AS name_column, value AS value_column FROM sample_values2
    ) select_values,
    (SELECT @name_order := 0) select_name_order,
    (SELECT @accumulating_value := 0) select_accumulating_value,
    (SELECT @aggregated_name_column := '') select_aggregated_name_column
  ) select_accumulating_values
;
+------------+-------------+--------------+--------------------+--------------------------+----------------------------+
| name_order | name_column | value_column | accumulating_value | accumulating_value_ratio | accumulating_value_radians |
+------------+-------------+--------------+--------------------+--------------------------+----------------------------+
|          1 | red         |            1 |                  1 |                      0.1 |           0.62831853071796 |
|          2 | blue        |            2 |                  3 |                      0.3 |            1.8849555921539 |
|          3 | orange      |            3 |                  6 |                      0.6 |            3.7699111843078 |
|          4 | white       |            4 |                 10 |                        1 |            6.2831853071796 |
+------------+-------------+--------------+--------------------+--------------------------+----------------------------+

The radians value will help us decide where in the pie chart lies each value.

Part 2: behind the scenes of the pie chart

We now explain how the pie chart works. Later on we combine with Part 1, to produce the complete chart.

We first generate a coordinates system (see SQL graphics):

SELECT
  GROUP_CONCAT(CONCAT(t2.value,'.',t1.value) order by t1.value separator ' ') as circle
FROM
  tinyint_asc t1,
  tinyint_asc t2,
  (select @size := 10) sel_size,
  (select @radius := (@size/2 - 1)) sel_radius
WHERE
  t1.value < @size
  AND t2.value < @size
GROUP BY t2.value
;
+-----------------------------------------+
| circle                                  |
+-----------------------------------------+
| 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 |
| 1.0 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 |
| 2.0 2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 2.9 |
| 3.0 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 |
| 4.0 4.1 4.2 4.3 4.4 4.5 4.6 4.7 4.8 4.9 |
| 5.0 5.1 5.2 5.3 5.4 5.5 5.6 5.7 5.8 5.9 |
| 6.0 6.1 6.2 6.3 6.4 6.5 6.6 6.7 6.8 6.9 |
| 7.0 7.1 7.2 7.3 7.4 7.5 7.6 7.7 7.8 7.9 |
| 8.0 8.1 8.2 8.3 8.4 8.5 8.6 8.7 8.8 8.9 |
| 9.0 9.1 9.2 9.3 9.4 9.5 9.6 9.7 9.8 9.9 |
+-----------------------------------------+

Taking a slightly big step further, we calculate the angle per coordinate, in relation to center of coordinate system. Calculation is in radians, but presented in degrees, since it’s more readable. Also, we note in which quarter of the graph each point lies.

SELECT
  group_concat(
    round(radians*180/PI())
    order by col_number separator ' ') as circle
FROM (
  SELECT
    t1.value AS col_number,
    t2.value AS row_number,
    @dx := (t1.value - (@size-1)/2) AS dx,
    @dy := ((@size-1)/2 - t2.value) AS dy,
    @abs_radians := IF(@dx = 0, PI()/2, (atan(abs(@dy/@dx)))) AS abs_radians,
    CASE
      WHEN SIGN(@dy) >= 0 AND SIGN(@dx) >= 0 THEN @abs_radians
      WHEN SIGN(@dy) >= 0 AND SIGN(@dx) <= 0 THEN PI()-@abs_radians
      WHEN SIGN(@dy) <= 0 AND SIGN(@dx) <= 0 THEN PI()+@abs_radians
      WHEN SIGN(@dy) <= 0 AND SIGN(@dx) >= 0 THEN 2*PI()-@abs_radians
    END AS radians
  FROM
    tinyint_asc t1,
    tinyint_asc t2,
    (select @size := 15) sel_size,
    (select @radius := (@size/2 - 1)) sel_radius
  WHERE
    t1.value < @size
    AND t2.value < @size) select_combinations
  GROUP BY row_number
;
+-------------------------------------------------------------+
| circle                                                      |
+-------------------------------------------------------------+
| 135 131 126 120 113 106 98 90 82 74 67 60 54 49 45          |
| 139 135 130 124 117 108 99 90 81 72 63 56 50 45 41          |
| 144 140 135 129 121 112 101 90 79 68 59 51 45 40 36         |
| 150 146 141 135 127 117 104 90 76 63 53 45 39 34 30         |
| 157 153 149 143 135 124 108 90 72 56 45 37 31 27 23         |
| 164 162 158 153 146 135 117 90 63 45 34 27 22 18 16         |
| 172 171 169 166 162 153 135 90 45 27 18 14 11 9 8           |
| 180 180 180 180 180 180 180 90 0 0 0 0 0 0 0                |
| 188 189 191 194 198 207 225 270 315 333 342 346 349 351 352 |
| 196 198 202 207 214 225 243 270 297 315 326 333 338 342 344 |
| 203 207 211 217 225 236 252 270 288 304 315 323 329 333 337 |
| 210 214 219 225 233 243 256 270 284 297 307 315 321 326 330 |
| 216 220 225 231 239 248 259 270 281 292 301 309 315 320 324 |
| 221 225 230 236 243 252 261 270 279 288 297 304 310 315 319 |
| 225 229 234 240 247 254 262 270 278 286 293 300 306 311 315 |
+-------------------------------------------------------------+

The above needs some formattign to present well, but that’s not the purpose; I’m only showing the above to explain the steps taken.

Part 3: combining the two

Next step is probably the most significant one: we’re going to present a rough, square, weird looking pie chart using the original values:

SELECT
  group_concat(
    (SELECT name_order FROM
      (
      SELECT
        name_order,
        name_column,
        value_column,
        accumulating_value,
        accumulating_value/@accumulating_value AS accumulating_value_ratio,
        2*PI()*accumulating_value/@accumulating_value AS accumulating_value_radians
      FROM (
        SELECT
          name_column,
          value_column,
          @name_order := @name_order+1 AS name_order,
          @accumulating_value := @accumulating_value+value_column AS accumulating_value,
          @aggregated_name_column := CONCAT(@aggregated_name_column, name_column, ',') AS aggregated_name_column
        FROM (
          SELECT name AS name_column, value AS value_column FROM sample_values2
          ) select_values,
          (SELECT @name_order := 0) select_name_order,
          (SELECT @accumulating_value := 0) select_accumulating_value,
          (SELECT @aggregated_name_column := '') select_aggregated_name_column
        ) select_accumulating_values
      ) select_for_radians
    WHERE accumulating_value_radians >= radians LIMIT 1
    )
    order by col_number separator ' ') as circle
FROM (
  SELECT
    t1.value AS col_number,
    t2.value AS row_number,
    @dx := (t1.value - (@size-1)/2) AS dx,
    @dy := ((@size-1)/2 - t2.value) AS dy,
    @abs_radians := IF(@dx = 0, PI()/2, (atan(abs(@dy/@dx)))) AS abs_radians,
    CASE
      WHEN SIGN(@dy) >= 0 AND SIGN(@dx) >= 0 THEN @abs_radians
      WHEN SIGN(@dy) >= 0 AND SIGN(@dx) <= 0 THEN PI()-@abs_radians
      WHEN SIGN(@dy) <= 0 AND SIGN(@dx) <= 0 THEN PI()+@abs_radians
      WHEN SIGN(@dy) <= 0 AND SIGN(@dx) >= 0 THEN 2*PI()-@abs_radians
    END AS radians
  FROM
    tinyint_asc t1,
    tinyint_asc t2,
    (select @size := 21) sel_size,
    (select @radius := (@size/2 - 1)) sel_radius
  WHERE
    t1.value < @size
    AND t2.value < @size) select_combinations
  GROUP BY row_number
;
+-------------------------------------------+
| circle                                    |
+-------------------------------------------+
| 3 3 3 3 3 3 3 2 2 2 2 2 2 2 2 2 2 2 2 2 2 |
| 3 3 3 3 3 3 3 3 2 2 2 2 2 2 2 2 2 2 2 2 2 |
| 3 3 3 3 3 3 3 3 2 2 2 2 2 2 2 2 2 2 2 2 2 |
| 3 3 3 3 3 3 3 3 2 2 2 2 2 2 2 2 2 2 2 2 1 |
| 3 3 3 3 3 3 3 3 3 2 2 2 2 2 2 2 2 2 2 1 1 |
| 3 3 3 3 3 3 3 3 3 2 2 2 2 2 2 2 2 1 1 1 1 |
| 3 3 3 3 3 3 3 3 3 2 2 2 2 2 2 2 1 1 1 1 1 |
| 3 3 3 3 3 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 1 |
| 3 3 3 3 3 3 3 3 3 3 2 2 2 1 1 1 1 1 1 1 1 |
| 3 3 3 3 3 3 3 3 3 3 2 2 1 1 1 1 1 1 1 1 1 |
| 3 3 3 3 3 3 3 3 3 3 2 1 1 1 1 1 1 1 1 1 1 |
| 3 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 4 4 |
| 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 4 4 4 |
| 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 |
| 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 |
| 3 3 3 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 |
| 3 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 |
| 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 |
| 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 |
| 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 |
| 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 |
+-------------------------------------------+

The interesting SQL trick is that everything goes within the GROUP_CONCAT clause. Instead of presenting the coordinate, we check on the enhanced values table, looking for the first row which has a greater angle (in radians) than the current pixel has. We then display 1, 2, etc. to denote the value.

The next step is actually very simple: instead of drawing the full square, limit to a circle!

SELECT
  group_concat(
    IF(round(sqrt(pow(col_number-(@size-1)/2, 2) + pow(row_number-(@size-1)/2, 2))) BETWEEN @radius/2 AND @radius,
    (SELECT name_order FROM
      (
      SELECT
        name_order,
        name_column,
        value_column,
        accumulating_value,
        accumulating_value/@accumulating_value AS accumulating_value_ratio,
        2*PI()*accumulating_value/@accumulating_value AS accumulating_value_radians
      FROM (
        SELECT
          name_column,
          value_column,
          @name_order := @name_order+1 AS name_order,
          @accumulating_value := @accumulating_value+value_column AS accumulating_value,
          @aggregated_name_column := CONCAT(@aggregated_name_column, name_column, ',') AS aggregated_name_column
        FROM (
          SELECT name AS name_column, value AS value_column FROM sample_values2
          ) select_values,
          (SELECT @name_order := 0) select_name_order,
          (SELECT @accumulating_value := 0) select_accumulating_value,
          (SELECT @aggregated_name_column := '') select_aggregated_name_column
        ) select_accumulating_values
      ) select_for_radians
    WHERE accumulating_value_radians >= radians LIMIT 1
    ), '-')
    order by col_number separator ' ') as circle
FROM (
  SELECT
    t1.value AS col_number,
    t2.value AS row_number,
    @dx := (t1.value - (@size-1)/2) AS dx,
    @dy := ((@size-1)/2 - t2.value) AS dy,
    @abs_radians := IF(@dx = 0, PI()/2, (atan(abs(@dy/@dx)))) AS abs_radians,
    CASE
      WHEN SIGN(@dy) >= 0 AND SIGN(@dx) >= 0 THEN @abs_radians
      WHEN SIGN(@dy) >= 0 AND SIGN(@dx) <= 0 THEN PI()-@abs_radians
      WHEN SIGN(@dy) <= 0 AND SIGN(@dx) <= 0 THEN PI()+@abs_radians
      WHEN SIGN(@dy) <= 0 AND SIGN(@dx) >= 0 THEN 2*PI()-@abs_radians
    END AS radians
  FROM
    tinyint_asc t1,
    tinyint_asc t2,
    (select @size := 21) sel_size,
    (select @radius := 7) sel_radius
  WHERE
    t1.value < @size
    AND t2.value < @size) select_combinations
  GROUP BY row_number
;
+-------------------------------------------+
| circle                                    |
+-------------------------------------------+
| - - - - - - - - - - - - - - - - - - - - - |
| - - - - - - - - - - - - - - - - - - - - - |
| - - - - - - - - - - - - - - - - - - - - - |
| - - - - - - - - 2 2 2 2 2 - - - - - - - - |
| - - - - - - 3 3 3 2 2 2 2 2 2 - - - - - - |
| - - - - - 3 3 3 3 2 2 2 2 2 2 2 - - - - - |
| - - - - 3 3 3 3 3 2 2 2 2 2 2 2 1 - - - - |
| - - - - 3 3 3 3 3 - - - 2 2 2 1 1 - - - - |
| - - - 3 3 3 3 3 - - - - - 1 1 1 1 1 - - - |
| - - - 3 3 3 3 - - - - - - - 1 1 1 1 - - - |
| - - - 3 3 3 3 - - - - - - - 1 1 1 1 - - - |
| - - - 3 3 3 3 - - - - - - - 4 4 4 4 - - - |
| - - - 3 3 3 3 3 - - - - - 4 4 4 4 4 - - - |
| - - - - 3 3 4 4 4 - - - 4 4 4 4 4 - - - - |
| - - - - 3 4 4 4 4 4 4 4 4 4 4 4 4 - - - - |
| - - - - - 4 4 4 4 4 4 4 4 4 4 4 - - - - - |
| - - - - - - 4 4 4 4 4 4 4 4 4 - - - - - - |
| - - - - - - - - 4 4 4 4 4 - - - - - - - - |
| - - - - - - - - - - - - - - - - - - - - - |
| - - - - - - - - - - - - - - - - - - - - - |
| - - - - - - - - - - - - - - - - - - - - - |
+-------------------------------------------+

That looks a lot more like a pie chart.

Part 4: doing the fancy work

We will now add (in one big step):

  • Stretching along the X-axis.
  • Condensing the spaces.
  • Coloring for the pie parts.
  • A legend.

The text in bold is the original query, and is the only thing you need to change in order to create your own pie charts.

SELECT
  group_concat(
    IF(round(sqrt(pow(col_number/@stretch-0.5-(@size-1)/2, 2) + pow(row_number-(@size-1)/2, 2))) BETWEEN @radius*2/3 AND @radius,
    (SELECT SUBSTRING(@colors, name_order, 1) FROM
      (
      SELECT
        name_order,
        name_column,
        value_column,
        accumulating_value,
        accumulating_value/@accumulating_value AS accumulating_value_ratio,
        @aggregated_data := CONCAT(@aggregated_data, name_column, ': ', value_column, ' (', ROUND(100*value_column/@accumulating_value), '%)', '|') AS aggregated_name_column,
        2*PI()*accumulating_value/@accumulating_value AS accumulating_value_radians
      FROM (
        SELECT
          name_column,
          value_column,
          @name_order := @name_order+1 AS name_order,
          @accumulating_value := @accumulating_value+value_column AS accumulating_value
        FROM (
          SELECT name AS name_column, value AS value_column FROM sample_values2 LIMIT 4
          ) select_values,
          (SELECT @name_order := 0) select_name_order,
          (SELECT @accumulating_value := 0) select_accumulating_value,
          (SELECT @aggregated_data := '') select_aggregated_name_column
        ) select_accumulating_values
      ) select_for_radians
    WHERE accumulating_value_radians >= radians LIMIT 1
    ), ' ')
    order by col_number separator '') as pie
FROM (
  SELECT
    t1.value AS col_number,
    t2.value AS row_number,
    @dx := (t1.value/@stretch - (@size-1)/2) AS dx,
    @dy := ((@size-1)/2 - t2.value) AS dy,
    @abs_radians := IF(@dx = 0, PI()/2, (atan(abs(@dy/@dx)))) AS abs_radians,
    CASE
      WHEN SIGN(@dy) >= 0 AND SIGN(@dx) >= 0 THEN @abs_radians
      WHEN SIGN(@dy) >= 0 AND SIGN(@dx) <= 0 THEN PI()-@abs_radians
      WHEN SIGN(@dy) <= 0 AND SIGN(@dx) <= 0 THEN PI()+@abs_radians
      WHEN SIGN(@dy) <= 0 AND SIGN(@dx) >= 0 THEN 2*PI()-@abs_radians
    END AS radians
  FROM
    tinyint_asc t1,
    tinyint_asc t2,
    (select @size := 23) sel_size,
    (select @radius := (@size/2 - 1)) sel_radius,
    (select @stretch := 4) sel_stretch,
    (select @colors := '#;o:X"@+-=123456789abcdef') sel_colors
  WHERE
    t1.value < @size*@stretch
    AND t2.value < @size) select_combinations
  GROUP BY row_number
UNION ALL
  SELECT
    CONCAT(
      REPEAT(SUBSTRING(@colors, value, 1), 2),
      '  ',
      SUBSTRING_INDEX(SUBSTRING_INDEX(@aggregated_data, '|', value), '|', -1)
    )
  FROM
    tinyint_asc
  WHERE
    value BETWEEN 1 AND @name_order
;
+----------------------------------------------------------------------------------------------+
| pie                                                                                          |
+----------------------------------------------------------------------------------------------+
|                                                                                              |
|                                   ;;;;;;;;;;;;;;;;;;;;;;;;;                                  |
|                          oooooooo;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;                         |
|                    ooooooooooooooo;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;                   |
|                oooooooooooooooooooo;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;               |
|             oooooooooooooooooooooooo                     ;;;;;;;;;;;;;;;;;;;;;###            |
|           oooooooooooooooooooo                                 ;;;;;;;;;###########          |
|         oooooooooooooooooo                                         ##################        |
|       ooooooooooooooooo                                               #################      |
|      ooooooooooooooooo                                                 #################     |
|      oooooooooooooooo                                                   ################     |
|     oooooooooooooooo                                                     ################    |
|      oooooooooooooooo                                                   ::::::::::::::::     |
|      ooooooooooooooooo                                                 :::::::::::::::::     |
|       ooooooooooooooooo                                               :::::::::::::::::      |
|         oooooooooooooo::::                                         ::::::::::::::::::        |
|           ooooooo:::::::::::::                                 ::::::::::::::::::::          |
|             ::::::::::::::::::::::::                     ::::::::::::::::::::::::            |
|                :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::               |
|                    :::::::::::::::::::::::::::::::::::::::::::::::::::::::                   |
|                          :::::::::::::::::::::::::::::::::::::::::::                         |
|                                   :::::::::::::::::::::::::                                  |
|                                                                                              |
| ##  red: 1 (10%)                                                                             |
| ;;  blue: 2 (20%)                                                                            |
| oo  orange: 3 (30%)                                                                          |
| ::  white: 4 (40%)                                                                           |
+----------------------------------------------------------------------------------------------+

Making the legend is by itself an interesting hack: aggregating an unknown number of rows using a session user variable, then splitting it again. Of course, this works well if we only have a small number of rows (values), as we expect in our case.

Showcase

OK. Why? Why?

Here are three charts I hope will convince the skeptic reader:

Given the sakila database, list the 8 largest tables (size in KB):

+------------------------------------------------------------------------------+
| pie_chart                                                                    |
+------------------------------------------------------------------------------+
|                                                                              |
|                            #######################                           |
|                    #######################################                   |
|               #################################################              |
|            ##################                   ##################           |
|         ###############                               ###############        |
|        ;;###########                                     #############       |
|      ;;;;;;;;;;;;;                                         #############     |
|      ;;;;;;;;;;;;                                           ############     |
|     ;;;;;;;;;;;;;                                           #############    |
|      ;;;;;;;;;;;;                                           @@@@@@@@@@@@     |
|      ;;;;;;;;;;;;;                                         """""""""""""     |
|        ;;;;;;;;;;;;;                                     XX"""""""""""       |
|         ;;;;;;;;;;;;;;;                               XXXXXXXXXXXXX""        |
|            ;;;;;;;;;;;;;;;;;;                   ::::::::::XXXXXXXX           |
|               ;;;;;;;;;;;;;;;;;;;;;;;;;;oooooooooo::::::::::::X              |
|                    ;;;;;;;;;;;;;;;;;;;;;;ooooooooooo::::::                   |
|                            ;;;;;;;;;;;;;;ooooooooo                           |
|                                                                              |
| ##  rental: 2850816 (43%)                                                    |
| ;;  payment: 2228224 (34%)                                                   |
| oo  inventory: 376832 (6%)                                                   |
| ::  film_text: 325440 (5%)                                                   |
| XX  film: 278528 (4%)                                                        |
| ""  film_actor: 278528 (4%)                                                  |
| @@  customer: 131072 (2%)                                                    |
| ++  staff: 98304 (1%)                                                        |
+------------------------------------------------------------------------------+

How much disk space does each storage engine consume (sum table size per engine)?

+------------------------------------------------------------------------------+
| pie_chart                                                                    |
+------------------------------------------------------------------------------+
|                                                                              |
|                            #######################                           |
|                    #######################################                   |
|               #################################################              |
|            ##################                   ##################           |
|         ###############                               ###############        |
|        #############                                     #############       |
|      #############                                         #############     |
|      ############                                           ############     |
|     #############                                           #############    |
|      ############                                           oooooooooooo     |
|      #############                                         ;;;;;;;;;;;oo     |
|        #############                                     ;;;;;;;;;;;;;       |
|         ###############                               ;;;;;;;;;;;;;;;        |
|            ##################                   #;;;;;;;;;;;;;;;;;           |
|               #####################################;;;;;;;;;;;;              |
|                    ###################################;;;;                   |
|                            #######################                           |
|                                                                              |
| ##  InnoDB: 1908732 (84%)                                                    |
| ;;  MyISAM: 284074 (12%)                                                     |
| oo  ARCHIVE: 84276 (4%)                                                      |
+------------------------------------------------------------------------------+

What were the most popular DMLs during the last 10 seconds?

+------------------------------------------------------------------------------+
| pie_chart                                                                    |
+------------------------------------------------------------------------------+
|                                                                              |
|                            #######################                           |
|                    #######################################                   |
|               #################################################              |
|            ##################                   ##################           |
|         ###############                               ###############        |
|        #############                                     #############       |
|      #############                                         #############     |
|      ############                                           ############     |
|     #############                                           #############    |
|      ############                                           oooo::::::::     |
|      #############                                         ooooooooooooo     |
|        #############                                     ooooooooooooo       |
|         ###############                               ooooooooooooooo        |
|            #################;                   ;;;;;;;;;ooooooooo           |
|               #############;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;oo              |
|                    ######;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;                   |
|                            ;;;;;;;;;;;;;;;;;;;;;;;                           |
|                                                                              |
| ##  com_select: 1876 (69%)                                                   |
| ;;  com_insert: 514 (19%)                                                    |
| oo  com_delete: 277 (10%)                                                    |
| ::  com_update: 63 (2%)                                                      |
+------------------------------------------------------------------------------+

Conclusion

ASCII graphics always look funny; some would say the same about top, wget,  cal, etc. (should I even mention lynx?)

I think it is possible to do most common charting with SQL: I’ve already shown how to do horizontal graphs and pie charts. Multi-column bar charts can also be worked out. These are not meant as a permanent solution; but it’s good to be able to visualize some values without having to install Nagios (along with apache, php, drivers, etc.), or otherwise exporting table, copying to desktop machines, loading into OpenOffice impress, generating graphs.

Sometimes you just need an immediate overlook. This is where I find SQL charting to be useful.

Sure, there are Perl and Python solutions for that; that’s easily achieved as well. But doing it from with the MySQL client gives, in my opinion, a level of confidence: you’ll always be able to produce the graph; perl-DBD-MySQL or no perl-DBD-MySQL; Linux or Windows.

Besides, it was fun doing it.

]]> http://code.openark.org/blog/mysql/sql-pie-chart/feed 71 Auto scaling, scaled SQL graphs concluded http://code.openark.org/blog/mysql/auto-scaling-scaled-sql-graphs-concluded http://code.openark.org/blog/mysql/auto-scaling-scaled-sql-graphs-concluded#comments Tue, 28 Jul 2009 12:38:08 +0000 shlomi http://code.openark.org/blog/?p=1050 I wasn’t sure I was to go this far. After catching breath the following have been added to Generic, auto scaling, scaled SQL graphs, and these will conclude my current hacks:

  • Displaying X-axis min/max values.
  • Support for Y-axis values precision.
  • Support for pre-defined scale range.

The addition of the above makes for presentable, usable graphs. See also sample graphs at the end of this post.

Step 8: adding X-axis values

I add minimum/maximum X-scale values to the graph. What was just ordering_column before, now turns to be the x in the y = f(x) function.

SELECT
  y_scale,
  horizontal_bar
  FROM
  (
  SELECT
    @row_number := @row_number+1,
    CASE @row_number
      WHEN 1 THEN ROUND(max_value)
      WHEN (@graph_rows+1)/2 THEN ROUND((max_value+min_value)/2)
      WHEN @graph_rows THEN ROUND(min_value)
      ELSE '~'
    END AS y_scale,
    horizontal_bar,
    @bar_length := IFNULL(@bar_length, CHAR_LENGTH(horizontal_bar))
  FROM
    (SELECT @row_number := 0) AS select_row
    INNER JOIN
    (
    SELECT
      min_value,
      max_value,
      value_column,
      GROUP_CONCAT(SUBSTRING(graph_bar, tinyint_asc.value, 1) ORDER BY ordering_column SEPARATOR '') AS horizontal_bar
    FROM
      (SELECT @row_number := 0) AS select_row,
      tinyint_asc
    INNER JOIN (
      SELECT
        ordering_column,
        @min_value AS min_value,
        @max_value AS max_value,
        value_column,
        @scaled_value := CONVERT((value_column-@min_value)*(@graph_rows-1)/(@max_value-@min_value), UNSIGNED) AS scaled_value,
        CONCAT(REPEAT('-',@scaled_value),'#',REPEAT('-',(@graph_rows-1)-@scaled_value)) AS graph_bar
      FROM
        (
        SELECT
          @min_value := LEAST(IFNULL(@min_value, value_column), value_column) AS min_value,
          @max_value := GREATEST(IFNULL(@max_value, value_column), value_column) AS max_value,
          @min_range := LEAST(IFNULL(@min_range, ordering_column), ordering_column) AS min_range,
          @max_range := GREATEST(IFNULL(@max_range, ordering_column), ordering_column) AS max_range,
          ordering_column,
          value_column
        FROM
          (
            SELECT id AS ordering_column, val AS value_column
            FROM sample_values LIMIT 100
          ) AS value_select,
          (SELECT @min_value := NULL) AS select_min,
          (SELECT @max_value := NULL) AS select_max,
          (SELECT @min_range := NULL) AS select_min_range,
          (SELECT @max_range := NULL) AS select_max_range,
          (SELECT @bar_length := NULL) AS select_bar_length,
          (SELECT @graph_rows := 15) AS select_graph_rows
        ) AS select_range
      ) AS select_vertical
    WHERE
      tinyint_asc.value BETWEEN 1 AND CHAR_LENGTH(graph_bar)
    GROUP BY
      tinyint_asc.value
    ORDER BY
      tinyint_asc.value DESC
    ) AS select_horizontal
  ) AS select_horizontal_untitled
UNION ALL
SELECT '', CONCAT('v', REPEAT(':', @bar_length-2), 'v')
UNION ALL
SELECT '', CONCAT(@min_range, REPEAT(' ', @bar_length-CHAR_LENGTH(@min_range)-CHAR_LENGTH(@max_range)), @max_range)
;
+---------+------------------------------------------------------------------------------------------------------+
| y_scale | horizontal_bar                                                                                       |
+---------+------------------------------------------------------------------------------------------------------+
| 2       | ------------########-------------------------------------------------------########----------------- |
| ~       | ----------##--------###-------------------------------------------------###--------###-------------- |
| ~       | -------###-------------##---------------------------------------------##--------------##------------ |
| ~       | ------#------------------##------------------------------------------#------------------##---------- |
| ~       | ----##---------------------#---------------------------------------##---------------------#--------- |
| ~       | ---#------------------------##-----------------------------------##------------------------##------- |
| ~       | -##---------------------------#---------------------------------#----------------------------#------ |
| 1       | #------------------------------##------------------------------#------------------------------#----- |
| ~       | ---------------------------------#---------------------------##--------------------------------##--- |
| ~       | ----------------------------------##------------------------#------------------------------------#-- |
| ~       | ------------------------------------#---------------------##--------------------------------------## |
| ~       | -------------------------------------##-----------------##------------------------------------------ |
| ~       | ---------------------------------------##-------------##-------------------------------------------- |
| ~       | -----------------------------------------###-------###---------------------------------------------- |
| 0       | --------------------------------------------#######------------------------------------------------- |
|         | v::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::v |
|         | 0                                                                                                 99 |
+---------+------------------------------------------------------------------------------------------------------+

The X-axis range is [0..99], displayed below. The code does not limit us to INTEGER. We may as well have FLOAT, DATE, TIMESTAMP etc. (examples follow).

Step 9: adding precision

The 0,1,2 Y-axis values play nicely here. But if my generic query is:

SELECT id AS ordering_column, val/3 AS value_column FROM sample_values LIMIT 100

The resulting graph looks like:

+---------+------------------------------------------------------------------------------------------------------+
| y_scale | horizontal_bar                                                                                       |
+---------+------------------------------------------------------------------------------------------------------+
| 1       | ------------#########------------------------------------------------------#########---------------- |
| ~       | --------####---------###-----------------------------------------------####---------###------------- |
| ~       | ------##----------------###------------------------------------------##----------------###---------- |
| ~       | ----##---------------------##-------------------------------------###---------------------##-------- |
| ~       | -###-------------------------##---------------------------------##--------------------------##------ |
| 0       | #------------------------------##-----------------------------##------------------------------##---- |
| ~       | ---------------------------------##-------------------------##----------------------------------##-- |
| ~       | -----------------------------------##---------------------##--------------------------------------## |
| ~       | -------------------------------------###----------------##------------------------------------------ |
| ~       | ----------------------------------------###---------####-------------------------------------------- |
| 0       | -------------------------------------------#########------------------------------------------------ |
|         | v::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::v |
|         | 0                                                                                                 99 |
+---------+------------------------------------------------------------------------------------------------------+

And the 0,0,1 values are completely inappropriate. We will now introduce precision:

SELECT
  y_scale,
  horizontal_bar
  FROM
  (
  SELECT
    @row_number := @row_number+1,
    CASE @row_number
      WHEN 1 THEN ROUND(max_value * POW(10, @value_precision))/POW(10, @value_precision)
      WHEN (@graph_rows+1)/2 THEN ROUND((max_value+min_value)*POW(10, @value_precision)/2)/POW(10, @value_precision)
      WHEN @graph_rows THEN ROUND(min_value*POW(10, @value_precision))/POW(10, @value_precision)
      ELSE '~'
    END AS y_scale,
    horizontal_bar,
    @bar_length := IFNULL(@bar_length, CHAR_LENGTH(horizontal_bar))
  FROM
    (SELECT @row_number := 0) AS select_row
    INNER JOIN
    (
    SELECT
      min_value,
      max_value,
      value_column,
      GROUP_CONCAT(SUBSTRING(graph_bar, tinyint_asc.value, 1) ORDER BY ordering_column SEPARATOR '') AS horizontal_bar
    FROM
      (SELECT @row_number := 0) AS select_row,
      tinyint_asc
    INNER JOIN (
      SELECT
        ordering_column,
        @min_value AS min_value,
        @max_value AS max_value,
        value_column,
        @scaled_value := CONVERT((value_column-@min_value)*(@graph_rows-1)/(@max_value-@min_value), UNSIGNED) AS scaled_value,
        CONCAT(REPEAT('-',@scaled_value),'#',REPEAT('-',(@graph_rows-1)-@scaled_value)) AS graph_bar
      FROM
        (
        SELECT
          @min_value := LEAST(IFNULL(@min_value, value_column), value_column) AS min_value,
          @max_value := GREATEST(IFNULL(@max_value, value_column), value_column) AS max_value,
          @min_range := LEAST(IFNULL(@min_range, ordering_column), ordering_column) AS min_range,
          @max_range := GREATEST(IFNULL(@max_range, ordering_column), ordering_column) AS max_range,
          ordering_column,
          value_column
        FROM
          (
            SELECT id AS ordering_column, val/3 AS value_column
            FROM sample_values LIMIT 100
          ) AS value_select,
          (SELECT @min_value := NULL) AS select_min,
          (SELECT @max_value := NULL) AS select_max,
          (SELECT @min_range := NULL) AS select_min_range,
          (SELECT @max_range := NULL) AS select_max_range,
          (SELECT @bar_length := NULL) AS select_bar_length,
          (SELECT @value_precision := 2) AS select_value_precision,
          (SELECT @graph_rows := 15) AS select_graph_rows
        ) AS select_range
      ) AS select_vertical
    WHERE
      tinyint_asc.value BETWEEN 1 AND CHAR_LENGTH(graph_bar)
    GROUP BY
      tinyint_asc.value
    ORDER BY
      tinyint_asc.value DESC
    ) AS select_horizontal
  ) AS select_horizontal_untitled
UNION ALL
SELECT '', CONCAT('v', REPEAT(':', @bar_length-2), 'v')
UNION ALL
SELECT '', CONCAT(@min_range, REPEAT(' ', @bar_length-CHAR_LENGTH(@min_range)-CHAR_LENGTH(@max_range)), @max_range)
;
+---------+------------------------------------------------------------------------------------------------------+
| y_scale | horizontal_bar                                                                                       |
+---------+------------------------------------------------------------------------------------------------------+
| 0.67    | ------------########-------------------------------------------------------########----------------- |
| ~       | ----------##--------###-------------------------------------------------###--------###-------------- |
| ~       | -------###-------------##---------------------------------------------##--------------##------------ |
| ~       | ------#------------------##------------------------------------------#------------------##---------- |
| ~       | ----##---------------------#---------------------------------------##---------------------#--------- |
| ~       | ---#------------------------##-----------------------------------##------------------------##------- |
| ~       | -##---------------------------#---------------------------------#----------------------------#------ |
| 0.33    | #------------------------------##------------------------------#------------------------------#----- |
| ~       | ---------------------------------#---------------------------##--------------------------------##--- |
| ~       | ----------------------------------##------------------------#------------------------------------#-- |
| ~       | ------------------------------------#---------------------##--------------------------------------## |
| ~       | -------------------------------------##-----------------##------------------------------------------ |
| ~       | ---------------------------------------##-------------##-------------------------------------------- |
| ~       | -----------------------------------------###-------###---------------------------------------------- |
| 0       | --------------------------------------------#######------------------------------------------------- |
|         | v::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::v |
|         | 0                                                                                                 99 |
+---------+------------------------------------------------------------------------------------------------------+

Watch the 0.33, 0.67 values generated here. This is much better.

Step 10: setting fixed scale range

When we compare two graphs, it’s best if they are scaled the same way. So far, our graphs have scaled automatically according to min/max values. What we add now is a way to predefine the range. For example, dealing with per-cent values, I may always wish to scale [0..100], regardless of min/max values.

The code below defines a scale of [-1, 1]. It automatically extends if the range is actually insufficient, so it’s safe to use. Let’s see the same last graph again:

SELECT
  y_scale,
  horizontal_bar
  FROM
  (
  SELECT
    @row_number := @row_number+1,
    CASE @row_number
      WHEN 1 THEN ROUND(max_value * POW(10, @value_precision))/POW(10, @value_precision)
      WHEN (@graph_rows+1)/2 THEN ROUND((max_value+min_value)*POW(10, @value_precision)/2)/POW(10, @value_precision)
      WHEN @graph_rows THEN ROUND(min_value*POW(10, @value_precision))/POW(10, @value_precision)
      ELSE '~'
    END AS y_scale,
    horizontal_bar,
    @bar_length := IFNULL(@bar_length, CHAR_LENGTH(horizontal_bar))
  FROM
    (SELECT @row_number := 0) AS select_row
    INNER JOIN
    (
    SELECT
      min_value,
      max_value,
      value_column,
      GROUP_CONCAT(SUBSTRING(graph_bar, tinyint_asc.value, 1) ORDER BY ordering_column SEPARATOR '') AS horizontal_bar
    FROM
      (SELECT @row_number := 0) AS select_row,
      tinyint_asc
    INNER JOIN (
      SELECT
        ordering_column,
        @min_value AS min_value,
        @max_value AS max_value,
        value_column,
        @scaled_value := CONVERT((value_column-@min_value)*(@graph_rows-1)/(@max_value-@min_value), UNSIGNED) AS scaled_value,
        CONCAT(REPEAT('-',@scaled_value),'#',REPEAT('-',(@graph_rows-1)-@scaled_value)) AS graph_bar
      FROM
        (
        SELECT
          @min_value := LEAST(IFNULL(@min_scale_value, value_column), IFNULL(@min_value, value_column), value_column) AS min_value,
          @max_value := GREATEST(IFNULL(@max_scale_value, value_column), IFNULL(@max_value, value_column), value_column) AS max_value,
          @min_range := LEAST(IFNULL(@min_range, ordering_column), ordering_column) AS min_range,
          @max_range := GREATEST(IFNULL(@max_range, ordering_column), ordering_column) AS max_range,
          ordering_column,
          value_column
        FROM
          (
            SELECT id AS ordering_column, val/3 AS value_column FROM sample_values LIMIT 100
          ) AS value_select,
          (SELECT @min_value := NULL) AS select_min,
          (SELECT @max_value := NULL) AS select_max,
          (SELECT @min_range := NULL) AS select_min_range,
          (SELECT @max_range := NULL) AS select_max_range,
          (SELECT @bar_length := NULL) AS select_bar_length,
          (SELECT @min_scale_value := -1) AS select_min_scale,
          (SELECT @max_scale_value := 1) AS select_max_scale,
          (SELECT @value_precision := 2) AS select_value_precision,
          (SELECT @graph_rows := 15) AS select_graph_rows
        ) AS select_range
      ) AS select_vertical
    WHERE
      tinyint_asc.value BETWEEN 1 AND CHAR_LENGTH(graph_bar)
    GROUP BY
      tinyint_asc.value
    ORDER BY
      tinyint_asc.value DESC
    ) AS select_horizontal
  ) AS select_horizontal_untitled
UNION ALL
SELECT '', CONCAT('v', REPEAT(':', @bar_length-2), 'v')
UNION ALL
SELECT '', CONCAT(@min_range, REPEAT(' ', @bar_length-CHAR_LENGTH(@min_range)-CHAR_LENGTH(@max_range)), @max_range)
;
+---------+------------------------------------------------------------------------------------------------------+
| y_scale | horizontal_bar                                                                                       |
+---------+------------------------------------------------------------------------------------------------------+
| 1       | ---------------------------------------------------------------------------------------------------- |
| ~       | ---------------------------------------------------------------------------------------------------- |
| ~       | ------------########-------------------------------------------------------########----------------- |
| ~       | ------######--------#######------------------------------------------######--------#######---------- |
| ~       | -#####---------------------####---------------------------------#####---------------------####------ |
| ~       | #------------------------------#####------------------------####------------------------------####-- |
| ~       | ------------------------------------#####-------------######--------------------------------------## |
| 0       | -----------------------------------------#############---------------------------------------------- |
| ~       | ---------------------------------------------------------------------------------------------------- |
| ~       | ---------------------------------------------------------------------------------------------------- |
| ~       | ---------------------------------------------------------------------------------------------------- |
| ~       | ---------------------------------------------------------------------------------------------------- |
| ~       | ---------------------------------------------------------------------------------------------------- |
| ~       | ---------------------------------------------------------------------------------------------------- |
| -1      | ---------------------------------------------------------------------------------------------------- |
|         | v::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::v |
|         | 0                                                                                                 99 |
+---------+------------------------------------------------------------------------------------------------------+

Values are still in range [0..0.667], but are presented on a wider scale.

Showcase

Here are some nice graphs generated, to show off.

+---------+----------------------------------------------------------------------------------+
| y_scale | horizontal_bar                                                                   |
+---------+----------------------------------------------------------------------------------+
| 14.17   | ----------------------------------------------------------------------###------- |
| ~       | ---------------------------------------------------------------------#---#------ |
| ~       | --------------------------------------------------------------------#-----#----- |
| ~       | -------------------------------------------------------##----------------------- |
| ~       | -----------------------------------------------------##--#---------#-------#---- |
| ~       | ----------------------------------------------------#-----#--------------------- |
| ~       | ---------------------------------------###-----------------#------#------------- |
| 7.09    | -------------------------------------##---#--------#------------------------#--- |
| ~       | ------------------------------------#------#----------------#------------------- |
| ~       | -----------------------####--------#--------#-----#--------------#-----------#-- |
| ~       | ---------------------##----#-----------------#---#-----------#------------------ |
| ~       | --------------------#-------##----#-----------------------------#--------------- |
| ~       | --------#####-----##----------#--#------------#-#-------------#---------------#- |
| ~       | ----####-----##--#-------------##----------------------------------------------# |
| 0       | ####-----------##------------------------------#---------------#---------------- |
|         | v::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::v |
|         | 2009-07-25 21:43:27                                          2009-07-29 06:02:27 |
+---------+----------------------------------------------------------------------------------+
+---------+---------------------------------------------------------------------------------+
| y_scale | horizontal_bar                                                                  |
+---------+---------------------------------------------------------------------------------+
| 0.99    | ##----------------------------------------------------------------------------- |
| ~       | --##--------------------------------------------------------------------------- |
| ~       | ----#-------------------------------------------------------------------------- |
| ~       | -----##------------------------------------------------------------------------ |
| ~       | -------#----------------------------------------------------------------------- |
| ~       | --------#---------------------------------------------------------------------- |
| ~       | ---------#--------------------------------------------------------------------- |
| 0.39    | ----------#-------------------------------------------------------------------- |
| ~       | -----------#------------------------------------------------------------------- |
| ~       | ------------#------------------------------------------------------------------ |
| ~       | -------------#--------------------########------------------------------------- |
| ~       | --------------#----------------###--------#####---------------################- |
| ~       | ---------------##-----------###----------------#####----######----------------# |
| ~       | -----------------##------###------------------------####----------------------- |
| -0.22   | -------------------######------------------------------------------------------ |
|         | v:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::v |
|         | 2009-07-25 21:43:27                                         2009-07-29 06:02:27 |
+---------+---------------------------------------------------------------------------------+
+---------+---------------------------------------------------------------------------------+
| y_scale | horizontal_bar                                                                  |
+---------+---------------------------------------------------------------------------------+
| -0      | --------------------######################------------------------------------- |
| ~       | ------------########----------------------#######--------------------------#### |
| ~       | --------####-------------------------------------#####-----------------####---- |
| ~       | ------##----------------------------------------------##------------###-------- |
| ~       | ----##--------------------------------------------------##---------#----------- |
| ~       | ---#------------------------------------------------------#------##------------ |
| ~       | --#--------------------------------------------------------#------------------- |
| -2.39   | -#----------------------------------------------------------#---#-------------- |
| ~       | ---------------------------------------------------------------#--------------- |
| ~       | #------------------------------------------------------------#----------------- |
| ~       | ------------------------------------------------------------------------------- |
| ~       | ------------------------------------------------------------------------------- |
| ~       | ------------------------------------------------------------------------------- |
| ~       | ------------------------------------------------------------------------------- |
| -4.78   | --------------------------------------------------------------#---------------- |
|         | v:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::v |
|         | 0                                                                          3.95 |
+---------+---------------------------------------------------------------------------------+
+---------+----------------------------------------------------------------------------------+
| y_scale | horizontal_bar                                                                   |
+---------+----------------------------------------------------------------------------------+
| 110.82  | ---------------------------------------------------oooooo----------------------- |
| ~       | ------------------------------------------------ooooooooooo--------------------- |
| ~       | ----------------------------------------------oooooooooooooo-------------------- |
| ~       | --------------------------------------------oooooooooooooooooo------------------ |
| ~       | ----------------------------------oooo----ooooooooooooooooooooo----------------- |
| ~       | o------------------------------ooooooooooooooooooooooooooooooooo---------------- |
| ~       | o----------------------------ooooooooooooooooooooooooooooooooooo---------------- |
| ~       | oo--------------------------ooooooooooooooooooooooooooooooooooooo--------------- |
| ~       | oo-------------------------ooooooooooooooooooooooooooooooooooooooo-------------- |
| ~       | ooo-----------------------ooooooooooooooooooooooooooooooooooooooooo------------- |
| ~       | oooo--------------------oooooooooooooooooooooooooooooooooooooooooooo------------ |
| ~       | oooo-------------------ooooooooooooooooooooooooooooooooooooooooooooo------------ |
| 85.93   | ooooo-----------------ooooooooooooooooooooooooooooooooooooooooooooooo----------- |
| ~       | oooooo---------------ooooooooooooooooooooooooooooooooooooooooooooooooo---------- |
| ~       | ooooooo-------------oooooooooooooooooooooooooooooooooooooooooooooooooo---------- |
| ~       | oooooooo-----------oooooooooooooooooooooooooooooooooooooooooooooooooooo--------- |
| ~       | ooooooooo--------ooooooooooooooooooooooooooooooooooooooooooooooooooooooo-------- |
| ~       | ooooooooooo----oooooooooooooooooooooooooooooooooooooooooooooooooooooooooo------- |
| ~       | ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo------- |
| ~       | oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo------ |
| ~       | ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo----- |
| ~       | oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo---- |
| ~       | ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo--- |
| ~       | oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo-- |
| 61.04   | oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo |
|         | v::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::v |
|         | 93                                                                           172 |
+---------+----------------------------------------------------------------------------------+

Why?

Why go for all this trouble? Isn’t this nothing but a cool hack?

Is this stuff useful? Practical?

I have some agenda with this, which I’ll be happy to share when comes into being. I think there are very practical uses for SQL-based graphs. I’m sure many readers can think of interesting uses, other than the ones I have in mind.

Conclusion

There are many more possibilities and featured which can be built into the graphs. I’m going to stop here, before this becomes too picky. I hope I have shown that some very nice graphs can be produced with SQL alone.

]]> http://code.openark.org/blog/mysql/auto-scaling-scaled-sql-graphs-concluded/feed 8 Generic, auto scaling, scaled SQL graphs http://code.openark.org/blog/mysql/generic-auto-scaling-scaled-sql-graphs http://code.openark.org/blog/mysql/generic-auto-scaling-scaled-sql-graphs#comments Mon, 27 Jul 2009 10:17:16 +0000 shlomi http://code.openark.org/blog/?p=1009 In Rotating SQL graphs horizontally, I have shown how to rotate an ASCII SQL graph into horizontal position.

I was dissatisfied with some parts of the solution, which I will show now how to fix:

  • I had to manually scale the graph values so as to fit nicely into screen.
  • I had to rely on hard coded scaling schemes.
  • I had to rely on hard coded column names.
  • I had no y-axis legend.

I will now present an SQL query which allows for pluggable queries, which creates self, auto scaling graphs, along with y-axis scales.

Using deeply nested subqueries, we will evolve a simple SELECT query into an elaborate graph. I will present the many steps required, followed by explanations and sample results. But in the end – the steps are unimportant. I’ll present a generic query, into which your own SELECT can be embedded, and which will provide you with the graph.

We’ll use the same example, found in graph.sql.

Recap

Following is the table data we want to use. Values represent a sinus function.

SELECT * FROM sample_values;
+-----+----------------------+
| id  | val                  |
+-----+----------------------+
|   0 |                    1 |
|   1 |     1.09983341664683 |
|   2 |     1.19866933079506 |
|   3 |     1.29552020666134 |
|   4 |     1.38941834230865 |
|   5 |      1.4794255386042 |
|   6 |     1.56464247339504 |
|   7 |     1.64421768723769 |
|   8 |     1.71735609089952 |
|   9 |     1.78332690962748 |
...
| 246 |    0.492103409609378 |
| 247 |    0.580639083926769 |
| 248 |    0.673364873895278 |
| 249 |    0.769354294072604 |
| 250 |    0.867648249902227 |
| 251 |    0.967264620669155 |
| 252 |     1.06720807252547 |
| 253 |     1.16648000353716 |
| 254 |     1.26408852138447 |
| 255 |     1.35905835402217 |
+-----+----------------------+
256 rows in set (0.00 sec)

Step 1: Generic query

The only requirements from your query is for it to be generic. It need to provide tow columns:

  • ordering_column: A column which orders the values, e.g. an AUTO_INCREMENT, some TIMESTAMP etc.
  • value_column: The values themselves.

In the above example, the generic query will be:

SELECT id AS ordering_column, val AS value_column FROM sample_values LIMIT 100

From this point on, everything else is self-computed. The above is the only thing that needs to change if you wish to provide your own graphs.

Problem: hard coded scaling

In the following example I generate a vertical graph, but I need to hard-code the scaling:

SELECT
  ordering_column,
  CONCAT(REPEAT('-',(value_column*10+1)-1),'#',REPEAT('-',22-(value_column*10+1))) AS graph_bar
FROM
  (
  SELECT id AS ordering_column, val AS value_column FROM sample_values LIMIT 100
  ) AS value_select
;
+-----------------+------------------------+
| ordering_column | graph_bar              |
+-----------------+------------------------+
|               0 | ----------#----------- |
|               1 | -----------#---------- |
|               2 | ------------#--------- |
|               3 | -------------#-------- |
|               4 | --------------#------- |
|               5 | ---------------#------ |
|               6 | ----------------#----- |
|               7 | ----------------#----- |
|               8 | -----------------#---- |
|               9 | ------------------#--- |
|              10 | ------------------#--- |
|              11 | -------------------#-- |
|              12 | -------------------#-- |
|              13 | --------------------#- |
|              14 | --------------------#- |
|              15 | --------------------#- |
|              16 | --------------------#- |

Step 2: towards auto-scaling

In this step we will find the minimum/maximum values for value_column. We will as yet do nothing with these.

SELECT
  @min_value := LEAST(@min_value, value_column) AS min_value,
  @max_value := GREATEST(@max_value, value_column) AS max_value,
  ordering_column,
  CONCAT(REPEAT('-',(value_column*10+1)-1),'#',REPEAT('-',22-(value_column*10+1))) AS graph_bar
FROM
  (
  SELECT id AS ordering_column, val AS value_column FROM sample_values LIMIT 100
  ) AS value_select,
  (SELECT @min_value := 1000000) AS select_min,
  (SELECT @max_value := -1000000) AS select_max
;
+---------------------+-----------------+-----------------+------------------------+
| min_value           | max_value       | ordering_column | graph_bar              |
+---------------------+-----------------+-----------------+------------------------+
|                   1 |               1 |               0 | ----------#----------- |
|                   1 | 1.0998334166468 |               1 | -----------#---------- |
|                   1 | 1.1986693307951 |               2 | ------------#--------- |
|                   1 | 1.2955202066613 |               3 | -------------#-------- |
|                   1 | 1.3894183423086 |               4 | --------------#------- |
|                   1 | 1.4794255386042 |               5 | ---------------#------ |
|                   1 |  1.564642473395 |               6 | ----------------#----- |
|                   1 | 1.6442176872377 |               7 | ----------------#----- |
|                   1 | 1.7173560908995 |               8 | -----------------#---- |
|                   1 | 1.7833269096275 |               9 | ------------------#--- |
|                   1 | 1.8414709848079 |              10 | ------------------#--- |
|                   1 | 1.8912073600614 |              11 | -------------------#-- |
|                   1 | 1.9320390859672 |              12 | -------------------#-- |
|                   1 | 1.9635581854172 |              13 | --------------------#- |
|                   1 | 1.9854497299885 |              14 | --------------------#- |
|                   1 |  1.997494986604 |              15 | --------------------#- |
|                   1 | 1.9995736030415 |              16 | --------------------#- |
|                   1 | 1.9995736030415 |              17 | --------------------#- |
|                   1 | 1.9995736030415 |              18 | --------------------#- |
|                   1 | 1.9995736030415 |              19 | -------------------#-- |
|                   1 | 1.9995736030415 |              20 | -------------------#-- |
|                   1 | 1.9995736030415 |              21 | -------------------#-- |
|                   1 | 1.9995736030415 |              22 | ------------------#--- |
|                   1 | 1.9995736030415 |              23 | -----------------#---- |
|                   1 | 1.9995736030415 |              24 | -----------------#---- |
|                   1 | 1.9995736030415 |              25 | ----------------#----- |
|                   1 | 1.9995736030415 |              26 | ---------------#------ |
|                   1 | 1.9995736030415 |              27 | --------------#------- |
|                   1 | 1.9995736030415 |              28 | -------------#-------- |
|                   1 | 1.9995736030415 |              29 | ------------#--------- |
|                   1 | 1.9995736030415 |              30 | -----------#---------- |
|                   1 | 1.9995736030415 |              31 | ----------#----------- |
|    0.94162585657242 | 1.9995736030415 |              32 | ---------#------------ |
|    0.84225430585675 | 1.9995736030415 |              33 | --------#------------- |
|    0.74445889797317 | 1.9995736030415 |              34 | -------#-------------- |
...

I’m using ugly values for initial min/max comparison (-100000, 1000000). I’ll soon get rid of them, don’t worry!

Step 3: formalizing min/max values

Using another sub-query, we will isolate the minimum/maximum values, and forget about the graph for the moment.

SELECT
  @min_value,
  @max_value,
  ordering_column,
  value_column
FROM
  (
  SELECT
    @min_value := LEAST(@min_value, value_column) AS min_value,
    @max_value := GREATEST(@max_value, value_column) AS max_value,
    ordering_column,
    value_column
  FROM
    (
    SELECT id AS ordering_column, val AS value_column FROM sample_values LIMIT 100
    ) AS value_select,
    (SELECT @min_value := 1000000) AS select_min,
    (SELECT @max_value := -1000000) AS select_max
  ) AS select_range
;
+---------------------+-----------------+-----------------+----------------------+
| @min_value          | @max_value      | ordering_column | value_column         |
+---------------------+-----------------+-----------------+----------------------+
| 7.6742435899169e-05 | 1.9995736030415 |               0 |                    1 |
| 7.6742435899169e-05 | 1.9995736030415 |               1 |     1.09983341664683 |
| 7.6742435899169e-05 | 1.9995736030415 |               2 |     1.19866933079506 |
| 7.6742435899169e-05 | 1.9995736030415 |               3 |     1.29552020666134 |
| 7.6742435899169e-05 | 1.9995736030415 |               4 |     1.38941834230865 |
| 7.6742435899169e-05 | 1.9995736030415 |               5 |      1.4794255386042 |
| 7.6742435899169e-05 | 1.9995736030415 |               6 |     1.56464247339504 |
| 7.6742435899169e-05 | 1.9995736030415 |               7 |     1.64421768723769 |
| 7.6742435899169e-05 | 1.9995736030415 |               8 |     1.71735609089952 |
| 7.6742435899169e-05 | 1.9995736030415 |               9 |     1.78332690962748 |
| 7.6742435899169e-05 | 1.9995736030415 |              10 |      1.8414709848079 |
| 7.6742435899169e-05 | 1.9995736030415 |              11 |     1.89120736006144 |
...

Step 4: scaling the values

In this step I’ll do two things:

  • Get rid of ugly hard coded values for min/max comparison. I’ll use the IFNULL function to check for initial conditions.
  • Introduce self scaling: I will assume a 21-rows high graph is desired, and will auto-scale the values to fit in. Don’t worry, I’ll get rid of those values later on!
SELECT
  @min_value AS min_value,
  @max_value AS max_value,
  value_column,
  @scaled_value := CONVERT((value_column-@min_value)*20/(@max_value-@min_value), UNSIGNED) AS scaled_value,
  CONCAT(REPEAT('-',@scaled_value),'#',REPEAT('-',20-@scaled_value)) AS graph_bar
FROM
  (
  SELECT
    @min_value := LEAST(IFNULL(@min_value, value_column), value_column) AS min_value,
    @max_value := GREATEST(IFNULL(@max_value, value_column), value_column) AS max_value,
    ordering_column,
    value_column
  FROM
    (
    SELECT id AS ordering_column, val AS value_column FROM sample_values LIMIT 100
    ) AS value_select,
    (SELECT @min_value := NULL) AS select_min,
    (SELECT @max_value := NULL) AS select_max
  ) AS select_range
;
+---------------------+-----------------+----------------------+--------------+-----------------------+
| min_value           | max_value       | value_column         | scaled_value | graph_bar             |
+---------------------+-----------------+----------------------+--------------+-----------------------+
| 7.6742435899169e-05 | 1.9995736030415 |                    1 |           10 | ----------#---------- |
| 7.6742435899169e-05 | 1.9995736030415 |     1.09983341664683 |           11 | -----------#--------- |
| 7.6742435899169e-05 | 1.9995736030415 |     1.19866933079506 |           12 | ------------#-------- |
| 7.6742435899169e-05 | 1.9995736030415 |     1.29552020666134 |           13 | -------------#------- |
| 7.6742435899169e-05 | 1.9995736030415 |     1.38941834230865 |           14 | --------------#------ |
| 7.6742435899169e-05 | 1.9995736030415 |      1.4794255386042 |           15 | ---------------#----- |
| 7.6742435899169e-05 | 1.9995736030415 |     1.56464247339504 |           16 | ----------------#---- |
| 7.6742435899169e-05 | 1.9995736030415 |     1.64421768723769 |           16 | ----------------#---- |
| 7.6742435899169e-05 | 1.9995736030415 |     1.71735609089952 |           17 | -----------------#--- |
| 7.6742435899169e-05 | 1.9995736030415 |     1.78332690962748 |           18 | ------------------#-- |
| 7.6742435899169e-05 | 1.9995736030415 |      1.8414709848079 |           18 | ------------------#-- |
| 7.6742435899169e-05 | 1.9995736030415 |     1.89120736006144 |           19 | -------------------#- |
| 7.6742435899169e-05 | 1.9995736030415 |     1.93203908596723 |           19 | -------------------#- |
| 7.6742435899169e-05 | 1.9995736030415 |     1.96355818541719 |           20 | --------------------# |
| 7.6742435899169e-05 | 1.9995736030415 |     1.98544972998846 |           20 | --------------------# |
| 7.6742435899169e-05 | 1.9995736030415 |     1.99749498660405 |           20 | --------------------# |
| 7.6742435899169e-05 | 1.9995736030415 |      1.9995736030415 |           20 | --------------------# |
| 7.6742435899169e-05 | 1.9995736030415 |     1.99166481045247 |           20 | --------------------# |
| 7.6742435899169e-05 | 1.9995736030415 |      1.9738476308782 |           20 | --------------------# |
| 7.6742435899169e-05 | 1.9995736030415 |     1.94630008768741 |           19 | -------------------#- |
| 7.6742435899169e-05 | 1.9995736030415 |     1.90929742682568 |           19 | -------------------#- |
| 7.6742435899169e-05 | 1.9995736030415 |     1.86320936664887 |           19 | -------------------#- |
| 7.6742435899169e-05 | 1.9995736030415 |     1.80849640381959 |           18 | ------------------#-- |
| 7.6742435899169e-05 | 1.9995736030415 |     1.74570521217672 |           17 | -----------------#--- |
| 7.6742435899169e-05 | 1.9995736030415 |     1.67546318055115 |           17 | -----------------#--- |
| 7.6742435899169e-05 | 1.9995736030415 |     1.59847214410396 |           16 | ----------------#---- |

I have now re-introduced the graph. Note that while values are in the range 0..2, the SQL query automatically scales to a 1-21 long graph bar.

Step 5: rotation

Shall we now present this as a horizontal graph?

SELECT
  GROUP_CONCAT(SUBSTRING(graph_bar, tinyint_asc.value, 1) ORDER BY ordering_column SEPARATOR '') AS `result`
FROM
  tinyint_asc
INNER JOIN (
  SELECT
    ordering_column,
    @min_value AS min_value,
    @max_value AS max_value,
    value_column,
    @scaled_value := CONVERT((value_column-@min_value)*20/(@max_value-@min_value), UNSIGNED) AS scaled_value,
    CONCAT(REPEAT('-',@scaled_value),'#',REPEAT('-',20-@scaled_value)) AS graph_bar
  FROM
    (
    SELECT
      @min_value := LEAST(IFNULL(@min_value, value_column), value_column) AS min_value,
      @max_value := GREATEST(IFNULL(@max_value, value_column), value_column) AS max_value,
      ordering_column,
      value_column
    FROM
      (
      SELECT id AS ordering_column, val AS value_column FROM sample_values LIMIT 100
      ) AS value_select,
      (SELECT @min_value := NULL) AS select_min,
      (SELECT @max_value := NULL) AS select_max
    ) AS select_range
  ) AS select_vertical
WHERE
  tinyint_asc.value BETWEEN 1 AND CHAR_LENGTH(graph_bar)
GROUP BY
  tinyint_asc.value
ORDER BY
  tinyint_asc.value DESC;
;
+------------------------------------------------------------------------------------------------------+
| result                                                                                               |
+------------------------------------------------------------------------------------------------------+
| -------------######---------------------------------------------------------######------------------ |
| -----------##------###---------------------------------------------------###------###--------------- |
| ---------##-----------#-------------------------------------------------#------------#-------------- |
| --------#--------------##---------------------------------------------##--------------##------------ |
| ------##-----------------#-------------------------------------------#------------------#----------- |
| -----#--------------------#-----------------------------------------#--------------------#---------- |
| ----#----------------------#---------------------------------------#----------------------#--------- |
| ---#------------------------#-------------------------------------#------------------------#-------- |
| --#--------------------------#-----------------------------------#--------------------------#------- |
| -#----------------------------#---------------------------------#----------------------------#------ |
| #------------------------------#-------------------------------#------------------------------#----- |
| --------------------------------#-----------------------------#--------------------------------#---- |
| ---------------------------------#---------------------------#----------------------------------#--- |
| ----------------------------------#-------------------------#------------------------------------#-- |
| -----------------------------------##----------------------#--------------------------------------#- |
| -------------------------------------#--------------------#----------------------------------------# |
| --------------------------------------#-----------------##------------------------------------------ |
| ---------------------------------------#---------------#-------------------------------------------- |
| ----------------------------------------##-----------##--------------------------------------------- |
| ------------------------------------------##-------##----------------------------------------------- |
| --------------------------------------------#######------------------------------------------------- |
+------------------------------------------------------------------------------------------------------+

We now have a self-scaling horizontal graph! Phew!

But I’m not satisfied yet.

Step 6: towards Y-axis scaling

We will now introduce min/max Y-scale values into the graph. Note the following:

  • I’ll be using a counter to count rows.
  • I’ll rely on previous knowledge of hard-coded 21 rows value (I promise, this will be fixed soon).
  • We already have min/max values. Using the two, I’ll also provide the mid-value.
  • I chose to use ROUND(). This may not be the best idea when your values are very small fractions. Remove the ROUND() if it does not fit in with your values.
SELECT
  @row_number := @row_number+1,
  CASE @row_number
    WHEN 1  THEN ROUND(max_value)
    WHEN 11 THEN ROUND((max_value+min_value)/2)
    WHEN 21 THEN ROUND(min_value)
    ELSE ''
  END AS y_scale,
  horizontal_bar
FROM
  (SELECT @row_number := 0) AS select_row
  INNER JOIN
  (
  SELECT
    min_value,
    max_value,
    value_column,
    GROUP_CONCAT(SUBSTRING(graph_bar, tinyint_asc.value, 1) ORDER BY ordering_column SEPARATOR '') AS horizontal_bar
  FROM
    (SELECT @row_number := 0) AS select_row,
    tinyint_asc
  INNER JOIN (
    SELECT
      ordering_column,
      @min_value AS min_value,
      @max_value AS max_value,
      value_column,
      @scaled_value := CONVERT((value_column-@min_value)*20/(@max_value-@min_value), UNSIGNED) AS scaled_value,
      CONCAT(REPEAT('-',@scaled_value),'#',REPEAT('-',20-@scaled_value)) AS graph_bar
    FROM
      (
      SELECT
        @min_value := LEAST(IFNULL(@min_value, value_column), value_column) AS min_value,
        @max_value := GREATEST(IFNULL(@max_value, value_column), value_column) AS max_value,
        ordering_column,
        value_column
      FROM
        (
        SELECT id AS ordering_column, val AS value_column FROM sample_values LIMIT 100
        ) AS value_select,
        (SELECT @min_value := NULL) AS select_min,
        (SELECT @max_value := NULL) AS select_max
      ) AS select_range
    ) AS select_vertical
  WHERE
    tinyint_asc.value BETWEEN 1 AND CHAR_LENGTH(graph_bar)
  GROUP BY
    tinyint_asc.value
  ORDER BY
    tinyint_asc.value DESC
  ) AS select_horizontal
;+------------------------------+---------+------------------------------------------------------------------------------------------------------+
| @row_number := @row_number+1 | y_scale | horizontal_bar                                                                                       |
+------------------------------+---------+------------------------------------------------------------------------------------------------------+
|                            1 | 2       | -------------######---------------------------------------------------------######------------------ |
|                            2 |         | -----------##------###---------------------------------------------------###------###--------------- |
|                            3 |         | ---------##-----------#-------------------------------------------------#------------#-------------- |
|                            4 |         | --------#--------------##---------------------------------------------##--------------##------------ |
|                            5 |         | ------##-----------------#-------------------------------------------#------------------#----------- |
|                            6 |         | -----#--------------------#-----------------------------------------#--------------------#---------- |
|                            7 |         | ----#----------------------#---------------------------------------#----------------------#--------- |
|                            8 |         | ---#------------------------#-------------------------------------#------------------------#-------- |
|                            9 |         | --#--------------------------#-----------------------------------#--------------------------#------- |
|                           10 |         | -#----------------------------#---------------------------------#----------------------------#------ |
|                           11 | 1       | #------------------------------#-------------------------------#------------------------------#----- |
|                           12 |         | --------------------------------#-----------------------------#--------------------------------#---- |
|                           13 |         | ---------------------------------#---------------------------#----------------------------------#--- |
|                           14 |         | ----------------------------------#-------------------------#------------------------------------#-- |
|                           15 |         | -----------------------------------##----------------------#--------------------------------------#- |
|                           16 |         | -------------------------------------#--------------------#----------------------------------------# |
|                           17 |         | --------------------------------------#-----------------##------------------------------------------ |
|                           18 |         | ---------------------------------------#---------------#-------------------------------------------- |
|                           19 |         | ----------------------------------------##-----------##--------------------------------------------- |
|                           20 |         | ------------------------------------------##-------##----------------------------------------------- |
|                           21 | 0       | --------------------------------------------#######------------------------------------------------- |
+------------------------------+---------+------------------------------------------------------------------------------------------------------+

Well, now we’re getting somewhere!

Step 7: cleaning up

In this step I’ll do the following:

  • Using yet another subquery (how many do we have already?), I’ll get rid of the counter column.
  • I’ll remove the 21-rows hard coding. There will only be one session variable set to this value; All calculations will scale themselves according to that value.
SELECT
  y_scale,
  horizontal_bar
  FROM
  (
  SELECT
    @row_number := @row_number+1,
    CASE @row_number
      WHEN 1  THEN ROUND(max_value)
      WHEN (@graph_rows+1)/2 THEN ROUND((max_value+min_value)/2)
      WHEN @graph_rows THEN ROUND(min_value)
      ELSE ''
    END AS y_scale,
    horizontal_bar
  FROM
    (SELECT @row_number := 0) AS select_row
    INNER JOIN
    (
    SELECT
      min_value,
      max_value,
      value_column,
      GROUP_CONCAT(SUBSTRING(graph_bar, tinyint_asc.value, 1) ORDER BY ordering_column SEPARATOR '') AS horizontal_bar
    FROM
      (SELECT @row_number := 0) AS select_row,
      tinyint_asc
    INNER JOIN (
      SELECT
        ordering_column,
        @min_value AS min_value,
        @max_value AS max_value,
        value_column,
        @scaled_value := CONVERT((value_column-@min_value)*(@graph_rows-1)/(@max_value-@min_value), UNSIGNED) AS scaled_value,
        CONCAT(REPEAT('-',@scaled_value),'#',REPEAT('-',(@graph_rows-1)-@scaled_value)) AS graph_bar
      FROM
        (
        SELECT
          @min_value := LEAST(IFNULL(@min_value, value_column), value_column) AS min_value,
          @max_value := GREATEST(IFNULL(@max_value, value_column), value_column) AS max_value,
          ordering_column,
          value_column
        FROM
          (
            SELECT id AS ordering_column, val AS value_column
            FROM sample_values LIMIT 100
          ) AS value_select,
          (SELECT @min_value := NULL) AS select_min,
          (SELECT @max_value := NULL) AS select_max,
          (SELECT @graph_rows := 21) AS select_graph_rows
        ) AS select_range
      ) AS select_vertical
    WHERE
      tinyint_asc.value BETWEEN 1 AND CHAR_LENGTH(graph_bar)
    GROUP BY
      tinyint_asc.value
    ORDER BY
      tinyint_asc.value DESC
    ) AS select_horizontal
  ) AS select_horizontal_untitled
;
+---------+------------------------------------------------------------------------------------------------------+
| y_scale | horizontal_bar                                                                                       |
+---------+------------------------------------------------------------------------------------------------------+
| 2       | -------------######---------------------------------------------------------######------------------ |
|         | -----------##------###---------------------------------------------------###------###--------------- |
|         | ---------##-----------#-------------------------------------------------#------------#-------------- |
|         | --------#--------------##---------------------------------------------##--------------##------------ |
|         | ------##-----------------#-------------------------------------------#------------------#----------- |
|         | -----#--------------------#-----------------------------------------#--------------------#---------- |
|         | ----#----------------------#---------------------------------------#----------------------#--------- |
|         | ---#------------------------#-------------------------------------#------------------------#-------- |
|         | --#--------------------------#-----------------------------------#--------------------------#------- |
|         | -#----------------------------#---------------------------------#----------------------------#------ |
| 1       | #------------------------------#-------------------------------#------------------------------#----- |
|         | --------------------------------#-----------------------------#--------------------------------#---- |
|         | ---------------------------------#---------------------------#----------------------------------#--- |
|         | ----------------------------------#-------------------------#------------------------------------#-- |
|         | -----------------------------------##----------------------#--------------------------------------#- |
|         | -------------------------------------#--------------------#----------------------------------------# |
|         | --------------------------------------#-----------------##------------------------------------------ |
|         | ---------------------------------------#---------------#-------------------------------------------- |
|         | ----------------------------------------##-----------##--------------------------------------------- |
|         | ------------------------------------------##-------##----------------------------------------------- |
| 0       | --------------------------------------------#######------------------------------------------------- |
+---------+------------------------------------------------------------------------------------------------------+

And that’s as far as we go in our example.

Proof of concept

Just to prove my point, I’ll present another SQL graph. This time the function is x*sin(x), which has a nice curve.

  • Take a look at the bold text in the query. This is the only thing that changes. Replace this with your generic query, and you’re done!
  • I’ve also set the number of rows to be 11. Also find bold text in query. Change this to scale the graph!

Note how the graph automatically scales to the range [-17,16].

SELECT
  y_scale,
  horizontal_bar
  FROM
  (
  SELECT
    @row_number := @row_number+1,
    CASE @row_number
      WHEN 1  THEN ROUND(max_value)
      WHEN (@graph_rows+1)/2 THEN ROUND((max_value+min_value)/2)
      WHEN @graph_rows THEN ROUND(min_value)
      ELSE ''
    END AS y_scale,
    horizontal_bar
  FROM
    (SELECT @row_number := 0) AS select_row
    INNER JOIN
    (
    SELECT
      min_value,
      max_value,
      value_column,
      GROUP_CONCAT(SUBSTRING(graph_bar, tinyint_asc.value, 1) ORDER BY ordering_column SEPARATOR '') AS horizontal_bar
    FROM
      (SELECT @row_number := 0) AS select_row,
      tinyint_asc
    INNER JOIN (
      SELECT
        ordering_column,
        @min_value AS min_value,
        @max_value AS max_value,
        value_column,
        @scaled_value := CONVERT((value_column-@min_value)*(@graph_rows-1)/(@max_value-@min_value), UNSIGNED) AS scaled_value,
        CONCAT(REPEAT('-',@scaled_value),'#',REPEAT('-',(@graph_rows-1)-@scaled_value)) AS graph_bar
      FROM
        (
        SELECT
          @min_value := LEAST(IFNULL(@min_value, value_column), value_column) AS min_value,
          @max_value := GREATEST(IFNULL(@max_value, value_column), value_column) AS max_value,
          ordering_column,
          value_column
        FROM
          (
            SELECT id AS ordering_column, SIN(id/5)*id/5 AS value_column FROM sample_values LIMIT 100
          ) AS value_select,
          (SELECT @min_value := NULL) AS select_min,
          (SELECT @max_value := NULL) AS select_max,
          (SELECT @graph_rows := 11) AS select_graph_rows
        ) AS select_range
      ) AS select_vertical
    WHERE
      tinyint_asc.value BETWEEN 1 AND CHAR_LENGTH(graph_bar)
    GROUP BY
      tinyint_asc.value
    ORDER BY
      tinyint_asc.value DESC
    ) AS select_horizontal
  ) AS select_horizontal_untitled
;
+---------+------------------------------------------------------------------------------------------------------+
| y_scale | horizontal_bar                                                                                       |
+---------+------------------------------------------------------------------------------------------------------+
| 16      | ---------------------------------------------------------------------------------------------------# |
|         | --------------------------------------------------------------------#######-----------------------#- |
|         | ---------------------------------------##-------------------------##-------#---------------------#-- |
|         | -----------------------------------####--####--------------------#----------##------------------#--- |
|         | ------########-------------------##----------##-----------------#-------------#----------------#---- |
| -1      | ######--------#####-----------###--------------##-------------##---------------#--------------#----- |
|         | -------------------###########-------------------##----------#------------------#------------#------ |
|         | ---------------------------------------------------##------##--------------------#----------#------- |
|         | -----------------------------------------------------######-----------------------#--------#-------- |
|         | -----------------------------------------------------------------------------------##----##--------- |
| -17     | -------------------------------------------------------------------------------------####----------- |
+---------+------------------------------------------------------------------------------------------------------

Conclusion

There goes my Saturday afternoon nap.

To be done:

  • Present X-axis values.
  • Do better work with value precision.
  • Present proper titles. This is easy with session variables. I’ll make a draft.

To be continued… (and concluded, I don’t mean to drag this much longer).

[Continues on: Auto scaling, scaled SQL graphs concluded]

]]> http://code.openark.org/blog/mysql/generic-auto-scaling-scaled-sql-graphs/feed 9 Rotating SQL graphs horizontally http://code.openark.org/blog/mysql/rotating-sql-graphs-horizontally http://code.openark.org/blog/mysql/rotating-sql-graphs-horizontally#comments Fri, 24 Jul 2009 04:00:12 +0000 shlomi http://code.openark.org/blog/?p=984 We all love graphs. We all love SQL hacks. We all know the SQL hack which displays a character-based graph (example follows for those unfamiliar).

But we all love horizontal graphs, not vertical ones. We are used to the X axis being horizontal, Y being vertical. Not vice versa.

In this post I’ll present a SQL hack which rotates a vertical graph to horizontal. In fact, the technique shown will rotate any ‘textual image’; but graphs are a nice example.

A vertical graph example

What’s prettier than a sinus curve? I have prepared a simple table that will serve nicely, and can be found in graph_tables.sql.

SELECT * FROM sample_values;
+-----+----------------------+
| id  | val                  |
+-----+----------------------+
|   0 |                    1 |
|   1 |     1.09983341664683 |
|   2 |     1.19866933079506 |
|   3 |     1.29552020666134 |
|   4 |     1.38941834230865 |
|   5 |      1.4794255386042 |
|   6 |     1.56464247339504 |
|   7 |     1.64421768723769 |
|   8 |     1.71735609089952 |
|   9 |     1.78332690962748 |
...
| 246 |    0.492103409609378 |
| 247 |    0.580639083926769 |
| 248 |    0.673364873895278 |
| 249 |    0.769354294072604 |
| 250 |    0.867648249902227 |
| 251 |    0.967264620669155 |
| 252 |     1.06720807252547 |
| 253 |     1.16648000353716 |
| 254 |     1.26408852138447 |
| 255 |     1.35905835402217 |
+-----+----------------------+
256 rows in set (0.00 sec)

The common trick for displaying a graph is something like:

SELECT
  id,
  CONCAT(REPEAT('-',(val*10+1)-1),'#',REPEAT('-',22-(val*10+1))) AS bar
FROM
  sample_values
LIMIT 100
+----+------------------------+
| id | bar                    |
+----+------------------------+
|  0 | ----------#----------- |
|  1 | -----------#---------- |
|  2 | ------------#--------- |
|  3 | -------------#-------- |
|  4 | --------------#------- |
|  5 | ---------------#------ |
|  6 | ----------------#----- |
|  7 | ----------------#----- |
|  8 | -----------------#---- |
|  9 | ------------------#--- |
| 10 | ------------------#--- |
| 11 | -------------------#-- |
| 12 | -------------------#-- |
| 13 | --------------------#- |
| 14 | --------------------#- |
| 15 | --------------------#- |
| 16 | --------------------#- |
| 17 | --------------------#- |
| 18 | --------------------#- |
| 19 | -------------------#-- |
| 20 | -------------------#-- |
| 21 | -------------------#-- |
| 22 | ------------------#--- |
| 23 | -----------------#---- |
| 24 | -----------------#---- |
| 25 | ----------------#----- |
| 26 | ---------------#------ |
| 27 | --------------#------- |
...
+----+------------------------+
100 rows in set (0.00 sec)

In the above I’ve explicitly stretched values for them to be presentable.

So that’s the vertical graph. It’s not easy to read, plus you need to scroll down on your terminal to find out what’s going on. Can we rotate it?

Rotating the graph

We will combine the String Walking technique with String Unwalking. What we need and assume is as follows:

  • The above graph pads the bars up to some fixed length.
  • Said length is not too high, so as to fit nicely within our screen.
  • We have an integers table, used for string walking.

The idea is to iterate the textual column character by character from end to start (done by string walking), and convert each such character column to row (done by string unwalking).

SELECT
  GROUP_CONCAT(SUBSTRING(bar,tinyint_asc.value,1) ORDER BY id SEPARATOR '') AS `Sinus graph`
FROM
  tinyint_asc
INNER JOIN (
  SELECT
    id,
    CONCAT(REPEAT('-',(val*10+1)-1),'#',REPEAT('-',22-(val*10+1))) AS bar
  FROM
    sample_values
  LIMIT 100) sel_graph
WHERE
  tinyint_asc.value BETWEEN 1 AND CHAR_LENGTH(bar)
GROUP BY
  tinyint_asc.value
ORDER BY
  tinyint_asc.value DESC;

+------------------------------------------------------------------------------------------------------+
| Sinus graph                                                                                          |
+------------------------------------------------------------------------------------------------------+
| ---------------------------------------------------------------------------------------------------- |
| -------------######---------------------------------------------------------######------------------ |
| -----------##------###---------------------------------------------------###------###--------------- |
| ---------##-----------#-------------------------------------------------#------------#-------------- |
| --------#--------------##---------------------------------------------##--------------##------------ |
| ------##-----------------#-------------------------------------------#------------------#----------- |
| -----#--------------------#-----------------------------------------#--------------------#---------- |
| ----#----------------------#---------------------------------------#----------------------#--------- |
| ---#------------------------#-------------------------------------#------------------------#-------- |
| --#--------------------------#-----------------------------------#--------------------------#------- |
| -#----------------------------#---------------------------------#----------------------------#------ |
| #------------------------------#-------------------------------#------------------------------#----- |
| --------------------------------#-----------------------------#--------------------------------#---- |
| ---------------------------------#---------------------------#----------------------------------#--- |
| ----------------------------------#-------------------------#------------------------------------#-- |
| -----------------------------------##----------------------#--------------------------------------#- |
| -------------------------------------#--------------------#----------------------------------------# |
| --------------------------------------#-----------------##------------------------------------------ |
| ---------------------------------------#---------------#-------------------------------------------- |
| ----------------------------------------##-----------##--------------------------------------------- |
| ------------------------------------------##-------##----------------------------------------------- |
| --------------------------------------------#######------------------------------------------------- |
+------------------------------------------------------------------------------------------------------+

Now that’s much prettier, isn’t it?

Crazy stuff challenge

For those eager to improve upon this, I raise a few challenges (no prize but fame, though). Use the given data (graph_tables.sql), to produce:

  • (Vertical) values on the X-axis for all graph points.
  • Min/max values for Y-axis, on separate column.
  • As I understand it, full, in place, values for Y-axis is not possible; please prove me wrong!
  • Other crazy stuff?

Please add solutions in comments!

[Continues on: Generic, auto scaling, scaled SQL graphs]

]]> http://code.openark.org/blog/mysql/rotating-sql-graphs-horizontally/feed 9