{"id":7632,"date":"2016-09-06T11:44:14","date_gmt":"2016-09-06T09:44:14","guid":{"rendered":"http:\/\/code.openark.org\/blog\/?p=7632"},"modified":"2016-09-06T11:44:36","modified_gmt":"2016-09-06T09:44:36","slug":"gh-ost-1-0-17-hooks-sub-second-lag-control-amazon-rds-and-more","status":"publish","type":"post","link":"https:\/\/code.openark.org\/blog\/mysql\/gh-ost-1-0-17-hooks-sub-second-lag-control-amazon-rds-and-more","title":{"rendered":"gh-ost 1.0.17: Hooks, Sub-second lag control, Amazon RDS and more"},"content":{"rendered":"

gh-ost<\/a> version 1.0.17<\/a> is now released, with various additions and fixes. Here are some notes of interest:<\/p>\n

Hooks<\/h3>\n

gh-ost<\/code> now supports hooks<\/a>. These are your own executables that gh-ost<\/code> will invoke at particular points of interest (validation pass, about to cut-over, success, failure, status, etc.)<\/p>\n

gh-ost<\/code> will set various environment variables for your executables to pick up, passing along such information as migrated\/ghost<\/em> table name, elapsed time, processed rows, migrated host etc.<\/p>\n

Sub-second lag control<\/h3>\n

At GitHub we’re very strict about replication lag. We keep it well under 1<\/code> second at most times. gh-ost<\/code> can now identify sub-second lag on replicas<\/a> (well, you need to supply with the right query). Our current production migrations are set by default with --max-lag-millis=500<\/code> or less, and our most intensive migrations keep replication lag well below 1sec<\/code> or even below 500ms<\/code><\/p>\n

No SUPER<\/h3>\n

The SUPER<\/code> privilege is required to set global binlog_format='ROW'<\/code> and for STOP SLAVE; START SLAVE;<\/code><\/p>\n

If you know<\/em> your replica has RBR, you can pass --assume-rbr<\/code> and skips those steps.<\/p>\n

RDS<\/h3>\n

Hooks + No Super = RDS, as seems to be the case. For --test-on-replica<\/code> you will need to supply your own gh-ost-on-stop-replication<\/code> hook, to stop your RDS replica at cut-over phase. See this tracking issue<\/a><\/p>\n

master-master<\/h3>\n

While active-active are still not supported, you now have greater control over master-master topologies by being able to explicitly pick your master (as gh-ost<\/code> arbitrarily picks one of the co-masters). Do so by passing --assume-master-host<\/code>. See cheatsheet<\/a>.<\/p>\n

tungsten replicator<\/h3>\n

Similarly, gh-ost<\/code> cannot crawl your tungsten<\/code> topology, and you are able to specify --tungsten --assume-master-host=the.master.com<\/code>. See cheatsheet<\/a>.<\/p>\n

Concurrent-rowcount<\/h3>\n

--exact-rowcount<\/code> is awesomeness, keeping quite accurate estimate of progress. With --concurrent-rowcount<\/code> we begin migration with a rough estimate, and execute select count(*) from your_table<\/code> in parallel, updating our estimate later on throughout the migration<\/p>\n

Stricter, safer<\/h3>\n

gh-ost<\/code> works in STRICT_ALL_TABLES<\/code> mode, meaning it would fail rather than set the wrong value to a column.<\/p>\n

In addition to unit-testing and production continuous test, a set of local tests<\/a> is growing, hopefully to run as CI tests later on.<\/p>\n

Fixed problems<\/h3>\n

Fixed time_zone<\/code> related bug, high unsigned<\/code> values bug; added strict check for triggers, relaxed config file parsing, and more. Thank you to community contributors for PRs, from ipv6<\/code> to typos!<\/p>\n

Known issues<\/h3>\n

Issues coming and going at all times — thank you for reporting Issues!<\/p>\n

We have a confirmed bug with non-UTF charsets<\/a> at this time. Some other minor issues and feature requests are open — we’ll take them as we go along.<\/p>\n

Feedback requests<\/h3>\n

We are not testing gh-ost<\/code> on RDS ourselves. We appreciate community feedback on this tracking issue<\/a>.<\/p>\n

We are not testing gh-ost<\/code> on Galera\/XtraDB cluster ourselves. We appreciate community feedback on this tracking issue<\/a>.<\/p>\n

We value submitted Issues and questions.<\/p>\n

Speaking<\/h3>\n

We will be presenting gh-ost<\/code> in the next month:<\/p>\n