{"id":3304,"date":"2011-02-15T10:01:15","date_gmt":"2011-02-15T08:01:15","guid":{"rendered":"http:\/\/code.openark.org\/blog\/?p=3304"},"modified":"2011-02-15T10:01:15","modified_gmt":"2011-02-15T08:01:15","slug":"upgrading-to-barracuda-getting-rid-of-huge-ibdata1-file","status":"publish","type":"post","link":"https:\/\/code.openark.org\/blog\/mysql\/upgrading-to-barracuda-getting-rid-of-huge-ibdata1-file","title":{"rendered":"Upgrading to Barracuda &#038; getting rid of huge ibdata1 file"},"content":{"rendered":"<p>Some of this is old stuff, but more people are now converting to InnoDB plugin, so as to enjoy table compression, performance boosts. Same holds for people converting to Percona&#8217;s XtraDB. InnoDB plugin requires <strong>innodb_file_per_table<\/strong>. No more shared tablespace file.<\/p>\n<p>So your <strong>ibdata1<\/strong> file is some <strong>150GB<\/strong>, and it won&#8217;t reduce. Really, it won&#8217;t reduce. You set <strong>innodb_file_per_table=1<\/strong>, do <strong>ALTER TABLE t ENGINE=InnoDB<\/strong> (optionally <strong>ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=8<\/strong>), and you get all your tables in file-per-table <strong>.ibd<\/strong> files.<\/p>\n<p>But the original <strong>ibdata1<\/strong> file is still there. It has to be there, don&#8217;t delete it! It contains more than your old data.<\/p>\n<p>InnoDB tablespace files never reduce in size, it&#8217;s an old-time annoyance. The only way to go round it, if you need the space, is to completely drop them and start afresh. That&#8217;s one of the things so nice about file-per-table: an <strong>ALTER TABLE<\/strong> actually creates a new tablespace file and drops the original one.<\/p>\n<h4>The procedure<\/h4>\n<p>The procedure is somewhat painful:<\/p>\n<ul>\n<li>Dump everything logically (either use <em>mysqldump<\/em>, <a href=\"http:\/\/www.maatkit.org\/doc\/mk-parallel-dump.html\">mk-parallel-dump<\/a>, or do it your own way)<\/li>\n<li>Erase your data (literally, delete everything under your <strong>datadir<\/strong>)<\/li>\n<li>Generate a new empty database<\/li>\n<li>Load your dumped data.<!--more--><\/li>\n<\/ul>\n<h4>Using replication<\/h4>\n<p>Replication makes this less painful. Set up a slave, have it follow up on the master.<\/p>\n<ul>\n<li>Stop your slave.<\/li>\n<li>Make sure to backup the replication position (e.g. write <strong>SHOW SLAVE STATUS<\/strong> on a safe location, or copy <strong>master.info<\/strong> file).<\/li>\n<li>Work out the dump-erase-generate-load steps on the slave.<\/li>\n<li>Reattach the slave to the master using saved data.<\/li>\n<\/ul>\n<p>For this to succeed you must keep enough binary logs on the master for the entire dump-load period, which could be lengthy.<\/p>\n<h4>Upgrading to barracuda<\/h4>\n<p>If you wish to upgrade your InnoDB tables to <em>Barracuda<\/em> format, my advice is this:<\/p>\n<ol>\n<li>Follow the steps above to generate a file-per-table working slave<\/li>\n<li>Stop the slave<\/li>\n<li>Configure <strong>skip_slave_start<\/strong><\/li>\n<li>Restart MySQL<\/li>\n<li>One by one do the <strong>ALTER TABLE<\/strong> into <em>Barracuda<\/em> format (<strong>ROW_FORMAT=COMPACT<\/strong> or <strong>ROW_FORMAT=COMPRESSED<\/strong>)<\/li>\n<\/ol>\n<p>Note that if you&#8217;re about to do table compression, the <strong>ALTER<\/strong> statements become <em>considerably<\/em> slower the better the compression is.<\/p>\n<p>If your dataset is very large, and you can&#8217;t keep so many binary logs, you may wish to break step <strong>5<\/strong> above into:<\/p>\n<ul>\n<li>ALTER a large table<\/li>\n<li>Restart MySQL<\/li>\n<li>Start slave, wait for it to catch up<\/li>\n<li>Restart MySQL again<\/li>\n<\/ul>\n<p>and do the same for all large tables.<\/p>\n<h4>Why all these restarts?<\/h4>\n<p>I&#8217;ve been upgrading to Barracuda for a long time now. I have clearly noticed that <strong>ALTER<\/strong> into a <strong>COMPRESSED<\/strong> format works considerably slower after the slave has done some &#8220;real work&#8221;. This in particular relates to the last &#8220;renaming table&#8221; stage. There was a bug with earlier InnoDB plugin versions which made this stage hang. It was solved. But it still takes some time for this last, weird stage, where the new replacement table is complete, and it&#8217;s actually been renamed in place of the old table, and the old table renamed into something like &#8220;#sql-12345.ibd&#8221;, and all that needs to be done is have it dropped, and&#8230; Well, it takes time.<\/p>\n<p>My observation is it works faster on a freshly started server. Which is why I take the bother to restart MySQL before each large table conversion.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Some of this is old stuff, but more people are now converting to InnoDB plugin, so as to enjoy table compression, performance boosts. Same holds for people converting to Percona&#8217;s XtraDB. InnoDB plugin requires innodb_file_per_table. No more shared tablespace file. So your ibdata1 file is some 150GB, and it won&#8217;t reduce. Really, it won&#8217;t reduce. [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"enabled":false},"version":2}},"categories":[5],"tags":[7,11,14,9,8],"class_list":["post-3304","post","type-post","status-publish","format-standard","hentry","category-mysql","tag-backup","tag-configuration","tag-innodb","tag-mysqldump","tag-replication"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p2bZZp-Ri","_links":{"self":[{"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/posts\/3304","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/comments?post=3304"}],"version-history":[{"count":14,"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/posts\/3304\/revisions"}],"predecessor-version":[{"id":3325,"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/posts\/3304\/revisions\/3325"}],"wp:attachment":[{"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/media?parent=3304"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/categories?post=3304"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/tags?post=3304"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}