{"id":6320,"date":"2013-05-02T07:14:55","date_gmt":"2013-05-02T05:14:55","guid":{"rendered":"http:\/\/code.openark.org\/blog\/?p=6320"},"modified":"2013-05-02T10:53:37","modified_gmt":"2013-05-02T08:53:37","slug":"on-compiling-tokudb-from-source","status":"publish","type":"post","link":"https:\/\/code.openark.org\/blog\/mysql\/on-compiling-tokudb-from-source","title":{"rendered":"On compiling TokuDB from source"},"content":{"rendered":"<p>Sharing my experience of compiling TokuDB + MariaDB <strong>5.5<\/strong>. Why? Because I must have <a href=\"http:\/\/code.openark.org\/blog\/mysql\/sphinx-sphinx_snippets-mysql-5-5\">this patch<\/a> to Sphinx <strong>2.0.4<\/strong>.<\/p>\n<p><strong>Note<\/strong>: I was using what seems to be the &#8220;old&#8221; method of compiling; quoting Leif Walsh:<\/p>\n<blockquote><p>&#8230; We are looking at deprecating that method of building (MariaDB source plus binary fractal tree handlerton). \u00a0It only really needed to be that complex when we were closed source.<\/p><\/blockquote>\n<p>I also tried the &#8220;new&#8221; method of compiling, which I couldn&#8217;t work out.<\/p>\n<p>Here&#8217;s how it goes: TokuDB is newly <a href=\"http:\/\/www.tokutek.com\/2013\/04\/announcing-tokudb-v7-open-source-and-more\/\">released as open source<\/a>. As such, it got a lot of attention, many downloads and I hope it will succeed.<\/p>\n<p>However as stable as the product may be, it&#8217;s new to open source, which means anyone compiling it from source is an early adopter (at least for the compilation process).<\/p>\n<h4>Installation process<\/h4>\n<p>This is an unorthodox, and actually weird process. See <a href=\"http:\/\/www.tokutek.com\/wp-content\/uploads\/2013\/04\/mariadb-5.5.30-tokudb-7.0.1-users-guide.pdf\">section 6 on the Tokutek docs<\/a>. In order to compile the project you must download:<\/p>\n<ul>\n<li>The source code tar.gz<\/li>\n<li><em>And<\/em> the binary (?!) tar.gz<\/li>\n<li>And the binary checksum<\/li>\n<li>And the Tokutek patches<\/li>\n<li>And the patches checksum<!--more--><\/li>\n<\/ul>\n<p>You extract the source tarball. But instead of doing the standard <strong>&#8220;.\/configure &amp;&amp; make &amp;&amp; sudo make install&#8221;<\/strong> you need to copy a shell script called\u00a0<strong>tokudb.build.bash<\/strong> <em>one directory level up<\/em>, and run it from there.<\/p>\n<p><strong>tokudb.build.bash<\/strong> lists <strong>gcc47<\/strong> and <strong>g++47<\/strong> on rows <strong>3, 4<\/strong>. Modify <strong>&#8220;gcc47&#8221;<\/strong> to <strong>&#8220;gcc&#8221;<\/strong>, modify <strong>&#8220;g++47&#8221;<\/strong> to\u00a0 <strong>&#8220;g++&#8221;<\/strong>. I&#8217;m assuming you don&#8217;t have a binary called <strong>gcc47<\/strong>. Why would you?<\/p>\n<h4>Dependencies<\/h4>\n<p>You will need <strong>CMake &gt;= 2.8<\/strong><\/p>\n<p>This means Ubuntu LTS <strong>10.04<\/strong> users are unable to compile out of the box; will need to manually install later version of CMake.<\/p>\n<p>Also needed is <strong>zlib1g-dev<\/strong>, <strong>rpmbuild<\/strong>.<\/p>\n<h4>While compiling<\/h4>\n<p>I ran out of disk space. <em>What?<\/em> I was using a <strong>10G<\/strong> partition I use for my compilations. Looking at <strong>&#8220;df -h&#8221;<\/strong> I get that:<\/p>\n<ul>\n<li>The source tarball is extracted (I did it)<\/li>\n<li>The binary tarball is also extracted (someone has to explain this for me)<\/li>\n<li>And inside the source directory we have:<\/li>\n<\/ul>\n<blockquote>\n<pre>bash$ df -h\r\n...\r\n1484\u00a0\u00a0\u00a0 build.RelWithDebInfo.rpms\r\n5540\u00a0\u00a0\u00a0 build.RelWithDebInfo<\/pre>\n<\/blockquote>\n<p>At about <strong>7GB<\/strong> (and counting) of build&#8230; <em>stuff?<\/em>.<\/p>\n<p><strong>UPDATE<\/strong>: just ran out on disk space <em>again<\/em>. Is this an incremental thing? Like every time my compilation fails and I recompile some files are not cleaned up? If so, put them on <strong>\/tmp<\/strong>! OK, moving everything to a <strong>300GB<\/strong> partition and starting all over.<\/p>\n<h4>More while compiling<\/h4>\n<p><strong><\/strong>I got errors on missing libraries. Like I was missing <strong>libssl<\/strong>, <strong>rpmbuild<\/strong>. This is what the <strong>&#8220;configure&#8221;<\/strong> script is for &#8212; to test for dependencies. It&#8217;s really a bummer to have to recompile 4-5 times (and it&#8217;s a long compilation), only to find out there&#8217;s another missing package.<\/p>\n<h4>After compiling<\/h4>\n<p>What is the result of the compilation? Not a &#8220;make install&#8221; prepared binary. The result is a MySQL-binary package. Se need to extract and put on <strong>\/usr\/local\/somewhere<\/strong> etc.<\/p>\n<h4>Conclusions<\/h4>\n<p>The compilation process is unexpected and non-standard. The output is unexpected.<\/p>\n<p>The correct way of doing this is a <strong>&#8220;.\/configure &amp;&amp; make &amp;&amp; sudo make install&#8221;<\/strong>. I don&#8217;t understand the need for a binary package while compiling from source. Isn&#8217;t this the chicken and the egg?<\/p>\n<p>A source distribution is no different from a binary distribution. You must have a testing environment to verify the source distribution actually works. This test environment is typically a bare-new-RedHat or a bare-new-Ubuntu etc. The machines at Tokutek are already installed with needed packages. Not so on my compilation machine. I suggest that <strong>apt-get<\/strong>s and <strong>yum install<\/strong>s for dependencies are added to the source distribution testing. This is the only reliable way for you guys at Tokutek to know that clients will actually be able to install via source.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Sharing my experience of compiling TokuDB + MariaDB 5.5. Why? Because I must have this patch to Sphinx 2.0.4. Note: I was using what seems to be the &#8220;old&#8221; method of compiling; quoting Leif Walsh: &#8230; We are looking at deprecating that method of building (MariaDB source plus binary fractal tree handlerton). \u00a0It only really [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"enabled":false},"version":2}},"categories":[5],"tags":[102],"class_list":["post-6320","post","type-post","status-publish","format-standard","hentry","category-mysql","tag-tokudb"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p2bZZp-1DW","_links":{"self":[{"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/posts\/6320","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/comments?post=6320"}],"version-history":[{"count":12,"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/posts\/6320\/revisions"}],"predecessor-version":[{"id":6331,"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/posts\/6320\/revisions\/6331"}],"wp:attachment":[{"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/media?parent=6320"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/categories?post=6320"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/tags?post=6320"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}