{"id":7784,"date":"2017-09-12T10:39:10","date_gmt":"2017-09-12T08:39:10","guid":{"rendered":"http:\/\/code.openark.org\/blog\/?p=7784"},"modified":"2017-09-12T11:14:41","modified_gmt":"2017-09-12T09:14:41","slug":"orchestrator-3-0-2-ga-released-raft-consensus-sqlite","status":"publish","type":"post","link":"https:\/\/code.openark.org\/blog\/mysql\/orchestrator-3-0-2-ga-released-raft-consensus-sqlite","title":{"rendered":"orchestrator 3.0.2 GA released: raft consensus, SQLite"},"content":{"rendered":"<p><strong>orchestrator<\/strong> 3.0.2 GA is <a href=\"https:\/\/github.com\/github\/orchestrator\/releases\/tag\/v3.0.2\">released<\/a> and available for download (see also <a href=\"https:\/\/packagecloud.io\/github\/orchestrator\">packagecloud repository<\/a>).<\/p>\n<p><code>3.0.2<\/code> is the first stable release in the <code>3.0*<\/code>\u00a0series, introducing (recap from <a href=\"http:\/\/code.openark.org\/blog\/mysql\/orchestratorraft-pre-release-3-0\">3.0 pre-release announcement<\/a>):<\/p>\n<h3>orchestrator\/raft<\/h3>\n<p><a href=\"https:\/\/raft.github.io\/\">Raft<\/a>\u00a0is a consensus protocol, supporting leader election and consensus across a distributed system.\u00a0 In an\u00a0<code>orchestrator\/raft<\/code>\u00a0setup\u00a0<code>orchestrator<\/code> nodes talk to each other via raft protocol, form consensus and elect a leader. Each\u00a0<code>orchestrator<\/code>\u00a0node has its own\u00a0<em>dedicated<\/em>\u00a0backend database. The backend databases do not speak to each other; only the\u00a0<code>orchestrator<\/code>\u00a0nodes speak to each other.<\/p>\n<p>No MySQL replication setup needed; the backend DBs act as standalone servers. In fact, the backend server doesn&#8217;t have to be MySQL, and\u00a0<code>SQLite<\/code>is supported.\u00a0<code>orchestrator<\/code>\u00a0now ships with\u00a0<code>SQLite<\/code>\u00a0embedded, no external dependency needed.<\/p>\n<p>For details, please refer to the documentation:<\/p>\n<ul>\n<li><a href=\"https:\/\/github.com\/github\/orchestrator\/blob\/master\/docs\/raft.md\">orchestrator\/raft: overview<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/github\/orchestrator\/blob\/master\/docs\/raft-vs-sync-repl.md\">orchestrator\/raft vs. shared backend DB setup, comparison<\/a><\/li>\n<\/ul>\n<h3>SQLite<\/h3>\n<p>Suggested and requested by many, is to remove\u00a0<code>orchestrator<\/code>&#8216;s own dependency on a MySQL backend.\u00a0<code>orchestrator<\/code>\u00a0now supports a SQLite backend.<\/p>\n<p><code>SQLite<\/code>\u00a0is a transactional, relational, embedded database, and as of\u00a0<code>3.0<\/code>\u00a0it is embedded within\u00a0<code>orchestrator<\/code>, no external dependency required.<\/p>\n<h3>orchestrator-client<\/h3>\n<p><a href=\"https:\/\/github.com\/github\/orchestrator\/blob\/master\/docs\/orchestrator-client.md\"><strong>orchestrator-client<\/strong><\/a>\u00a0is a client shell script which mimics the command line interface, while running\u00a0<code>curl | jq<\/code>\u00a0requests against the HTTP API. It stands to simplify your deployments: interacting with the orchestrator service via <code>orchestrator-client<\/code>\u00a0is easier and only requires you to place a shell script (this is as opposed to installing the <code>orchestrator<\/code>\u00a0binary + configuration file).<\/p>\n<p><code>orchestrator-client<\/code>\u00a0is the way to interact with your <code>orchestrator\/raft<\/code>\u00a0cluster. <code>orchestrator-client<\/code>\u00a0now has its own <code>RPM\/deb<\/code>\u00a0release package.<\/p>\n<p>You may still use the web interface, web API ; and a special\u00a0<code>--ignore-raft-setup<\/code>\u00a0keeps power at your hand (use at your own risk).<\/p>\n<h3>State of orchestrator\/raft<\/h3>\n<p><code>orchestrator\/raft<\/code>\u00a0is a big change:<!--more--><\/p>\n<ul>\n<li>In the way it is deployed<\/li>\n<li>In the way it is operated<\/li>\n<li>In the high availability it provides<\/li>\n<li>and more<\/li>\n<\/ul>\n<p>This is why it has been tested in production for a few months.<\/p>\n<p><code>orchestrator\/raft<\/code>\u00a0now runs in production at GitHub ; we&#8217;ve decommissioned the &#8220;old&#8221; <code>orchestrator<\/code>\u00a0setup having run both in parallel for a while. It drives our failovers and deploys over three data centers.<\/p>\n<p>We are using MySQL as backend to our <code>orchestrator<\/code>\u00a0cluster. We will introduce more staging tests for SQLite-based setups.<\/p>\n<h3>Roadmap<\/h3>\n<p>There&#8217;s much to do, and we chose to release a version that has a way to go. We expect:<\/p>\n<ul>\n<li>Dynamic raft cluster join\/leave operations (right now the cluster is static and configuration based)<\/li>\n<li>New nodes joining the cluster to auto-populate data from the cluster. This is actually a built-in feature to the <code>hashicorp\/raft<\/code>\u00a0library that we use, however we intentionally did not use this functionality, and expect to re-introduce it. At this time, adding a newly provisioned node to the cluster requires a backup\/restore or dump\/load of DB data from an existing node.<\/li>\n<li>Partitioning of probe tasks across nodes<\/li>\n<li>Various thoughts on proxy integrations<\/li>\n<li>More&#8230;<\/li>\n<\/ul>\n<p>We will focus on making operations simpler, and of course keep stability and reliability at highest priority.<\/p>\n<h3>orchestrator tutorial<\/h3>\n<p>In two weeks time I will be presenting the <a href=\"https:\/\/www.percona.com\/live\/e17\/sessions\/practical-orchestrator-tutorial\">practical orchestrator tutorial<\/a>, a 3 hour practical walkthrough on deployment, configuration, reasoning and more.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>orchestrator 3.0.2 GA is released and available for download (see also packagecloud repository). 3.0.2 is the first stable release in the 3.0*\u00a0series, introducing (recap from 3.0 pre-release announcement): orchestrator\/raft Raft\u00a0is a consensus protocol, supporting leader election and consensus across a distributed system.\u00a0 In an\u00a0orchestrator\/raft\u00a0setup\u00a0orchestrator nodes talk to each other via raft protocol, form consensus and [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"enabled":false},"version":2}},"categories":[5],"tags":[57,108],"class_list":["post-7784","post","type-post","status-publish","format-standard","hentry","category-mysql","tag-open-source","tag-orchestrator"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p2bZZp-21y","_links":{"self":[{"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/posts\/7784","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/comments?post=7784"}],"version-history":[{"count":9,"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/posts\/7784\/revisions"}],"predecessor-version":[{"id":7793,"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/posts\/7784\/revisions\/7793"}],"wp:attachment":[{"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/media?parent=7784"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/categories?post=7784"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/tags?post=7784"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}