Skip to content

# Tag: GitHub

## orchestrator: what’s new in CI, testing & development

MySQL dbdeployer docker GitHub Open Source orchestrator testing May 11, 2020May 12, 2020

Recent focus on development & testing yielded with new orchestrator environments and offerings for developers and with increased reliability and trust. This post illustrates the new changes, and see Developers section on the official documentation for more details.

## Testing

In the past four years `orchestrator` was developed at GitHub, and using GitHub’s environments for testing. This is very useful for testing `orchestrator`‘s behavior within GitHub, interacting with its internal infrastructure, and validating failover behavior in a production environment. These tests and their results are not visible to the public, though.

Now that `orchestrator` is developed outside GitHub (that is, outside GitHub the company, not GitHub the platform) I wanted to improve on the testing framework, making it visible, accessible and contribute-able to the community. Thankfully, the GitHub platform has much to offer on that front and `orchestrator` now uses GitHub Actions more heavily for testing.

GitHub Actions provide a way to run code in a container in the context of the repository. The most common use case is to run CI tests on receiving a Pull Request. Indeed, when GitHub Actions became available, we switched out of Travis CI and into Actions for `orchestrator`‘s CI.

Today, `orchestrator` runs three different tests:

- Build, unit testing, integration testing, code & doc validation
- Upgrade testing
- System testing

To highlight what each does: Continue reading » “orchestrator: what’s new in CI, testing & development”

## The state of Orchestrator, 2020 (spoiler: healthy)

MySQL GitHub MySQL Open Source orchestrator February 18, 2020February 18, 2020

This post serves as a pointer to my previous announcement about The state of Orchestrator, 2020.

Thank you to Tom Krouper who applied his operational engineer expertise to content publishing problems.

## The state of Orchestrator, 2020 (spoiler: healthy)

MySQL GitHub Open Source orchestrator February 18, 2020

Yesterday was my last day at GitHub, and this post explains what this means for `orchestrator`. First, a quick historical review:

- 2014: I began work on `orchestrator` at Outbrain, as https://github.com/outbrain/orchestrator. I authored several open source projects while working for Outbrain, and created `orchestrator` to solve discovery, visualization and simple refactoring needs. Outbrain was happy to have the project developed as a public, open source repo from day 1, and it was released under the Apache 2 license. Interestingly, the idea to develop `orchestrator` came after I attended Percona Live Santa Clara 2014 and watched “ChatOps: How GitHub Manages MySQL” by one Sam Lambert.
- 2015: Joined Booking.com where my main focus was to redesign and solve issues with the existing high availability setup. With Booking.com’s support, I continued work on `orchestrator`, pursuing better failure detection and recovery processes. Booking.com was an incredible playground and testbed for `orchestrator`, a massive deployment of multiple MySQL/MariaDB flavors and configuration.
- 2016 – 2020: Joined GitHub. GitHub adopted `orchestrator` and I developed it under GitHub’s own org, at https://github.com/github/orchestrator. It became a core component in github.com’s high availability design, running failure detection and recoveries across sites and geographical regions, with more to come. These 4+ years have been critical to `orchestrator`‘s development and saw its widespread use. At this time I’m aware of multiple large-scale organizations using `orchestrator` for high availability and failovers. Some of these are GitHub, Booking.com, Shopify, Slack, Wix, Outbrain, and more. `orchestrator` is the underlying failover mechanism for vitess, and is also included in Percona’s PMM. These years saw a significant increase in community adoption and contributions, in published content, such as Pythian and Percona technical blog posts, and, not surprisingly, increase in issues and feature requests.

###
2020

GitHub was very kind to support moving the `orchestrator` repo under my own https://github.com/openark org. This means all issues, pull requests, releases, forks, stars and watchers have automatically transferred to the new location: https://github.com/openark/orchestrator. The old links do a “follow me” and implicitly direct to the new location. All external links to code and docs still work. I’m grateful to GitHub for supporting this transfer.

I’d like to thank all the above companies for their support of `orchestrator` and of open source in general. Being able to work on the same product throughout three different companies is mind blowing and an incredible opportunity. `orchestrator` of course remains open source and licensed with Apache 2. Existing Copyrights are unchanged.

As for what’s next: some personal time off, please understand if there’s delays to reviews/answers. My intention is to continue developing `orchestrator`. Naturally, the shape of future development depends on how `orchestrator` meets my future work. Nothing changes in that respect: my focus on `orchestrator` has always been first and foremost the pressing business needs, and then community support as possible. There are some interesting ideas by prominent `orchestrator` users and adopters and I’ll share more thoughts in due time.

## Speaking at August Penguin, MySQL Track, GitHub sponsored

MySQL gh-ost GitHub Open Source orchestrator Speaking September 3, 2017September 3, 2017

This Thursday I’ll be presenting at August Penguin, conveniently taking place September 7th, 8th, Ramat Gan, Israel.

I will be speaking as part of the MySQL track, 2nd half of Thursday. The (Hebrew) schedule is here.

My talk is titled Reliable failovers, safe schema migrations: open source solutions to MySQL problems. I will describe some of the open source MySQL infrastructure work we run at GitHub ; how it solves reliability, availability and usability. I’ll describe some of our internal workflows and our use of chat and chatops.

I’m proud to announce GitHub sponsors the event. We won’t have a booth, but please do grab me in the hallways or over lunch to chat!

And, yes, octocat stickers will be made available 🙂

## Practical Orchestrator, BoF, GitHub and other talks at Percona Live 2017

MySQL GitHub orchestrator PerconaLive April 19, 2017April 19, 2017

Next week I will be presenting Practical Orchestrator at Percona Live, Santa Clara.

As opposed to previous `orchestrator` talks I gave, and which were either high level or algorithmic talks, Practical Orchestrator will be, well… practical.

The objective for this talk is that attendees leave the classroom with a good grasp of `orchestrator`‘s powers, and know how to set up `orchestrator` in their environment.

We will walk through discovery, refactoring, recovery, HA. I will walk through the most important configuration settings, share advice on what makes a good deployment, and tell you how we and others run `orchestrator`. We’ll present a few scripting/automation examples. We will literally set up `orchestrator` on my computer.

It’s a 50 minute talk and it will be fast paced!

### ProxySQL & Orchestrator BoF

ProxySQL is all the rage, and throughout the past 18 months René Cannaò and myself discussed a few times the potential for integration between ProxySQL and Orchestrator. We’ve also received several requests from the community.

We will run a BoF, a very informal session where we openly discuss our thoughts on possible integration, what makes sense and what doesn’t, and above all else would love to hear the attendees’ thoughts. We might come out of this session with some plan to pick low hanging fruit, who knows?

The current link to the BoF sessions is this. It seems terribly broken, and hopefully I’ll replace it later on.

### GitHub talks

GitHub engineers will further present these talks: Continue reading » “Practical Orchestrator, BoF, GitHub and other talks at Percona Live 2017”

## Introducing gh-ost: triggerless online schema migrations

MySQL Development GitHub Open Source operations Performance tools August 1, 2016

I’m thoroughly happy to introduce gh-ost: triggerless, controllable, auditable, testable, trusted online schema change tool released today by GitHub.

gh-ost now powers our production schema migrations. We hit some serious limitations using pt-online-schema-change on our large volume, high traffic tables, to the effect of driving our database to a near grinding halt or even to the extent of causing outages. With gh-ost, we are now able to migrate our busiest tables at any time, peak hours and heavy workloads included, without causing impact to our service.

gh-ost supports testing in production. It goes a long way to build trust, both in integrity and in control. Are your databases just too busy and you cannot run existing online-schema-change tools? Have you suffered outages due to migrations? Are you tired of babysitting migrations that run up to 3:00am? Tired of being the only one tailing logs? Please, take a look at gh-ost. I believe it changes online migration paradigm.

For a more thorough overview, please read the announcement on the GitHub Engineering Blog, and proceed to the documentation.

gh-ost is open sourced under the MIT license.

## Orchestrator progress

MySQL Development GitHub Open Source orchestrator secondary December 23, 2015

This comes mostly to reassure, having moved into GitHub: orchestrator development continues.

I will have the privilege of working on this open source solution in GitHub. There are a few directions we can take orchestrator to, and we will be looking into the possibilities. We will continue to strengthen the crash recovery process, and in fact I’ve got a couple ideas on drastically shortening Pseudo-GTID recovery time as well as other debts. We will look into yet other directions, which we will share. My new and distinguished team will co-work on/with orchestrator and will no doubt provide useful and actionable input.

Orchestrator continues to be open for pull requests, with a temporal latency in response time (it’s the Holidays, mostly).

Some Go(lang) limitations (namely the import path, I’ll blog more about it) will most probably imply some changes to the code, which will be well communicated to existing collaborators.

Most of all, we will keep orchestrator a generic solution, while keeping focus on what we think is most important – and there’s some interesting vision here. Time will reveal as we make progress.

## Joining GitHub

General GitHub Open Source November 30, 2015

Today was my last day at Booking.com, and shortly I will be joining the team at GitHub.

I’d like to thank the many kind, friendly & smart people I’ve worked with at Booking.com!

The challenges at Booking.com are big. There is such a diversity within the technology stack; even within the database range. A solution that works on all the various Booking.com production environments is something to value. Indeed, the Booking.com Production environment it is an amazing playground for developers, offering high volume, large numbers, and differing workloads to tackle. Your code just gets hammered down and you get very quick feedback on whether you did it right or wrong.

I was happy to have worked on serious reliability and operational topics, and to have made a meaningful contribution.

Joining GitHub, I’m to be a systems engineer in a great team (friends included), building great products, in and around the database zone, delivering open source, pretty much expecting to do awesome stuff! That, and the swag.

## Forking Golang repositories on GitHub and managing the import path

Development GitHub golang November 23, 2015November 24, 2015

Problem: there’s an awesome Golang project on GitHub which you want to fork. You want to develop & collaborate on that fork, but the golang import path, in your source code, still references the original path, breaking everything.

A couple solutions offered below. First, though, let’s get some names.

### A sample case, the problem at hand

There’s an awesome tool on http://github.com/awsome-org/tool. You successfully fork it onto http://github.com/awesome-you/tool.

You want to collaborate on http://github.com/awesome-you/tool; you wish to pull, commit & push. Maybe you want to send pull requests to the origin.

The following is commonly found throughout .go files in the repository:

>     import (
>         "github.com/awesome-org/tool/config"
>         "github.com/awesome-org/tool/driver"
>         "github.com/awesome-org/tool/net"
>         "github.com/awesome-org/tool/util"
>     )

If you:

>     go get http://github.com/awesome-you/tool

golang creates your $GOPATH/src/github.com/awesome-you/tool/, which is awesome. However, as you resolve dependencies via

>     cd $GOPATH/src/github.com/awesome-you/tool/ ; go get ./...

golang digs into the source code, finds references to github.com/awesome-org/tool/config, github.com/awesome-org/tool/driver etc, and fetches those from http://github.com/awsome-org/tool and onto $GOPATH/src/github.com/awesome-org/tool/, which is not awesome. You actually have two copies of the code, one from your fork, one from the origin, and your own fork will be largely ignored as it mostly points back to the origin.

### A bad solution

The dirty, bad solution would be for you to go over the source code and replace “github.com/awesome-org/tool” entries with “github.com/awesome-you/tool”. It is bad for two reasons:

- You will not be able to further pull changes from upstream
- You will not be able to pull-request and push your own changes upstream

Continue reading » “Forking Golang repositories on GitHub and managing the import path”