command line – code.openark.org http://shlomi-noach.github.io/blog/ Blog by Shlomi Noach Wed, 17 Sep 2014 07:08:20 +0000 en-US hourly 1 https://wordpress.org/?v=5.3.3 32412571 zookeepercli: lightweight, powerful, controlled command line client for ZooKeeper https://shlomi-noach.github.io/blog/linux/zookeepercli-lightweight-powerful-controlled-command-line-client-for-zookeeper https://shlomi-noach.github.io/blog/linux/zookeepercli-lightweight-powerful-controlled-command-line-client-for-zookeeper#comments Wed, 17 Sep 2014 07:08:20 +0000 https://shlomi-noach.github.io/blog/?p=6980 I’m happy to announce the availability of zookeepercli: a lightweight, simple, fast and controlled command line client for ZooKeeper.

zookeepercli allows for:

  • Basic CRUD-like operations: createsetdeleteexistsgetls (aka children).
  • Extended operations: lsr (ls recursive),  creater (create recursively)
  • Well formatted and controlled output: supporting either txt or json format
  • Single, no-dependencies binary file, based on a native Go ZooKeeper library by github.com/samuel/go-zookeeper (LICENSE)

I was dissatisfied with existing command line access to ZooKeeper. Uncontrolled and noisy output as well as large footprint were among the reasons. zookeepercli overcomes the above and provides with often required powers.

Usage samples:


$ zookeepercli --servers srv-1,srv-2,srv-3 -c create /demo_only "path placeholder"
$ zookeepercli --servers srv-1,srv-2,srv-3 -c create /demo_only/key1 "value1"
$ zookeepercli --servers srv-1,srv-2,srv-3 -c create /demo_only/key2 "value2"
$ zookeepercli --servers srv-1,srv-2,srv-3 -c create /demo_only/key3 "value3"

$ zookeepercli --servers srv-1,srv-2,srv-3 -c ls /demo_only
key3
key2
key1

# Same as above, JSON format output:
$ zookeepercli --servers srv-1,srv-2,srv-3 --format=json -c ls /demo_only
["key3","key2","key1"]

$ zookeepercli --servers srv-1,srv-2,srv-3 -c delete /demo_only/key1
$ zookeepercli --servers srv-1,srv-2,srv-3 -c delete /demo_only/key2
$ zookeepercli --servers srv-1,srv-2,srv-3 -c delete /demo_only/key3
$ zookeepercli --servers srv-1,srv-2,srv-3 -c delete /demo_only

# Create a path recursively (auto-generate parent directories if not exist):
$ zookeepercli --servers=srv-1,srv-2,srv-3 -c creater "/demo_only/child/key1" "val1"
$ zookeepercli --servers=srv-1,srv-2,srv-3 -c creater "/demo_only/child/key2" "val2"

$ zookeepercli --servers=srv-1,srv-2,srv-3 -c get "/demo_only/child/key1"
val1

# This path was auto generated due to recursive create:
$ zookeepercli --servers=srv-1,srv-2,srv-3 -c get "/demo_only" 
zookeepercli auto-generated

# ls recursively a path and all sub children:
$ zookeepercli --servers=srv-1,srv-2,srv-3 -c lsr "/demo_only" 
child
child/key1
child/key2 

zookeepercli is released as open source by Outbrain under the Apache 2.0 license.

Quick links:

]]>
https://shlomi-noach.github.io/blog/linux/zookeepercli-lightweight-powerful-controlled-command-line-client-for-zookeeper/feed 9 6980
Tool of the day: q https://shlomi-noach.github.io/blog/mysql/tool-of-the-day-q https://shlomi-noach.github.io/blog/mysql/tool-of-the-day-q#comments Thu, 08 Aug 2013 09:26:02 +0000 https://shlomi-noach.github.io/blog/?p=6417 If you work with command line and know your SQL, q is a great tool to use:

q allows you to query your text files or standard input with SQL. You can:

SELECT c1, COUNT(*) FROM /home/shlomi/tmp/my_file.csv GROUP BY c1

And you can:

SELECT all.c2 FROM /tmp/all_engines.txt AS all LEFT JOIN /tmp/innodb_engines.txt AS inno USING (c1, c2) WHERE inno.c3 IS NULL

And you can also combine with your favourite shell commands and tools:

grep "my_term" /tmp/my_file.txt | q "SELECT c4 FROM - JOIN /home/shlomi/static.txt USING (c1)" | xargs touch

Some of q‘s functionality (and indeed, SQL functionality) can be found in command line tools. You can use grep for pseudo WHERE filtering, or cut for projecting, but you can only get so far with cat my_file.csv | sort | uniq -c | sort -n. SQL is way more powerful for working with tabulated data, and so q makes for a great addition into one’s toolbox.

The tool is authored by my colleague Harel Ben-Attia, and is in daily use over at our company (it is in fact installed on all production servers).

It is of course free and open source (get it on GitHub, where you can also find documentation), and very easy to setup. Enjoy!

]]>
https://shlomi-noach.github.io/blog/mysql/tool-of-the-day-q/feed 4 6417
MySQL command line vs. visual editors https://shlomi-noach.github.io/blog/mysql/mysql-command-line-vs-visual-editors https://shlomi-noach.github.io/blog/mysql/mysql-command-line-vs-visual-editors#comments Mon, 30 Jan 2012 15:04:34 +0000 https://shlomi-noach.github.io/blog/?p=4639 Students in my training classes usually prefer to use some kind of visual editor for MySQL. Typically this would be the software they’re using at work. Sometimes they just bring over their laptops with the software installed. Or they would use MySQL Workbench, which is what I usually have pre-installed on their desktops.

I see MySQL Workbench, SQLyog, Toad for MySQL, or several more.

I always humbly suggest they close down their software and open up a command line.

It isn’t fancy. It may not even be convenient (especially on Windows, in my opinion). And repeating your last command with a minor modification requires a lot of key stroking. Or you would copy+paste from some text editor. Most students will give it a shot, then go back to their favorite editor.

Well, again and again I reach the same conclusion:

Visual editors are not as trustworthy as the command line.

Time and again students show me something on their editor. Behavior seems strange to me. Opening up a console shows a completely different picture.

Things like:

  • The visual editor would open a new connection for every new query (oh, so the @user_defined_variable I’ve just assigned turns NULL, or the TEMPORARY TABLE disappears).
  • The visual editor will only show 1,000 results, via LIMIT 0,1000. “But the same query runs so much faster on my machine!”. Well, sure, a filesort of 1,000,000 rows that can satisfy the first 1,000 will quit early!
  • The visual editor shows table definition graphically. “I didn’t realize the index did(n’t) cover this and that columns. I didn’t realize it only covered first n characters of my VARCHAR.”. That’s because you can’t beat SHOW CREATE TABLE, the definite table structure description.
  • The visual editor allows for export/import/copy/transfer of tables and rows with just one click! “Why is it so complicated in the command line to purge 1,000,000 rows from a table?”. Ummm, did you realize the visual editor would typically use a naive approach of doing everything in one huge transaction?
  • The visual editor is smart. But sometimes you don’t want smart. You just assume simple. I personally take great precaution with smart solutions. Luckily, with scripts you have so much greater control (i.e. command line options, “dry-run” mode, etc.) that I have greater confidence in them.

I do like it when a visual editor plays it both smart and safe, in such way that before doing its smart work it actually presents you with the query it’s going to issue. Which is why I always considered MySQL Query Browser (now replaced by Workbench) to be the visual editor of choice in my classes.

But, at the end of the day, I strongly believe: if you don’t know how to do it with command line, you can’t really know how it’s done.

]]>
https://shlomi-noach.github.io/blog/mysql/mysql-command-line-vs-visual-editors/feed 14 4639