{"id":6417,"date":"2013-08-08T11:26:02","date_gmt":"2013-08-08T09:26:02","guid":{"rendered":"http:\/\/code.openark.org\/blog\/?p=6417"},"modified":"2013-08-08T20:03:29","modified_gmt":"2013-08-08T18:03:29","slug":"tool-of-the-day-q","status":"publish","type":"post","link":"https:\/\/code.openark.org\/blog\/mysql\/tool-of-the-day-q","title":{"rendered":"Tool of the day: q"},"content":{"rendered":"<p>If you work with command line and know your SQL, <strong>q<\/strong> is a great tool to use:<\/p>\n<p><a href=\"https:\/\/github.com\/harelba\/q\"><strong>q<\/strong><\/a> allows you to query your text files or standard input with SQL. You can:<\/p>\n<blockquote>\n<pre>SELECT c1, COUNT(*) FROM \/home\/shlomi\/tmp\/my_file.csv GROUP BY c1<\/pre>\n<\/blockquote>\n<p>And you can:<\/p>\n<blockquote>\n<pre>SELECT all.c2 FROM \/tmp\/all_engines.txt AS all LEFT JOIN \/tmp\/innodb_engines.txt AS inno USING (c1, c2) WHERE inno.c3 IS NULL<\/pre>\n<\/blockquote>\n<p>And you can also combine with your favourite shell commands and tools:<\/p>\n<blockquote>\n<pre>grep \"my_term\" \/tmp\/my_file.txt | q \"SELECT c4 FROM - JOIN \/home\/shlomi\/static.txt USING (c1)\" | xargs touch<\/pre>\n<\/blockquote>\n<p>Some of <strong>q<\/strong>&#8216;s functionality (and indeed, SQL functionality) can be found in command line tools. You can use <strong>grep<\/strong> for pseudo <strong>WHERE<\/strong> filtering, or <strong>cut<\/strong> for projecting, but you can only get so far with <strong>cat my_file.csv | sort | uniq -c | sort -n<\/strong>. SQL is way more powerful for working with tabulated data, and so <strong>q<\/strong> makes for a great addition into one&#8217;s toolbox.<\/p>\n<p>The tool is authored by my colleague <a href=\"https:\/\/github.com\/harelba\">Harel Ben-Attia<\/a>, and is in daily use over at our company (it is in fact installed on all production servers).<\/p>\n<p>It is of course free and open source (<a href=\"https:\/\/github.com\/harelba\/q\">get it on GitHub<\/a>, where you can also find documentation), and very easy to setup. Enjoy!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>If you work with command line and know your SQL, q is a great tool to use: q allows you to query your text files or standard input with SQL. You can: SELECT c1, COUNT(*) FROM \/home\/shlomi\/tmp\/my_file.csv GROUP BY c1 And you can: SELECT all.c2 FROM \/tmp\/all_engines.txt AS all LEFT JOIN \/tmp\/innodb_engines.txt AS inno USING [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"enabled":false},"version":2}},"categories":[32,5],"tags":[78,57,21,100],"class_list":["post-6417","post","type-post","status-publish","format-standard","hentry","category-linux","category-mysql","tag-command-line","tag-open-source","tag-sql","tag-tools"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p2bZZp-1Fv","_links":{"self":[{"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/posts\/6417","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/comments?post=6417"}],"version-history":[{"count":12,"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/posts\/6417\/revisions"}],"predecessor-version":[{"id":6466,"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/posts\/6417\/revisions\/6466"}],"wp:attachment":[{"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/media?parent=6417"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/categories?post=6417"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/code.openark.org\/blog\/wp-json\/wp\/v2\/tags?post=6417"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}