hardscrabble 🍫

By Max Jacobson

Psst. Check out my RubyConf 2017 talk, There are no rules in Ruby.

blog posts

how to run shell commands from ruby if you care about their output or if they failed

31 Jan 2016

Recently I made a new gem called shell whisperer which you might find useful for when your ruby programs need to run shell commands.

Let’s say you want to write a script which prints a summary of the current directory. The desired output is:

There are 213 files in this git repo.
Last commit: fixed a typo

There are two questions we need to ask the shell in order to print this output.

First question: how many files are there in this git repo?

First answer: we can ask git to list the files in the repo, and pipe the list to the word counting command to get the answer:

git ls-files | wc -l

Second question: what is the last commit?

Second answer: we can ask git for the log, limited to the most recent commit, and formatted to include just the first line:

git log -1 --pretty="%s"

So far so good, but how do we run these commands from Ruby?

The language provides two ways that I’m aware of:

# backtick style
`git ls-files | wc -l`

Or:

# system style
system 'git ls-files | wc -l'

What is the difference? I don’t want to go into all of the nuances (see this SO post for that and more) but I’ll share how I think of the difference:

  1. if you use the backtick (`) style, the return value is whatever was output by the command – but only STDOUT, not STDERR, so you’ll miss error messages
  2. if you use the system style, the output from the command will go to STDOUT, as if you had run puts and output some text, and the return value will signify whether the command failed or not

So our program might look something like this:

count = `git ls-files`.each_line.count
message = `git log -1 --pretty="%s"`.chomp
puts "There are #{count} files in this git repo."
puts "Last commit: #{message}"

And this is okay.

The issue becomes: well, what do you if you care that the command might fail? The system style allowed for checking the return value to see whether it succeeded or failed, but there’s a reason we’re not using the system style: we care about capturing the output of the command. So with the backtick style, we can capture the output, but (seemingly) we can’t capture the successfulness.

Well, we can, it’s just a little awkward:

count = `git ls-files`.each_line.count
raise 'list failed somehow' unless $?.success?
message = `git log -1 --pretty="%s"`.chomp
raise 'message failed somehow' unless $?.success?
puts "There are #{count} files in this git repo."
puts "Last commit: #{message}"

Which, OK, kind of cool, but what if we want to know why it failed?

This is possible:

count_or_failure_reason = `git ls-files 2>&1`.each_line.count
raise count_or_failure_reason unless $?.success?
message_or_failure_reason = `git log -1 --pretty="%s" 2>&1`.chomp
raise message_or_failure_reason unless $?.success?
puts "There are #{count_or_failure_reason} files in this git repo."
puts "Last commit: #{message_or_failure_reason}"

Let me attempt to explain this. The 2>&1 part means that we want the STDERR stream to be directed to the STDOUT stream, so that we’ll capture either one (or both). This gives us access to the reason the command failed, if it failed, but still gives us access to the output if it succeeds.

I found myself doing this in multiple places, so I decided to wrap this pattern up in a tiny gem, which allows you to instead write your program like this:

require 'shell_whisperer'
count = ShellWhisperer.run('git ls-files').each_line.count
message = ShellWhisperer.run('git log -1 --pretty="%s"').chomp
puts "There are #{count} files in this git repo."
puts "Last commit: #{message}"

If any of the commands fail, that error message will be re-raised as a ShellWhisperer::CommandFailed exception, so you can handle that as you please.

The node.js community seems to be all about tiny modules, and I think that idea is very cool, and I’m hoping to find more opportunities to do that with Ruby.

how to re-draw the line you just printed in Ruby, like to make a progress bar

15 Dec 2015

Here’s something I learned recently. Let’s say you have a program that is going to take a long time, and you want to mark the progress over time. You can print out some information like this:

tasks = Array.new(1000)
tasks.each.with_index do |task, index|
  sleep rand(0..0.1) # (something slow)
  percentage = (index + 1) / tasks.count.to_f
  puts "#{(percentage * 100).round(1)}%"
end

Which looks kinda like this:

progress bar before picture

Which is, let’s say, serviceable, but not, let’s say, beautiful. It stinks that it printed out all those lines when it didn’t really need to. I would rather it had sort of animated while it went. But how is this done?

This is one of those questions that’s itched at the back of my mind for a while and which, when I finally googled it, was a bit disappointing. It’s just another unix escape character, like \n (which prints a new line). It’s \r, which I now think of as “the backspace to the beginning of the line” magic character.

Armed with this knowledge and some clunky math we can write something like this:

begin
  tasks = Array.new(1000)
  tasks.each.with_index do |task, i|
    width = `tput cols`.to_i
    sleep rand(0..0.1) # (something slow)
    percentage = (i + 1) / tasks.count.to_f
    summary = "#{(percentage * 100).round(1)}% ".rjust("100.0% ".length)
    remaining_chars_for_progress_bar = width - summary.length - 2
    chunks = (percentage * remaining_chars_for_progress_bar).ceil
    spaces = remaining_chars_for_progress_bar - chunks
    bar = "\r#{summary}[#{ '#' * chunks }#{' ' * spaces}]"
    print bar
  end
rescue Interrupt
  system "say 'I was almost done, jeez'" if RUBY_PLATFORM.include?("darwin")
end

progress bar after gif

Probably you shouldn’t use this – there’s a very nice gem called ruby-progressbar which will work across platforms and lets you customize some things. But it’s nice information to have, because now you can do things like this:

barnyard

I’ll leave it as an exercise to the reader how to write this one.