Hardscrabble 🍫

By Max Jacobson

refactoring very old code

13 Apr 2014

Before I went to Flatiron School last summer I cobbled together a web app called Layabout. This weekend, for the first time in a very long time, I worked up the nerve to look at the source code. You may have heard the phrase “code smell” but I’m not sure it applies to this code. This code is like that ominous stink you’d almost rather ignore?

Hmm.

Layabout is an Instapaper client that focuses on the videos you’ve bookmarked. I was the only one who ever used it. I stopped using it in part beause it was buggy, and in part because I wanted to try out Pocket for a little while, which ended up being most of a year.

Inspired by John Resig’s recent blog post “Write Code Every Day” encouraging employed programmers to maintain side projects in a sustainable, I decided to revisit Layabout and see if anything I’ve learned in the year since I last edited it would come in handy. I also enjoy laughing at myself from the past.

The app had been running on Heroku unused for a year. When I logged in1 and tried to watch a video, it immediately crashed. I found the codebase locally and ran heroku logs to try to get some more information about the crash but it simply said it was a 500 error. Hmm.

So I tried running the app locally2 and was surprised to find that it worked even worse than it did on Heroku! Online at least I could type my password in before it crashed, but locally no pages were rendering at all. Instead it was complaining about not being able to find the views. I googled the error message and learned that some versions of Sinatra require you to explicitly tell it where your views are. I had never had this error before, but I was having it now. I looked at my Gemfile and saw that in my list of dependencies, I didn’t specify my desired version number for any of them. Code smell! You gotta lock that down, because things can change and break, especially when you don’t edit your code for over a year.3

After resolving that error I was able to reproduce the crash and see a much more verbose error message. You know how some YouTube videos have embedding disabled? I’m using a library, ruby-oembed, to fetch embed codes from video URLs, and it raises an exception when the video has embedding disabled. I used to think that meant my thing needed to crash but I’ve since learned about exception handling. Here’s a before:

def get_embed (vid_site, id)
  if vid_site == :youtube
    url = "http://www.youtube.com/watch?v=#{id}"
    return OEmbed::Providers::Youtube.get(url).html
  elsif vid_site == :vimeo
    url = "http://www.vimeo.com/#{id}"
    return OEmbed::Providers::Vimeo.get(url, maxwidth: "500", portrait: false, byline: false, title: false).html
  elsif vid_site == :hulu
    url = "http://www.hulu.com/watch/#{id}"
    return OEmbed::Providers::Hulu.get(url).html
  else
    return "<p>Failed to get embed code</p>"
  end
end

And after:

def get_embed (vid_site, id)
  begin
    case vid_site
    when :youtube
      OEmbed::Providers::Youtube.get("http://www.youtube.com/watch?v=#{id}").html
    when :vimeo
      OEmbed::Providers::Vimeo.get("http://www.vimeo.com/#{id}", maxwidth: "500", portrait: false, byline: false, title: false).html
    when :hulu
      OEmbed::Providers::Hulu.get("http://www.hulu.com/watch/#{id}").html
    else
      "Unsupported site"
    end
  rescue Exception => e
    puts e.inspect
    "<p>Sorry, couldn't get the embed code for this one. Maybe it doesn't allow embedding? Or maybe it was deleted? Sorry.</p>"
  end
end

(I guess I didn’t know about case statements or implicit return values then either.)

It’s still not that great honestly. I wish it were more object oriented. I wish there were tests. I wish I could open source it without exposing my Instapaper API keys. I know it’s possible to scrub your history and I’m going to look into that, and then keep them in a separate file ignored by git; or maybe I’ll look into getting new keys. But I’m glad to be revisiting this project because it’s where I got excited about web development. One last before and after:

Layabout needs to ignore bookmarks that it doesn’t know how to embed. It also needs clean URLs to reliably lookup their embed codes. Here’s an old method which helped with this task:


def grok_url (url)
  # TODO support hulu short urls
  if url =~ /youtube\.com\/embed\//
    id = url.match(/\/embed\/([A-Za-z0-9_-]+)/)[1].to_s
    site = :youtube
  elsif url =~ /youtube\.com/
    id = url.match(/v=([A-Za-z0-9_-]+)/)[1].to_s
    site = :youtube
  elsif url =~ /youtu\.be/
    id = url.match(/(http:\/\/youtu.be\/)([A-Za-z0-9\-_]+)/)[2].to_s
    site = :youtube
  elsif url =~ /vimeo\.com\/m\//
    id = url.match(/\/m\/(\d+)/)[1].to_s
    site = :vimeo
  elsif url =~ /vimeo\.com/
    id = url.match(/vimeo\.com\/([\d]+)/)[1].to_s
    site = :vimeo
  elsif url =~ /hulu\.com\/watch/
    id = url.match(/watch\/([\d]+)/)[1].to_s
    site = :hulu
  else
    return false, false, false
  end
  return true, site, id
end

I used it like this:

video_links = Array.new
if folder_id == :readlater
  all_links = ip.bookmarks_list(:limit => 500)
else
  all_links = ip.bookmarks_list(:limit => 500, :folder_id => folder_id)
end
all_links.each do |link|
  if link["type"] == "bookmark" # filters out the first two irrelevant items
    is_video, vid_site, video_id = grok_url link["url"]
    if is_video == true
      link["video_id"] = video_id
      link["title"] = cleanup_title link["title"] # prob not necessary
      link["vid_site"] = vid_site
      link["description"] = make_clicky link["description"]
      video_links.push link
    end
  end
end
@videos = video_links

Honestly I think this is cool code even though it’s not how I’d write it now, because this is what it looks like to make it work with limited knowledge. I didn’t know about any fancy iterators. I legit didn’t know about object oriented programming. I didn’t know it’s not necessary to check if a boolean == true, but it wasn’t hurting anyone! Whenever I’m talking to beginners and see code that doesn’t take advantage of less-than-obvious techniques, but works, I’m psyched.

Anyway this is what I changed it to yesterday:

@videos = (if folder_id == :readlater
  ip.bookmarks_list(:limit => 500)
else
  ip.bookmarks_list(:limit => 500, :folder_id => folder_id)
end).map do |link|
  if link['type'] == 'bookmark' # filters out the first two irrelevant items
    snob = FilmSnob.new(link['url'])
    if snob.watchable?
      link.tap do |link|
        link['video_id'] = snob.id
        link['title'] = cleanup_title link['title']
        link['vid_site'] = snob.site
        link['description'] = make_clicky link['description']
      end
    end
  end
end.compact

I don’t even necessarily think that’s better, but it satisfies my learned allergy to local variables4 and makes me feel cool for using map and tap.

Ah, and what’s FilmSnob? Let’s revisit Resig’s blog post:

I decided to set a couple rules for myself:

  1. I must write code every day. I can write docs, or blog posts, or other things but it must be in addition to the code that I write.
  2. It must be useful code. No tweaking indentation, no code re-formatting, and if at all possible no refactoring. (All these things are permitted, but not as the exclusive work of the day.)
  3. All code must be written before midnight.
  4. The code must be Open Source and up on Github.

Because Layabout isn’t currently open-sourceable, I extracted part of it into a rubygem called film_snob. It’s basically just that grok_url method, but made object-oriented, tested, and designed to be more extensible in case I ever want to support more than three video sites. I hope to also extract the embedcode fetching from Layabout into film_snob, but without the ruby-oembed dependency.


update I did implement the oembed stuff!! Yeah!!

  1. By giving my Instapaper credentials directly to the app, which I now know is kind of a no-no in the age of Oauth (though I’ve given my Instapaper credentials directly to some other apps like Reeder and Tweetbot so idk. Definitely a smell. 

  2. Which was totally possible because all of the API keys were hardcoded into the app. Smell! 

  3. What’s weird to me is that the version number seems to have regressed? The assumption that views are in a folder called views was introduced in 1.1, and I seem to have been taking advantage of that, but then it forgets to assume that? I don’t know. Versions are hard. 

  4. except the snob variable which I’m not sure how to avoid; suggestions welcome!! 

Atom Editor

13 Apr 2014

I tried out Atom Editor, the new text editor from GitHub.

I’m excited that GitHub is the kind of company that makes a new text editor. GitHub plays such a big part in my life as a programmer that it’s easy to forget that they’re basically irrelevant to almost all my friends and family. But maybe they won’t be forever.

Atom looks and feels uncannily like Sublime Text. I saw some speculation that GitHub had acquired Sublime Text and I kind of hope that’s true, although there’s been no confirmation or real indication that this is the case. It’s very flagrant. But, there are several ways in which it’s much better:

  • It comes with a simple, native package manager
    • textmate does too, I think, but it’s SO CONFUSING I’ve been trying to figure out how it works for years
    • sublime requires a third party thing which is confusing to install, but otherwise awesome
    • vim also requires a third party thing which is kind of confusing to install, but then, it’s vim, and everything is confusing
  • it reliably auto-installs its command line utility for launching the editor without needing to do any fancy symlinking business unlike Sublime
  • it’s written mostly in CoffeeScript and has many open source components to look at and maybe learn from
  • has some (but less than I expected) native integration with Git and GitHub, for example:
    • in the footer it will tell you how many lines were added and removed in this file since the last commit
    • has a native package to open the current file on GitHub

For the above reasons, this is probably the text editor that I’ll recommend to people who are new to programming.


I wrote the above when I first got the beta a few weeks ago and expected to elaborate before posting. I haven’t used Atom very much since thenor even heard much about it. I went to a vim meetup and an emacs meetup last week. I use vim but I’m curious to learn what emacs is all about. One of the speakers at the vim meetup said he uses and practices vim because it makes him more productive. I don’t know if I buy that – I’ve seen people be amazingly productive with a bunch of tools. But I think it makes it more fun and enjoyable, which I guess can lead to productivity. I’m reminded of a meme you hear in the writing community: writing about writing isn’t the point, writing is.

shebangs and bins

19 Jan 2014

I had a thought the other night as I was falling asleep: I guess I could create my own shebang? I asked Siri to remind me later and fell asleep.

OK what’s a shebang? I’m going to try to explain my understanding of it step by step, forgive me if this is obvious or boring.

So let’s say you write a ruby program that looks like this:

# hello.rb
puts "hello world"

And you run it from the command line by writing ruby hello.rb and hitting enter.

Or maybe write basically the same thing in python like this:

# hello.py
print "hello world"

And run it from the command line by writing python hello.py and hitting enter.

And the same thing in CoffeeScript

# hello.coffee
console.log "hello world"

And run it by writing coffee hello.coffee and hitting enter.

Three different files are interpreted as three different programming languages by three different intrepreters. We indicate which language it is written in and should be interpreted as in two ways in those examples:

  1. which command we invoked (ruby, python, and coffee)
  2. the file extension (rb, py, coffee)

The file extension isn’t really necessary. See:

# hello.sandwich
puts "hello world"

? This file can be run with ruby hello.sandwich and it works totally fine. Probably continue to use conventional file extensions because they signal to your text editors and friends what type of file they are. But I think it’s kind of helpful to know that they’re not totally necessary or magical.

OK so what if you want to call this program without specifying the interpreter every single time? This is where shebangs come in:

#!/usr/bin/env ruby
# hello.sandwich
puts "hello world"

Now this is a file with a nonsense file extension but something new: the first line (the shebang) indicates which program is meant to interpret this file. Interestingly, it’s not a direct path to that binary file on your file system (which in the case of Ruby programmers who use rvm to switch between Ruby versions, is often changing – mine is currently at /Users/maxjacobson/.rvm/rubies/ruby-2.1.0-p0/bin/ruby but only because I’m checking out the newly released Ruby 2.1.0), but instead consults your environment and says “hey, I’m looking for something called ruby, who wants to handle this one?”

How do you call this file if you don’t specify the interpreter? Basically you just provide a path to the file, like this: ./hello.sandwich. You’ll probably get this error: bash: ./hello.sandwich: Permission denied. I’m sure there are very good security reasons for this but it’s kind of annoying honestly. You need to run chmod +x hello.sandwich which transforms the script from a generic text file to an “executable” file. If you do that, now you can run ./hello.sandwich and your program will run thru the ruby interpreter.

What is the ./ part? It’s kind of ugly, and it’s the next thing we want to get rid of. It’s a path to the file. The same way you might run cd .. to change directories up a directory, you can run cd . to remain in your current directory. I know programming has irrevocably ruined my sense of humor because I now find it funny that you can run cd ././././././. to remain where you are. So ./hello.sancwich is a path to a file in the current directory.

Let’s actually rename the file to have no file extension (mv hello.sandwich sandwich) and run it again ./sandwich. Still works. It’s totally possible to run this file without the ./, but we need to put it somewhere special. We need to put it on our $PATH, which is a list of all of the places on your computer you keep these executable files. To see your $PATH you can run echo $PATH and you’ll see them spat out all colon separated. You can move your file to one of those directories or create a new directory, probably located at ~/bin or ~/Dropbox/bin (via), and add it to your $PATH.

$PATH is a global variable in your command line environment and it can be edited like this: export PATH="whatever" but probably don’t do that because it will totally overwrite the variable in your current terminal session and you won’t have access to any of the executables you depend on and you won’t be able to do anything. The common way to append a directory to your $PATH is like this: export PATH=~/bin:$PATH, which sets the variable to ~/bin, a colon, and then everything that used to be in the variable. Perfect. If you want this directory to always be accessible, not just in the current terminal session, you should add that line of code to your ~/.bashrc or ~/.bash_profile file1 and it will be included in every new terminal session.

So, if your sandwich file is in a directory that’s on your $PATH, and it has the shebang that specifies its interpreter, you can now run simply sandwich from anywhere on your computer and it will call that program. It’s kind of amazing how it does it, right? You write the word sandwich, hit enter, and it looks thru a list of directories for a file with that name; when it finds one, it checks its permissions, then looks at the first line to figure out how to interpret the file; that line points to an environment-checking program and passes in the argument “ruby”2, which finds the correct ruby interpreter, and then calls it, evaluating the remainder of the program.

OK so that’s awesome. But what if I want to write this program:

#!/usr/bin/env sandwich
# grilled_cheese
make_sandwich(
  ingredients: [
    "rye",
    "american cheese",
    "some sprinkles of gorgonzola",
    "maybe some sweet chili sauce",
    "butter"
    ],
  directions: [
    "get ingredients",
    "assemble a sandwich",
    "melt a little butter in a pan",
    "put the sandwich in the pan",
    "apply some downward pressure",
    "flip",
    "apply some downward pressure",
    "enjoy"
  ]
)

OK so I put “sandwich” in my shebang instead of a real programming language. What. Haha.

Let’s make the grilled_cheese file executable and put it in our bin, and then run grilled_cheese. We want to confirm that it’s looking up and finding the sandwich program and running it. When we see “hello world” again, we know it is. But it’s not making a sandwich! But then, how can it? Our code isn’t even being evaluated; if it were, we would probably have an error because the make_sandwich method is not availabe in the Ruby Standard Library (unfortunately).

Let’s edit our sandwich program to get a sense of what it knows about our grilled cheese program.

#!/usr/bin/env ruby
# sandwich
puts ARGV.inspect

Now when I run grilled_cheese, instead of seeing “hello world” I see ["/Users/maxjacobson/bin/grilled_cheese"]. Ahaa! Just because we invoke the sandwich program from the grilled cheese program doesn’t mean we will automatically evaluate the grilled cheese code. We just pass the filename as an argument, much like when we run ruby hello.rb. It’s a lot like if we ran sandwich grilled_cheese.

OK so knowing that, maybe our sandwich file should look something like this:

#!/usr/bin/env ruby
# sandwich

def make_sandwich(options)
  options[:ingredients].each do |ingredient|
    print "Finding #{ingredient}..."
    sleep 1
    print " done!\n"
  end
  options[:directions].each do |step|
    print "Doing #{step}..."
    sleep 1
    print " done!\n"
  end
end

eval File.read(ARGV.first)

So, this defines our missing method, takes in the file as an argument, reads the file as a string, and then evaluates the file with eval. OK so the only thing I know about eval is that you probably should never use it because it’s dangerous af. But maybe this is the kind of context where it makes sense?


I’m sure there are actually practical reasons to use custom-written programs in the shebangs of other custom-written programs and if you think of any please let me know.

  1. Unfortunately still don’t have a really solid understanding of the difference between those. 

  2. env is available to use from the command line as well: try running env ruby -v