Hardscrabble šŸ«

By Maxwell Jacobson

See also: the archives and an RSS feed

analytics

April 2, 2015

A few months ago I installed Google Analytics on hardscrabble, and a few weeks ago I installed Clicky, a competing service.

I felt a little skeezy, a little Not Punk, when I did that.

Aren’t I supposed to not care what people think and write from the heart?

Well, to be honest, I do. I check the stats sometimes. And I’m letting it affect what I do and write.

But hopefully in a good way!

For example: I learned, much to be my surprise, that this is consistently my most popular post. It’s not a mega hit or anything, but it gets a few visits every day. People want a good solution to that particular problem, and there aren’t that many solutions at all.

Unfortunately, my solution from January 2014 is pretty bad. I feel bad for those people. I want to help them. Maybe analytics can tell me who on this Earth I can help, and maybe that’s beautiful.

Anyway, I’m writing this on Editorial on my iPad (with a bluetooth keyboard; I’m not crazy), and I’m about to publish it using a MUCH IMPROVED, lighter-weight workflow. I want to battle-test it a little bit more before I share it. Maybe this won’t even

ruby keyword arguments aren't obvious

March 29, 2015

Since Ruby 2.0, methods can be defined with keyword arguments instead of the traditional ordinal arguments. I really like them. But they’re not obvious. I find myself thinking, maybe too often, ā€œwait, how the hell do these work?ā€

This post is a stream of consciousness exploration through the things about keyword arguments that confuse or confused me or people I know. Wading through this post might help you emerge with a firmer understanding of how to use them. I hope it makes sense if you read it. Please let me know. I want to write about a thing I find confusing in a way that honors that confusion but is also clear and readable, which maybe isn’t a thing I can do.

Let’s start with an example of keyword arguments working as-advertised:

def email(from:, to:, subject:, body:)
  "From: #{from}\nTo: #{to}\nSubject: #{subject}\n\n#{body}"
end

That’s a kind of strange method that takes a few required keyword arguments and makes a string from them. I like using keyword arguments here because now you can call the method however you like, as long as you provide all of the keywords; the order doesn’t matter, and it’s clear which one is which.

So, for example, here’s a perfectly valid way to call that method (note the order has changed):

email(
  subject: "Thanks!",
  from: "Max",
  to: "Susan",
  body: "The soup was great!"
)

We’re able to use required keyword arguments in Ruby 2.1 and forward. What if your app is using Ruby 2.0.0? You’d still like to reap the clarity benefits of keyword arguments, but now you must provide a default value for every keyword. What default makes sense for an email? I’m not sure. I guess you can do this?

def email(from: nil, to: nil, subject: nil, body: nil)
  raise ArgumentError if [from, to, subject, body].any?(&:nil?)
  "From: #{from}\nTo: #{to}\nSubject: #{subject}\n\n#{body}"
end

Which kind of simulates the behavior of Ruby 2.1 required keyword arguments. But it’s not great, because sometimes nil is actually a value that you want to be providing. So maybe you do something heavier, like this?

class AbsentArgument
end

def email(from: AbsentArgument.new, to: AbsentArgument.new, subject: AbsentArgument.new, body: AbsentArgument.new)
  raise ArgumentError if [from, to, subject, body].any? { |arg| arg.is_a?(AbsentArgument) }
  "From: #{from}\nTo: #{to}\nSubject: #{subject}\n\n#{body}"
end

Which is kind of clunky-looking but maybe more explicit?

Let’s be happy required keyword arguments are an official thing now and not worry about that and just hope we can all always use Ruby 2.1 or newer.

Keyword arguments kind of look like hashes. Are they hashes? I don’t know. You can use hashes with them:

arguments = {
  from: "Max",
  to: "Beth",
  subject: "Thanks!",
  body: "Your soup was great!"
}
email(**arguments)

That works. That ** coerces the hash into keyword arguments, kind of like the * coerces an array into ordinal arguments:

def sum(a_number, another_number)
  a_number + another_number
end

nums = [1, 1]
sum(*nums)

Except, the ** isn’t actually necessary, this works fine too:

arguments = {
  from: "Max",
  to: "Beth",
  subject: "Thanks!",
  body: "Your soup was great!"
}
email(arguments)

So I guess they don’t do anything there?

OK so when you are calling a method you can use a pre-existing hash for the keyword arguments. What about when you’re defining a method? This probably won’t work but I just don’t know because it doesn’t feel obvious. Let’s try.

Here’s our new example method, which works fine:

def stir_fry(ingredients: [], heat: 100)
  heat.times do
    ingredients = ingredients.shuffle
  end
  ingredients
end

stir_fry(ingredients: ['broccoli', 'peppers', 'tofu'], heat: 45)

So let’s try to define the method again, but this time let’s use a hash.

arguments = {
  ingredients: [],
  heat: 100
}
def stir_fry(arguments)
  heat.times do
    ingredients = ingredients.shuffle
  end
  ingredients
end

stir_fry(ingredients: ['broccoli', 'peppers', 'tofu'], heat: 45)

Do you think it works? It doesn’t work at all. I’m sorry.

Wait, so what even is the ** thing? Let’s review * again; I showed above how to use it to coerce an array into ordinal arguments when calling a method, but it can also be used in a method definition to indicate that a method takes an arbitrary number of arguments:

def sum(*numbers)
  total = 0
  numbers.each { |num| total += num }
  total
end

sum(1, 2, 3)
sum(*[1, 2, 3])

We can do something like that with ** to indicate that we want to catch all unrecognized keyword arguments into an object:

def stir_fry(ingredients: [], heat: 100, **options)
  heat.times do
    ingredients = ingredients.shuffle
  end
  if (sauce = options[:sauce])
    ingredients.push(sauce)
  end
  ingredients
end

stir_fry(ingredients: ['broccoli', 'peppers', 'tofu'], sauce: "teriyaki", heat: 45)

In that example, sauce is an optional keyword that isn’t defined in the method definition. Normally if we provide sauce, and sauce wasn’t specifically expected, that will cause an error, so this is kind of a cool way to say: ā€œI don’t care what keyword values you throw at me! I’ll just make a hash out of the ones I don’t recognizeā€. It doesn’t even care that sauce came in the middle of the expected keyword arguments. This is pretty similar to the convention in ordinal method definitions where the last argument is called options and it has a default value of an empty hash, but when you do that, the order really matters:

def ordinal_stir_fry(ingredients, heat, options = {})
  heat.times do
    ingredients = ingredients.shuffle
  end
  if (sauce = options[:sauce])
    ingredients.push(sauce)
  end
  ingredients
end

ordinal_stir_fry(["potato"], 5, sauce: 'Catsup') # This one works
ordinal_stir_fry(["shoe"], {sauce: 'Water'}, 5) # This one doesn't

What is even happening there? The curly braces become necessary to avoid a syntax error, and then the method receives the wrong values in the wrong names. But, looking at it, I think it’s clear that something is a little bit off, because the second parameter looks different from the third; it kind of feels like the hash belongs at the end, because that’s such a strong convention for ordinally-defined Ruby methods.

The **options example is neat but again, it’s not obvious. When looking at it, you don’t know which of the keyword arguments are specifically expected and which ones will land in the greedy **options bucket. You have to reference the method definition, just like with stinky ordinal methods.

Let’s look at default values in some more detail. It seems clear; you can provide a default value, which will be used when the method is called without that keyword value provided. What happens when you provide nil?

class Fish
  attr_reader :breed, :color

  def initialize(breed: "Koi", color: "Yellow")
    @breed = breed
    @color = color
  end
end

fish = Fish.new(color: nil)
fish.breed #=> "Koi"
fish.color #=> ????????

What do you feel like it should be? I guess it should be nil, because that’s the value you provided for that keyword, and yes that’s right, it’s nil. That works for me, but I know it’s not obvious because I found myself trying to defend this behavior to a friend recently, who was sad that it didn’t behave as his intuition desired, that a nil value would be replaced by the default value. To console him I attempted to write some code that would satisfy his expectations, and I came up with this:

class AbsentArgument
end

class Fish
  attr_reader :breed, :color

  def initialize(breed: AbsentArgument.new, color: AbsentArgument.new)
    @breed = validate(breed, default: "Koi")
    @color = validate(color, default: "Yellow")
  end

  private

  def validate(value, default:)
    if value.is_a?(AbsentArgument) || value.nil?
      default
    else
      value
    end
  end
end


fish = Fish.new(color: nil)
fish.breed #=> "Koi"
fish.color #=> "Yellow"

So if you want that behavior you basically can’t use Ruby keyword argument default values, because default values don’t work that way.

Oh, here’s another thing. I thing I only realized this this month, that I had been doing this:

class Disease
  attr_reader :name

  def initialize(name: name)
    @name = name
  end
end

gout = Disease.new(name: "Gout")
gout.name #=> "Gout"
rando = Disease.new
gout.name #=> nil

This was fulfilling the behavior I guess I wanted: when I provide a name, it should use that name; when I don’t provide a name, the name should be nil. It was working without error, and I’m not sure where I picked up the pattern of writing keyword arguments this way, but it actually totally makes no sense! If I wanted the default value to be nil, why not just write it like this?

class Disease
  attr_reader :name

  def initialize(name: nil)
    @name = name
  end
end

What was even happening in that earlier example? Well, when I wasn’t providing a name value, it was calling the name method which was only available because I had added the attr_reader for name, and that method was returning nil, so nil was being assigned to the @name instance variable. I had no idea that’s what was happening, I just thought that I was writing the boilerplate necessary to achieve that behavior. That feels kind of dangerous; maybe you don’t realize that your default values can call methods, and you’re calling a method that doesn’t exist? For example:

class Furniture
  attr_reader :color

  def initialize(kind: kind, color: color)
    @kind = kind
    @color = color
  end

  def description
    "#{@color} #{@kind}"
  end
end

couch = Furniture.new(kind: "Couch", color: "Grey")
couch.description #=> "Grey Couch"

You could have tests for this code and ship it to production and never realize that a bug hides within it. As long as you always provide a kind keyword value, you’ll never face it and it will work properly, because it will never attempt to call the kind method… which doesn’t exist.

So, to make it blow up, simply omit the kind keyword value:

Furniture.new(color: "Red")
# undefined local variable or method `kind' for #<Furniture:0x00000101110a38> (NameError)

Sinister!

Happily, I’m noticing now that Ruby 2.2.1 actually warns you when you write code like this. 2.0.0 does not, which is where I’ve been happily making this mistake for many months.

The warning:

def candy(flavor: flavor)
end
# warning: circular argument reference - flavor

What about when you combine ordinal arguments with keyword arguments? You can. Is it obvious how that should work? Not to me. Let’s take a look.

def stir_fry(servings = 1, zest, ingredients: [], **options)
  dish = (ingredients * servings)
  zest.times do
    dish = dish.shuffle
  end
  dish
end

stir_fry(8, ingredients: ["pepper", "seitan"], sauce: "fancy")

What the hell is happening there? Maybe it’s clear to you. it’s not to me. The first two arguments are ordinal, and the first one has a default value. So Ruby compares the arguments we provide when we call the method to the arguments in the method definition, and sees that we provided what looks like one ordinal value, and a few keyword values, so the one ordinal value must be zest, because servings has a default value and zest does not (Ruby here is smarter than I realized).

It kind of feels like Ruby is going to let us make this method definition more confusing, for example by moving the keyword arguments before the ordinal arguments, but it actually won’t let you. It will raise a syntax error:

# syntax error:
def shenanigans(amp: 11, guitar)
end

# if it were a valid method definition, I guess you would call it like this:
shenanigans(amp: 5, "Fender")
# or, omitting the non-required parameter
shenanigans("Martin")

For me it wasn’t obvious that this wouldn’t be allowed, but I’m glad it’s not, because reading those method calls feels kind of backwards.

Similarly:

# not a syntax error:
def shenanigans(amp, *outfits, **options)
end

shenanigans("Orange", "Leotard", "Leather Jacket", at: Time.now)

# but this is a syntax error:
def shenanigans(amp, style: "flamenco", *outfits, **options)
end

That one also fails because it tries to put some ordinal arguments (the *outfits bit) after some keyword arguments.

Well, I think that’s everything I can think of. Good luck out there.

my first vim macro

March 29, 2015

The time has come for me to make a vim macro. Here’s what it looks like in action:

vim macro demo

I made it because I am working on a post that will include a few ruby examples, and I got frustrated typing out Liquid’s verbose syntax for codeblocks (I wish Jekyll supported GitHub-style fenced code blocks). It wasn’t too hard to make it because it’s really simple. I don’t know how to make a smarter one, for example one that puts ruby as the default language, but let’s you start typing immediately to replace it with something else, and then hit tab to jump to the middle (a behavior I’ve seen with snippets in Sublime Text for example).

Here’s what it looks like, straight from my vimrc:

  
    nnoremap hii i```ruby<ESC>o```<ESC>O
  

I’ll translate to English:

When I’m in normal mode, and I type hii, act like I typed the following stuff, which I would’ve done if I had typed it out manually:

  • i to go into insert mode
  • type out the first line’s stuff
  • <ESC>o to exit insert mode, create a newline, go to that new line, and go back to insert mode
  • type out the second line’s stuff
  • <ESC>O to exit insert mode, create a newline above the second line, go to that new line, and go back to insert mode

It’s not much and I’m not even sure I’m using the word macro right but it’s mine.

Edit April 2015: Turns out hii was a pretty bad choice, because it starts with h, which I type all the time in normal mode. It’s the left arrow! Whenever I type h now, it doesn’t immediately move left, it hangs a moment while it waits to see if I really mean just h, or if I’m gonna continue and write hii. So, I’ve changed it from hii to <Leader>hii and now I’m happy again.

Note: the default Leader character is \.

managing your Instapaper bookmarks from the command line

March 22, 2015

I’ve previously written about extracting a gem from a web app called Layabout, but I neglected to mention that I later open sourced Layabout as well:

Today I have the opportunity to write about it, because I finished a kind of fun new feature: a command line interface for efficiently managing your bookmarks. Using it requires cloning the codebase and requesting API credentials, so it’s not a super accessible tool, but for power users it might be worth it. Here’s a quick demo:

bin/rake explore demo

Here’s the code for the CLI as of this moment: https://github.com/maxjacobson/layabout/blob/255eed15be2e55de804083bfdcf8651538af7bb0/lib/tasks/explore.rake

(I like linking to the code as of a certain commit, because who knows, maybe I’ll rename the file later, and then the link to the file on master branch won’t actually work?)

I think it’s kind of interesting code. Each action is an object that describes its help text, the commands it can handle, and how it can handle them.

inhumanity

March 22, 2015

Yesterday morning I grabbed my pants off the floor and a mouse scurried away. I shouted, ā€œahh!ā€, ran to my bedroom, and jumped on my bed, pants still in hand.

I had my Improv 201 graduation class performance and I was running late, so I didn’t have time to worry about it. I just got dressed and left with my eyes closed.

On my way to the show, I tried to remember the things we’d been taught over the last 8 weeks, but my mind kept going back to the mouse. It was small and brown; not a startling creature. And yet, very startling!

The show went well, I think. It was fun for me anyway. Afterward we had some food and drinks and then I went home and fell asleep and slept for 15 hours. I wake up around 10am without thinking about the mouse. I’m trying to keep my mind clear, because I’m having a surgery tomorrow, and I want to feel mentally and emotionally prepared so I can be the best patient I can be. And then I start seeing the mouse out of the corner of my eye.

At first it’s to my right, flitting from behind my radiator to behind my couch. I tense up. I pause The Mindy Project. I think it could tell I was occupied.

I resume The Mindy Project. I need to take the last of my antibiotics so I step into the kitchen to take a swig of rice milk. There I see the brown mouse again, by the oven, which is precisely on the other side of the wall from the radiator. I say out loud, ā€œI don’t like thisā€.

The next time I see the mouse, I’m back at my desktop, and it’s walking back into my living room via the hallway. I say, ā€œhey!ā€ and it turns around and walks back around the corner. I look away and see that he’s walking toward me again. I say ā€œhey, buddy!ā€ and he goes back. So I look away again, and now he’s straight up sprinting past me toward the couch again.

I grab my wallet, keys, and a hoodie, and I leave my apartment and call my mother. She advises me to trap the thing with a glue trap and feel nothing. She says I’ll feel satisfied, like a hunter.

From a nearby cafe I order some humane mouse traps on Amazon.

My grandma calls me and says she’s had good experiences with spring-loaded mouse traps, and that I’ll feel satisfied.

I check a nearby general store. They have glue traps for mice and spring traps for rats. With my grandma still on the phone, I buy the glue trap.

  • 13:15 – I set the traps, one in my hallway and one near the couch, and go sit in bed and watch another sitcom, Undateable, on Hulu. In my field of vision, I can see the first trap.
  • 14:00 – I notice I can see something in the trap and go check on it. It’s just the dab of peanut butter I’d put there earlier.
  • 15:11 – I see the mouse walk right past the trap and into my living room
  • 15:16 – I hear what sounds like panicked squeaking

My mom suggested the mouse would die as soon as it got stuck, of a panic attack, but it just kept squeaking for several minutes while I listened in horror. I feel no satisfaction until later, when I’m sitting in my living room again and nothing scampers through my periphery.

reverse polish notation

March 17, 2015

Earlier tonight Carlos tweeted this:

Nowhere in there does he specifically ask me to provide my take on that problem, but I did anyway. I don’t know why.

The problem is, I think, to write a reverse polish notation calculator in Ruby. Carlos used TDD to drive his solution. I looked at it and thought it was cool, and then I wanted to do the same thing, and I made a video, because I am a ham.

Here it is:

It is very long. There are a few moments where I removed sound to take away coughs. I might have missed some. I probably did!

You’ll hear every thought that passes through my mind as I arrive at a solution, which I pushed here: https://github.com/maxjacobson/calculator. A lot of it is me struggling to understand the very premise of the notation, which confused me perhaps too much?

Using tests helped me get this working because when it wasn’t working, I wasn’t sure which part wasn’t working, and I was able to add more tests to describe the parts, until I knew which parts were behaving how I expected and which weren’t. That’s really helpful. Accomplishing that meant extracting some of the responsibilities into a separate, small, testable class, which I think is a good example of lettings tests drive the design of your code. Ultimately the implementation of that class is kind of awkward and not great, but it’s also really contained and could be easily rewritten because there are tests to catch mistakes in the refactoring.

šŸƒ

git fib, a helpful little script for git commit squashers

March 8, 2015

Sometimes I squash git commits. Generally I do this when I’m making a pull request which takes some time, and it accumulates a lot of commits, many of which were just kind of trying things out. The sum of all the commits adds up to one valuable addition, but each one is kind of garbage. Here’s a good post from Thoughtbot: Git Interactive Rebase, Squash, Amend and Other Ways of Rewriting History.

Here’s what it generally looks like when I do it:

squashing a commit

And this works pretty well for me.

Usually.

Sometimes I do a really big squash, where 35 commits become one, which brings me to one behavioral detail of squashes that I’ve always found counter-intuitive: squashes always go up. When you’re in that interactive list of commits, and you’re telling each commit what to do, it’s relative to the previous chronological commit. When you tell a commit to squash, you’re telling it to squash itself into the previous commit. When you tell this to the 34 newest commits, you’re making your very first commit absorb all of the changes. That’s probably fine, but imagine if those commits took place over the course of 3 days, or 3 weeks. Each commit has a timestamp of when the work was done, and your big, squashed commit has a timestamp of… 3 weeks ago.

That sort of feels wrong to me. When the pull request is merged, the commit will sink down in the commit log below all the work that came before it.

Sooo sometimes I find myself taking things into my own hands to fix that. How? Well, it’s kind of weird. Changing the last commit’s message is pretty easy: git commit --amend; changing the last commit’s author is pretty easy too: git commit --amend --author="Kimmy Schmidt <kimmy.schmidt@example.com>". But changing the last commit’s timestamp is kind of tricky. As far as I know, it’s not built in to git itself, so it takes a few commands to achieve. Here’s what I’ve been doing:

  • squash all the commits
  • use my mouse to copy and paste the last commit message to the clipboard
  • uncommit the last commit (git reset HEAD~1)
  • re-stage all the changes (git add -A)
  • make a fresh commit, pasting in the old commit message (and probably having to fix some formatting issues because pasting into vim never works)
  • sometimes I’ll amend the last commit’s author to give someone else credit if they did the lion’s share of the work, because this strategy automatically uses my git name and email even if the commit we’re trying to time shift was created by someone else

Here’s what I do now:

  • git fib

Here’s a gif (ignore the part when I got stuck in a dquote confusion please):

git fibbing a commit

Get it here: https://github.com/maxjacobson/dotfiles/blob/master/bin/git-fib

I learned with git-sleep that scripts whose filename begins with git- can be referenced without the hyphen, making git a nicely extensible tool. I still think that’s so cool.

I’m pretty proud of this because it’s a kind of gnarly shell script and it works a lot better than I expected.

Some things I might do:

  • rename it to something that makes more sense
  • put it on homebrew so it’s easier to install
  • suppress more of the git output (not sure, maybe it’s nice to have it?)

šŸƒ


EDIT 2019: At some point in the intervening few years, I noticed someone run git commit --amend --date, and I made a note in my to do list to update this script and this post, and today I did.

playing with emacs for the first time

March 1, 2015

Today I’m playing with emacs. There has been a confluence of events which have led me to this moment:

  1. First, the other night I bumped into Harry Schwartz at a programming meetup, and he gave me a sticker from his meetup group, New York Emacs. I put it in my wallet.
  2. Second, yesterday I bumped into Andrew Leung, and somehow we got to talking about Emacs. He’s been using it for many, many years. I gave him the sticker from my wallet because I thought he would have more use for it than I would.
  3. Today I’m kind of bored.

To my non-surprise, the version of Emacs which came installed on my Mac is a few major versions behind. To my surprise, the version of emacs on Homebrew is also a bit old. Neither version has syntax highlighting for Ruby, which was provided by default in the newest version. Installing the newer version wasn’t that bad. I downloaded the code from their ftp server (emacs-24.4.tar.gz), unarchived it, and followed the instructions in the INSTALL file. Then, sweet, ruby files were colorful. That makes a huge difference for me.

I really don’t know what I’m doing, but I feel like I should learn more about emacs if I’m going to recommend people learn Vim, which I generally use and recommend, so today I took a little while to play with it and I’ve learned enough to do the super basic things. Here’s everything I know how to do:

  • emacs . from the command line to open emacs in a file browser
  • arrow keys and return to browse and open files
  • C-x C-s to save
  • C-x C-f <enter> to return to the file browser from a file
  • C-a to go to the beginning of a line
  • C-e to go to the end of a line
  • C-n to go to the next line
  • C-p to go to the previous line
  • C-_ to undo a change
  • C-s to search within a file
  • C-x C-c to quit (it prompts you to save any unsaved changes at this point)

(Note:

something like C-x can be read as ā€œpress the control key, and then press the x key while still holding down the control keyā€

something like C-x C-f can be read as ā€œpress the control key, and then press the x key while still holding down the control key; then keep holding down the control key, and press the f key, and then feel free to go get a coffee, you’re doneā€)

And that’s it. Things I’d like to learn:

  • More commands to navigate the text within a file (deleting lines, copying, cutting, pasting)
  • How to do some simple configuration (I want line numbers)
  • How to more efficiently open files (I want a fuzzy file opener like ctrlp.vim)

That’s really all that’s blocking me from actually using it to do work. I’m using it to write this post, and I’m totally surviving.

If I were to go down the rabbit hole, I’d probably want to learn the whole Emacs Lisp thing. Richard Stallman, the creator of Emacs, says this:

The most powerful programming language is Lisp. If you don’t know Lisp (or its variant, Scheme), you don’t know what it means for a programming language to be powerful and elegant. Once you learn Lisp, you will understand what is lacking in most other languages.

It’s hard to read that and not get some programming FOMO.

I attended this talk from Harry called ā€œAn Introduction to Emacs Lispā€, which I may need to rewatch.

another cookie crumb

February 21, 2015

In Wednesday’s post about building Firefox from source the other day, I mentioned one thing I really liked about the experience of first looking at this code base: it kept dropping little hints about what I might need to do next in a way that was actually insightful and helpful. For example, after running their script which installs dependencies, there was some helpful output pointing me to where I could get the code base.

Today I checked in with the project and pulled the change from the last few days. There has been a flurry of activity, none of which means much of anything to me as an outside observer.

I was curious if I would be able to run ./mach build, and if it would take as long on the second run. Instead I got this interesting output:

0:00.35 /usr/bin/make -f client.mk -s
0:01.33
0:01.33 The CLOBBER file has been updated, indicating that an incremental
0:01.33 build since your last build will probably not work. A full/clobber
0:01.33 build is required.
0:01.33
0:01.33 The reason for the clobber is:
0:01.33
0:01.33  Bug 1119335 - (DOMString or sequence<DOMString> or ConstrainDOMStringParameters)
0:01.33  needs binding flush (Bug 1103153).
0:01.33
0:01.33 Clobbering can be performed automatically. However, we didn't
0:01.33 automatically clobber this time because:
0:01.33
0:01.33   Automatic clobbering is not enabled
0:01.33   (add "mk_add_options AUTOCLOBBER=1" to your mozconfig).
0:01.33
0:01.33 The easiest and fastest way to clobber is to run:
0:01.33
0:01.33  $ mach clobber
0:01.33
0:01.33 If you know this clobber doesn't apply to you or you're feeling lucky
0:01.33 -- Well, are ya? -- you can ignore this clobber requirement by
0:01.33 running:
0:01.33
0:01.33  $ touch /Users/maxjacobson/src/gecko-dev/obj-x86_64-apple-darwin14.3.0/CLOBBER
0:01.33 make: *** [/Users/maxjacobson/src/gecko-dev/obj-x86_64-apple-darwin14.3.0/CLOBBER] Error 1
0:01.36 78 compiler warnings present.

That’s super interesting! Things I learned from this:

  • There’s something called ā€œThe CLOBBER fileā€ which is some kind of place to leave little messages about why someone else will need to do a full build after pulling in a change
  • Sometimes it’s possible for mach to do an ā€œincremental buildā€, which I assume wouldn’t take very long
  • There’s a tool for clobbering, which I guess clears away stuff from earlier builds, requiring mach build to do a full, non-incremental build
  • Mozilla people make nerdy movie references

I took a look at The CLOBBER file by using my fuzzy file opener to look for a file called CLOBBER and found it, in the root-level of the project. It contains more details about how to use it:

# To trigger a clobber replace ALL of the textual description below,
# giving a bug number and a one line description of why a clobber is
# required. Modifying this file will make configure check that a
# clobber has been performed before the build can continue.
#
# MERGE NOTE: When merging two branches that require a CLOBBER, you should
#             merge both CLOBBER descriptions, to ensure that users on
#             both branches correctly see the clobber warning.
#
#                  O   <-- Users coming from both parents need to Clobber
#               /     \
#          O               O
#          |               |
#          O <-- Clobber   O  <-- Clobber
#
# Note: The description below will be part of the error message shown to users.
#
# Modifying this file will now automatically clobber the buildbot machines \o/
#

# Are you updating CLOBBER because you think it's needed for your WebIDL
# changes to stick? As of bug 928195, this shouldn't be necessary! Please
# don't change CLOBBER for WebIDL changes any more.

Bug 1119335 - (DOMString or sequence<DOMString> or ConstrainDOMStringParameters)
needs binding flush (Bug 1103153).

It’s probably gratuitous to just copy this whole thing in here, especially when the last few lines are designed to change, but it’s pretty interesting, and I have a weird superstition about linking to files on GitHub, for example, because what if that file gets moved and my link goes dead?

This file is like a cache. As long as it stays the same, you can keep building. As soon as it changes, you need to clobber before you can build. It seems like a clever solution to me (I especially like the detail about combining clobber notes so when two people insist on a clobber, they each get the benefit of the other’s). You just need to remember when to expire the cache, which leaves a little cookie crumb for the next developer to work on the project.

building firefox for the first time

February 18, 2015

recently I came across two blog posts:

They kind of tell similar stories. Both Firefox and Chromium1 are open source and welcome contributions from the general public, but both of them are probably developed primarily by the companies behind them. These posts are sort of encouraging people to consider contributing back to the app they spend most of their computer time in and hoping to make it sound more doable and less terrifying.

I came away from the Chromium post with a lot of clear and friendly knowledge about how I might make a contribution but actually more convinced that I’m not qualified to do so, because I don’t know any of the languages she mentioned in the post and don’t currently plan to learn them. If I did, I’d know exactly where to start and how to proceed.

The firefox blog post was an offer to help mentor a select few through getting up and running and making a first few contributions. I thought about it for a few minutes while I read the blog post and filed it away in the back of my mind, but I was too slow, and the offer has now closed. I thought maybe I could do it because having a mentor helps a lot and, from the sounds of it, the front-end of Firefox is built using the tools of the web, which I already kind of know and work with. From the post:

We will use JavaScript, CSS, and XUL (similar to HTML).

Sure.

Also, Firefox is my day-to-day browser, so selfishly I like the idea of submitting something that would benefit me. I first got excited about the internet in Firefox. I’ve occasionally switched to Chrome and Safari but I always come back to Firefox. I don’t know. And I kind of think it’s having a moment? Well just someone wrote one blog post about it recently:

I like dhh’s perspective there; sometimes I feel like a browser hipster and like I’m the only web developer not using Chrome (I know I’m not) but I do think he’s right that it’s important that any one company doesn’t have too strong a hold on how the internet works.

So tonight I took the first step and built Firefox locally. It was surprisingly easy but also surprisingly time-consuming. If you want to do the same I recommend following the Simple Firefox build instructions but I’m going to share here anyway the steps I took.

  • Visited that guide
  • Followed that link to Linux and MacOS build preparation which recommended I paste the following into my terminal to install the prerequisite dependencies:
wget -q https://hg.mozilla.org/mozilla-central/raw-file/default/python/mozboot/bin/bootstrap.py && python bootstrap.py
  • That asked me if I was interested in building Firefox for Desktop or Android. I answered Desktop, and it started doing stuff, and outputting what it was doing, and ultimately (actually this was pretty fast) it said it worked and made this suggestion:
Your system should be ready to build Firefox for Desktop! If you have not already,
obtain a copy of the source code by running:

    hg clone https://hg.mozilla.org/mozilla-central

Or, if you prefer Git:

    git clone https://git.mozilla.org/integration/gecko-dev.git
  • So I ran the second command, because I prefer git (well, because I know git)
  • And then I waited for like ten minutes? More? I don’t know. I had never cloned a 7 gigabyte git repo before. It felt new and strange. The .git directory is nearly 6 gigabytes, suggesting the sheer history of the project weighs significantly more than the current state of it. The repository has 407088 commits going back 17 years, 8 years before git existed.
  • After the project finished cloning, I ran cd gecko-dev and no magical incantation appeared giving me the next bread crumb to follow, so I returned to the guide.
  • It suggested using Firefox’s ./mach script to build and run the app. This felt kind of familiar, kind of like Ruby’s rake. I tried running just ./mach without any arguments and saw that it has a shitload of commands including google, so I ran ./mach google jake and amir (because I was just watching Jake and Amir episodes before this flight of fancy overtook me) and it opened Firefox to https://www.google.com/search?q=jake%20and%20amir, which, alright, sure.
  • I ran ./mach build and watched it do a shitload of stuff to build Firefox from the source I’d downloaded. I don’t have to estimate how long this took, because it output a running timestamp as it did everything. It took 37 minutes and 56.85 seconds. I guess that’s not bad? I hope I don’t need to wait that long every time I make a change (if I even figure out how to make a change). And when it was finished, I saw this output:
37:56.85 We know it took a while, but your build finally finished successfully!
To take your build for a test drive, run: |mach run|
For more information on what to do now, see https://developer.mozilla.org/docs/Developer_Guide/So_You_Just_Built_Firefox
  • I ran ./mach run and like lightning a bluish moonish earth appears in my Mac’s dock and Firefox Nightly opened.
  • I visited hardscrabble.net just kind of vainly to see what my site looks like in my freshly baked browser
  • I visited the URL from that last output (sidenote: I really love the trail of breadcrumbs this process has been) and found a kind of hilariously brief web page which mainly includes notes on how it should be succinct but also a tag indicating it needs more content?
  • I followed the first listed interesting link about how to run Firefox’s tests, and before I could overthink what’s about to happen I ran ./mach xpcshell-test and the most delightful thing happened, and I think it needs to be captured in a video:

I’m sorry this blog post became a bullet point list halfway through. That wasn’t really fair.

Probably in the morning I’ll forget I ever wanted to contribute to Firefox but hopefully not. I’m on the record now.

  1. Chromium is basically Chrome, but open source.Ā