Adam @ Heroku
a tornado of razorblades

In The News

February 27, 2008 at 04:09 PM

Mirko Stocker interviewed me for this InfoQ article on rush.

Comments: 0 (view/add your own) Tags: rush

The Legacy of the Self-Made Man

February 26, 2008 at 01:10 AM

Prior to the industrial revolution, status in most societies was based on one thing only: heredity. No matter how much you accomplished - or didn't - you stayed in the same station of life.

This began to change a few hundred years ago, with the rise of classical liberalism and general shifts in the organization of society. Birthright was still important, but the merit of an individual's achievement could potentially allow them to transcend the station to which they were born. America became the first large-scale experiment of a society that could be called a meritocracy. Rugged individualism became a part of our culture. The cowboy, the frontiersman, the prospector - one man against the primal forces of nature, making his way in the world with nothing but his intellect, hard work, and relentless determination.

This grew to its peak around the turn of the 20th century, with the emergence of what came to be known as the self-made man. A perfect example is Andrew Carnegie. Born into poverty in Scottland, he emigrated to America at age 13 and began working menial jobs. He used each job as an opportunity to learn new skills and accumulate meager savings, thereby springboarding into the next, and slightly less menial, position. By age 18 he had taken on management positions; by age 20 he had begun making investments; and by 25 he had begun operating his own business. Over the next twenty years he grew an empire in the steel market and became one of the richest men in America. He spent the last part of his life as a philanthropist: he eschewed excessive personal luxuries, and preferred instead to give back to the world with what he had made. The quintessential rags-to-riches story; the embodiment of the American dream.

In the meantime, new political philosophies were afoot. Marx and others proposed that individual achievement be deemphasized. Instead, the products of the labor of all members of society should be grouped together. No special reward should go to those who contribute more than others. These ideas came to America in the form of the Progressive movement, championed by Theodore Roosevelt. One of his nicknames was "the Trust Buster," because his administration used government power to break up large corporate conglomerates. This turned the tables on the founders and owners of those corporations - men like Andrew Carnegie. No longer were they the self-made man, embodiment of the American dream. They had been recast as power-hungry, cigar-chomping fat cats; men seized by uncontrollable ambition and thirst for wealth, taking advantage of the capitalism and free markets to crush the average citizen. America watched this drama play out in worker strikes and antitrust legislation and they everyone rooted for the "little guy." Nearly overnight, the self-made man had become the villain.

With the self-made man out of a fashion, a replacement arose in the middle of the century: the man in the gray suit. This is a company man: he plays by the rules, starting at a low position at a promising firm, and gradually building respect from his peers and his superiors by doing his job well, but never attempting to be outstanding in any way. He patiently climbs the corporate ladder at an average pace, for to try to do so faster would be greedy and overreaching. He builds a family, providing for his wife and children, who live in a new type residence: the tract home, which itself is in a new type of urban area, the suburbs. He is respected for fitting in, for knowing his place in the larger mechanism of society, and for not asking for more than his fair share.

Toward the end of the 20th century, the pendulum of fashion started to swing back the other way. Now the man in the gray suit is the object of scorn. He's a conformist; a cog in the machine; a mindless drone, living in the rat race, with nothing unique to offer the world.

And finally, at the very end of the 20th century and the first decade of the 21st, we've come full circle. The self-made man is back: he's now called a technology entrepreneur. Young, hip, ambitious, with just a hint of a rebellious, anti-authority, anti-conformist streak. He makes his mark on the world by dismantling tired old institutions which no longer serve the needs of society. He may have emigrated to America, and perhaps he retains just a hint of an accent from his place of birth. He's got nothing but disdain for large, bureaucratic organizations or the entrenched interests of large groups. But at the same time, he's no lone wolf. He probably teams up with a few close-knit business partners; and he's closely connected to one or more communities of like-minded people. He's loudly opinionated, saying things that upset the status quo, but his articulate honesty causes people to listen. They get the sense that this irreverent-yet-earnest young guy is smart and really tells it like it is.

He and his company might be profiled in Time Magazine or appear on the cover of Business Week. They have legions of adoring fanboys (and, to a lesser extent, fangirls), giving the whole thing an air of rockstardom. He might have a piercing, or a tattoo. Or maybe not, but he certainly never wears a suit, and probably not even "business casual." His clothes say: judge me by what I do, not how I dress.

Who are the new self-made men? Off the top of my head, I can think of quite a few: Larry Page, Sergey Brin, Jeff Bezos, David Heinemeier Hansson, Linus Torvalds, Max Levchin, Steve Jobs, Evan Williams, Joe Kraus, Paul Graham, Steve Shih Chen, Jeff Hawkins. They aren't all in software, either: Martin Eberhard and Chris Larsen are bringing the "silcion valley way" to other industries. Nor are they all men: consider Gina Bianchini, Pam Marrone, and Ryan Phelan.

Of course this is all rather good news for me. I've always fancied myself a future self-made man, and never understood how this fell out of vogue for so long. For me at least, the timing couldn't be better for it to be back - and I don't think I'm alone.

rush Mailing List

February 23, 2008 at 07:10 PM

I've created a mailing list for rush, since discussion in the comments was getting a bit unwieldy.

Comments: 1 (view/add your own) Tags: rush

Test Assumptions, Not Methods

February 23, 2008 at 04:27 PM

Specs should test assumptions, not methods.

How many specs should there be for each method? One per assumption. For example, this method:

def thumbnail_filename
  "images/thumbs/#{basename}.jpg"
end

...has one assumption, so it needs one spec:

it "stores thumbnail jpegs in the thumbs subdir" do
  @image.should_receive(:basename).and_return("test")
  @image.thumbnail_filename.should == "images/thumbs/test.jpg"
end

But this method:

def popular?
  !archived? and recent_views > 50
end

...has three assumptions:

it "is never considered popular when archived" do
  @item.should_receive(:archived?).and_return(true)
  @item.should_receive(:recent_views).and_return(9999999)
  @item.should_not be_popular
end

it "is not considered popular when there are not many recent views" do
  @item.should_receive(:archived?).and_return(false)
  @item.should_receive(:recent_views).and_return(1)
  @item.should_not be_popular
end

it "is considered popular when there are lots of recent views" do
  @item.should_receive(:archived?).and_return(false)
  @item.should_receive(:recent_views).and_return(1000)
  @item.should be_popular
end
Comments: 4 (view/add your own) Tags: bdd

Clarity and Brevity

February 22, 2008 at 11:22 PM

Michael Easter puts it wonderfully:

"Clarity and brevity are often opposing forces, but there is a sweet-spot where they combine to form beauty. We spend our entire careers chasing that sweet-spot. It moves."

I'm Tired of Logging In

February 21, 2008 at 06:58 PM

Client certificates - it's like authorized_keys for your web browser. I've wanted this for ages. Passwords are so lame - insecure, hard to remember, just annoying all around. Even worse is the whole process of logging in; particularly on banks and other financial websites where they make you go through a multi-page wizard. Asymmetric key cryo is what it's all about.

I look forward to the day when we all keep our private keys on a usb dongle, such that we can sit down at any computer and have instant access to all of the sites and services that we use.

Comments: 2 (view/add your own) Tags: ux

Suckage

February 21, 2008 at 06:07 PM

Entrepreneurship is about being able to tell when something sucks, why it sucks, and then visualizing how to make it stop sucking. When something sucks, it's an opportunity for an entrepreneur. The more it sucks, the better the opportunity.

Take cell phones. Cell phones suck. I thought this was obvious to everyone; but at some point I realized that most people don't think of it that way. The fact that they have to navigate ten difficult-to-comprehend menus to get to the most used function on their phone - well, that's just how it is.

To me, that sucked. Apple noticed this too, and had the clout to make a deal with the telcos that gave them enough control to make the iPhone, a cell phone that doesn't suck. As a result, they dominated that market practically overnight.

Paul Buchheit puts it another way: you have to be continually annoyed by things in order to spot good ideas. If you're always content with the status quo, you're unlikely to start thinking about how they could be made better. This doesn't prevent you from becoming an entrepreneur, by the way - but it does mean you need to partner with someone who does have a natural sensitivity to suckage.

This is why you often hear entrepreneurs throw around phrases like "pain point" or "looking for the market with the most pain." Pain - also known as suckage - means opportunity. This is the astonishing effect of capitalism: greed is channeled into motivation to solve other people's pain.

gem install rush

February 20, 2008 at 04:21 PM

rush now has RDocs and a gem.

Comments: 2 (view/add your own) Tags: rush

rush, the Ruby Shell

February 19, 2008 at 01:07 PM

The unix shell (bash) and remote login (ssh) are centerpieces of the server and app deployment process. While building Heroku, however, Orion and I became aware that these tools are pretty far out of step with modern, agile development practices.

I've wanted a Ruby-syntax replacement for the unix shell from almost the moment I began using Ruby. Whenever I can, I write shell scripts as Ruby scripts with lots of backticks. But the "everything is text" mechanism starts to show its age when you end up with Ruby code like this:

my_ip = `ifconfig | grep inet | grep -v 127.0.0.1 | grep -v inet6`.match(/inet ([\d.]+)/)[1]

Yergh. (If you've never had occasion to write code like this in the wild, just check out god's process lookup methods.)

What we really want - the modern way - is to query the unix system (filesystem, processes, network, services) as if they were a database. This avoids the fragility of text pipes, the complexities of firing up a complete new environment on each system call, and would allow unit tests of system-level code.

This is why I've created rush. It's a replacement for bash and ssh which uses Ruby syntax. More than that: it IS ruby. Imagine an irb shell in which you can do everything you can do at the unix command line, but without any backticks. That's the vision; what I've got so far is a good start in that direction.

I said it replaces ssh, so this isn't just a shell: you can use it to control an arbitrary number of remote boxes, using the exact same interface as you would locally. Copy a file, or grep through a logfile, or kill a process - whether the machine is remote or local, the interface is identical.

Unlike the character-based connection of ssh, the rush client connects to the rushd process on the remote server and passes discrete commands. This is very similar to connecting to a remote database. When you run a SQL query, it makes no difference to the programmer whether the connection is a remote box or a local one; the client handles this seamlessly. You can even connect to multiple databases from the same client. rush goes even a step further by allowing you to pass data seamlessly between any number of local and remote connections.

A quick example:

local = Rush::Box.new('localhost')
remote = Rush::Box.new('my.remote.server')
local['/etc/hosts'].copy_to remote['/etc/']

Check the rush website for more examples and to try it out.

One of the inspirations for rush as a shell was this preview of MSH, the Microsoft shell. I get the feeling that this is vaporware (though I don't really know, not being in the Microsoft world at all), but the concepts introduced in the preview really struck home. Treating data returned from shell commands - like file matches from grep or processes from ps - as discrete objecs, rather than text which can be parsed, is the obvious next evolution for shells.

There are some other deficiencies in the bash+ssh model:

  • Consistency. Bash is a full-fledged programming language; more specifically, it's a DSL for managing a unix system. But it could also be considered a collection of smaller languages. Standard tools like cp, mv, ps, grep, sed, and sort all have their own unique syntax. You may combine several of these in a single command, which is a bit like mixing several different programming languages on one line. I've been using unix shells on a daily basis for well over a decade, and still I sometimes forget the syntax for a particular command. Compare this to Ruby, or any other modern scripting language, where just a few months of working with the language is enough to teach you 90% of the language's syntax.
  • Quoting. Bash commands often have many layers of quoting. Consider:
    ssh remote "rm `grep '^class Thing' lib/* -l`"
    
    This has four layers of quoting: the bash command line on the client, the bash command line on the server, the backticks, and the regexp. This leads to both confusion (do I need one backslash or two to escape this quote character?) and is riddled with security holes.
  • Quirks and limitations. Two that I frequently bump into are running out of space in the command line buffer space with backticks, such as:
    grep some_method `find . -name \*.rb`
    
    On a large project, you'll need to rewrite this with xargs:
    find . -name \*.rb | xargs grep some_method
    
    If the directory has filename with spaces in them, you have to use the null separator option on both find and xargs:
    find . -name \*.rb -print0 | xargs -0 grep some_method
    
    Ick. In rush, this would be:
    dir['**/*.rb'].search(/some_method/)
    
  • Exceptions. Bash commands have three outputs: stdout, stderr, and the shell return value. Most of the time you're only interested in one and can ignore the others. But for more advanced uses, you need two, or perhaps all three. Explicitly checking for return values (or worse, pattern matching against stderr) is not a lot of fun. Exceptions are the modern way to handle errors.

Go give it a try, and then tell me what you think.

Comments: 32 (view/add your own) Tags: ruby, rush, ssh, unix

Making OS X Spaces Sane

February 18, 2008 at 01:33 PM

If you use Spaces on Leopard, you should install this fix immediately. It blocks the annoying default behavior of Spaces, which is to try to force all windows from the same application onto the same virtual desktop.

Spaces still isn't quite up to par with standard unix virtual desktops - such as not remembering the stack order of windows when you switch away from a desktop and then return to it. But the fix above makes it substantially more useable.

Comments: 2 (view/add your own) Tags: desktop, mac, ux

Changesets, Not Snapshots

February 18, 2008 at 01:37 AM

It seems that Git fever is going around.

An important concept to keep in mind with decentralized revision control is that the entire paradigm is one of changesets, not snapshots. Subversion and friends take a snapshot of your source tree and save it. Branching and merging is thus quite painful, because it's hard to tell the intentions of the code authors when all you have is a before snapshot and an after snapshot.

A changeset is a single commit - mostly the patch (diff) of the code, but also some metadata about the patch, such as the parent node in the commit tree. This is what allows git and friends to seem smarter (in fact, nearly omniscent) when it comes to merging. It isn't about a better merge algorithm; it's about well-formed data.

Developing in decentralized revision control should lead you to think about your commits in a different way. Each commit should be a single, atomic bundle. If you find yourself thinking "Huh, I should commit - it's been a while since my last one" - that's wrong. Your commit will mostly likely contain several changes, and thus not a discrete changeset.

Imagine that each commit were a patch that was going to be emailed to the maintainer of the project, or posted in Trac. If you sent a patch that had two unrelated changes in it, the maintainer would send it back and ask you to separate them into two patches. Think of your commit tree the same way. Each commit should stand on its own merit, isolated from other changes.

git provides tools to keep your commit history well-formed. git stash is one of my favorites. But the crown jewel is git rebase -i HEAD~10. If you haven't used this yet, try it at your next opportunity. It loads your last 10 commits into a text editor and lets you manipulate them in a freeform fashion.

The "squash" option is particularly nice. If you've ever had a commit log that looks like this:

r50 | adam | amazing feature that works perfectly
r51 | adam | oops, it works now
r52 | adam | oops, one other thing
r53 | adam | ok really this time

...you can use squash to combine these all into a single commit.

You'll understand better how this can be possible when you internalize the fact that commits and pushes are not the same thing. A commit indicates that you have completed a single, atomic change. A push publishes your changes out to others on your team, but is not a commit event in and of itself.

Work and Play

February 12, 2008 at 01:32 AM

A chapter in Virginia Postrel's book Dynamism asks the question: What classifies something as "work", and other things as "play"? This is one of those trick questions which seems like it has an easy answer. The first thing that pops to mind - "You get paid to do work" - fails right away. I don't get paid to do my dishes at home, but that's definitely work. I do get paid to build software, but quite often a lot of that feels like play.

The book is excellent, so you should read it; but I'll spoil the answer for the chapter in question: work has specific goals to be achieved, while play is freeform. If someone hires me to build the a web application, it's work because there are requirements and deadlines to be met. Perhaps I'm given a fair amount of creative autonomy, but when we get near the end, some sort of basic goals (defined by the person forking over the cash for the project) must be met.

I might, however, build a very similar app as a personal project; and this would qualify as play because I'm responsible for nothing but pleasing my own curiosity. I can take a six month detour learning a new language or screwing around with a different visual style. I can decide halfway through that I don't want a web app after all, but instead a web service, or maybe I really just wanted to paint a painting.

Why is this important? We know that most people are happiest when they have the right balance of work and play in their lives. But play serves an important purpose: it's a chance to experiment, to learn new skills and techniques, to develop new patterns and approaches. This then feeds back into our work, because we can take the most effective techniques discovered during play and put them to work achieving the more concrete goals of work.

As developers, this is pretty obvious. If we focus all our energy on work projects, that means we don't have time to try new technologies or new techniques. When you're focused on a goal, you don't have time to tinker. But tinkering is how we learn. New technologies are rarely a swap-in replacement. If you tried to use a non-relational database in your next web app, that means you have to rethink the role of schema, migrations, queries, and validation. Sometimes there are direct equivalencies for commands, and this is how people new to the technology tend to use it. That's why Ruby written by someone who just came from PHP tends to read like PHP, and why someone using decentralized revision control for the first time will tend to stick to the commands and techniques which closely match how things worked in a centralized model.

At my last company I did a lot of hiring, and one of the most important questions I asked people was: do you have a personal project? If the answer was yes, I wanted to see it. The candidate was usually reluctant, because they thought of any such project as being something silly or unimportant. After all, it was just for play. One person I hired was a fanatical watcher of the show Smallville, and had built an imdb-like cast and episode tracker. She was reluctant to show it, but it impressed the hell out of my and we hired her. Another candidate showed us a site he had built that cataloged downloadable PC game demos. The site was very amateur in its visual style, but he had built his own in-place editor framework that was quite impressive, considering that this was pre-Rails and pre-ajax. We hired him too.

At the time I thought that I was so interested in personal projects because they showed personal motivation, the ability to self-manage, and a love for software development that extended beyond just the potential paychecks. Those things are true. But thinking about Postrel's definition of work and play, and the importance of play in learning and personal growth, I now see what is possibly an even more important reason to ask to see a candidate's personal project. Someone who pursues personal projects is going to keep learning, trying out the up-and-coming technologies and figuring out how to integrate them into their work process. Someone who is 100% focused on work work* all the time is fated to have their skills slowly fall out of date.

* That's not a typo: saying words twice in a row changes their meaning. I believe this was pioneered by teenage girls, as in, "Do you like him, or do you like like him?"

Minimalist RSpec Matching

February 07, 2008 at 07:08 PM

I've been thinking about the "too much magic" problem of RSpec. There are two sides to this: too much magic in its internals, and too much magic in its matcher syntax. These are related. Shoulda and test_spec are popular because people like asserts. You never forget the syntax for assert_equals - at least, once you can remember the order the arguments go in (I always want to write the value first and the expected value second).

Lately I've made peace with this by limiting my use of RSpec's matchers to just a couple of syntaxes:

var.should == 'expected'
var.should match(/pattern/)
var.should be_true   # or be_false
lambda { ... } should raise_error(SomeException)

These are easy to remember and they read nicely. var.should == 'expected' is nearly as good as assert_equal in its simplicity, with the further benefit that the expected value goes on the right, which I find more natural.

Very occassionaly I will use var.should > 0, but that's about it. I also rarely find much use for should_not, except sometimes against a match. All of the other matchers I now avoid. Here's a conversion chart from the 'standard' RSpec way to Adam's minimalist approach:

RSpec Full Minimalist
item.should be_kind_of(Item) item.class.should == Item
file.should be_exists file.exists?.should be_true
cart.should have(3).items cart.items.size.should > 3

The argument against my approach is that it gives RSpec slightly less information about what's failing. My experience so far has been that it's been more than offset by the peace of mind of having less to rememeber. Actually, the peace of mind is because I no longer have to mentally scold myself every time I forget the right matcher to use.

I also feel that val.should == 1 has a distinctive look that's easy to scan a page for. Not quite as good as assert, but close.

Comments: 0 (view/add your own) Tags: bdd

Theory vs Practice

February 02, 2008 at 11:47 PM

Specs (or tests) show that your code works in theory. A running app in production shows that your code works in practice.

Put this way, the obvious question becomes: what's the point of BDD? Working in practice is what matters.

Actually, both are equally important.

I've worked on (and created, in my less-enlightened past) lots of apps that are thrown together collections of PHP pages, ad-hoc daemons, and so on. These apps work in practice, but not in theory. They've been jury-rigged and duct-taped into working in practice, but when the first earthquake comes along they fall apart.

Another way to state this is:

  • Code that works in theory is code that works by design.
  • Code that works in practice only is code that works by accident.

Code that works in theory may not work in practice - only production deployment can tell you that. An app which has a large suite of running specs, but no users, is really no better than a non-speced, jury-rigged / it-works-if-you-just-don't-jiggle-it-too-hard app. Only when it has been proven in both theory and practice is your app truly sound.

HTML Paste

February 01, 2008 at 09:29 PM

When using web-based editors like GMail or Google Docs, I often grab something from a web page or other document and paste it in. In almost every case, I just want the text - no formatting. Perhaps I'm pasting an email address, a web url, or a longish phrase that I don't want to manually retype.

Using most of these editors, the formatting information is carried across. This is a good thing in some circumstances, but in most cases it's quite annoying. Pasting a name into an email and have it be a slightly different typeface, size, and weight from the surrounding text just looks bad.

The solution I've been using is to pop open a blank TextMate document (on the Mac) or a gedit window (on Ubuntu), paste the text in, then re-select it and copy it again for paste into the final location. This is pretty lame though. Anyone have a better technique? Maybe operating systems should start adding a context menu item, "Paste as Plain Text."

Comments: 2 (view/add your own) Tags: desktop, ux

Shifting Resources to Higher Value Uses

February 01, 2008 at 06:13 PM

The following quote is often attributed to the economist Jean Baptiste Say, writing in the 19th century:

"The entrepreneur shifts economic resources out of an area of lower and into an area of higher productivity and greater yield."

(I say "often attributed to" because I was unable to find the original source. I first saw it in The 80/20 Principle, and have since seen it in other works.)

I love this line, because it's a perfect definition of the word "entrepreneur," without saying anything about founding a company, running a business, or making money. Entrepreneurism is an approach to the world, not an occupation.

You can be entrepreneurial in the most mundane of situations. Let's say you're out taking a walk with your spouse. They underdressed and are shivering. You overdressed and are carrying your jacket in your hand, since you're too warm to want to wear it. If you moved the jacket (an economic resource) from an area of lower productivity (your hand) to an area of higher productivity (keeping your spouse warm), you've just acted as an entrepreneur. Both parties end up wealthier, in the broad sense of wealth == happiness.

Although entrepreneurship is usually about making trades that make all parties involved better off, charity is also a fundamentally an entrepreneurial activity. When you give $20 to a charity to help the poor, it's because you believe that that $20 will be more valuable to a poor person than to yourself. This is why there are no charities for billionaires: a billionaire will most likely value that $20 much less than you do.

Just a note on the etymology: entrepreneur comes from French, where entreprendre is the verb for "to undertake," or perhaps simply "to do." So entrepreneur means "doer." Makes perfect sense to me, even if it does make it a pain in the ass to spell.