I've always liked to build systems with a bunch of small apps that talk to each other through various protocols. Orion and I built TrustCommerce in this manner, and that gave it some pretty impressive fault-tolerance and scalability.

I had heard the term SOA (service-oriented architecture), but had always dismissed it as enterprisey talk. (Bland-yet-pompous three-letter acronyms make my brain turn off.)

At some point, it dawned on me that what I like to do - build small apps communicating with each other over the network - is exactly what SOA means. In its modern incarnation, service architectures use REST calls, which follows the unix tradition of small sharp tools, loosely coupled by a simple but extremely flexible protocol.

Heroku is no one app - even aside from all the server software and configs, our code is currently split among around two dozen apps, each with their own repository, and most with their own database. (Some have no database.) Most of these are Rails apps, though some are bare Ruby on a Mongrel or bare Rack on a Thin, and some are Sinatra apps.

This is why my Railsconf talk was about HTTP routing. When you've got dozens of apps, some of which respond to complex domains (for example, edit.*.heroku.com), a powerful http router outside your application VM becomes damned near indispensable.

Service architectures are the solution to the longtime problem of apps growing to monstrous proportions. Once you exceed a couple dozen models and/or controllers, it starts to be very hard for new developers to grock everything that's going on, and the barrier of entry becomes very high.

With a service architecture, each app has a simplicity that's reminiscent of the "make your own blog" sample apps you see in tutorials. Rails seems to better retain its beauty in this state. Probably everything does.

But it introduces new challenges. You've got dependencies between repositories - any time you change the interface, you have to be sure to roll out the server and client apps together. The relationship between internal apps is thus very similar to external ones - you need versioning and dependency management. Heroku has an app we call our architecture atlas which tracks all the components, dependencies between them, and documents their APIs.

Managing authentication becomes a big job. We do a lot with custom HTTP headers on this (again, one of the main topics in my Railsconf talk), but I've got my eye out for even more sophisticated solutions. OAuth is one that has piqued my interest.

Perhaps the hardest question is where to draw the dividing line between one app and the next. Does a given model go in app A or app B? And that requires a lot of hard thinking about your design. An app should do just one thing, and without having to touch other apps too much. This often means splitting an app apart, and occasionally fusing two apps back together. This is the same process as managing object classes within an app: as each item's responsibility within the architecture changes, code moves around.

I find it useful to think of each internal component as its own service that could potentially be spun off as its own company. If it has its own code repo, database, tests, API, and docs, then turning it into a standalone service would just be a matter of giving it a slick marketing name and putting up a website. It's not that you'd want to do this, mind you: but if your service architecture is designed well, it would be easy to.

Rack is one of the most important developments in the Ruby web space in the past year. I suspect it's been slow to get attention because the benefits are a bit subtle. Witness the Rails core team being confused about Rack just a few months ago. So if you don't get what the deal is with Rack, don't feel bad - you're in good company.

James covered Rack in his Railsconf talk, partially at my insistence. (His talk was about Mongrel handlers, but Rack middleware is a newer and better way to achieve the same end.) It's worth noting that he asked the crowd - a couple hundred Rubyists - whether they had heard of Rack, and almost every single hand went up. But when he asked if they knew what it was for, not a single hand was raised.

So what's the deal with Rack? In short, Rack provides a standard interface between the web app server and the app framework. This is useful in light of the multiplying number of web app servers (Webrick, Mongrel, Thin, Ebb...) and frameworks (Rails, Merb, Sinatra, Ramaze...). A standard not only reduces the amount of code the framework authors have to write, but it makes the layers in the stack more pluggable. Pluggability encourages experimentation (which means more innovation over time), and generally makes the whole stack more robust.

One implication of Rack is that you can skip the app framework altogether. I've always liked using use standalone Mongrels running tiny Ruby apps without a framework for internal daemons. These days, I generally use Sinatra for that purpose - but there's still something cool about skipping the use of any framework and just coding down to the metal.

Want to try it out yourself? Stick this code into hello.ru:

class HelloHandler
   def call(env)
      [ 200, { 'Content-type' => 'text/plain' }, 'hello, world' ]
   end
end

run HelloHandler.new

Then at the shell:

rackup hello.ru &
curl http://localhost:9292/

Congratulations, you've just made a frameworkless Ruby web application in five lines of code.

A Rack handler is anything that can respond to the call method and returns an array with the status code, output headers, and output body. Handlers can be the end of the request chain, or do input and output filtering anywhere in the middle (hence "middleware"). Here's an example from Marc-André Cournoyer. Though his example is presented for Thin, you can run his code on any Rack-compatible server.

In the real world, what is Rack middleware useful for? We recently ported the Heroku toolbar to Rack middleware. The previous implementation was several hundred lines of very hard-to-follow monkeypatching of ActionController, combined with a rarely-used and poorly-maintained plugin framework for Mongrel call GemPlugins. (Which I nominate for Most Confusing Name Ever.) That code was hard to read and nearly impossible to spec, but it's the only way we could make it work with the traditional Mongrel/Rails setup. It was also very tightly coupled to a particular version of Rails and Mongrel.

Ricardo (one of the new Heroku devs) banged out the Rack middle port in just a couple of days. It's a fraction the number of lines of code, and can be speced normally. Plus, our toolbar is now compatible with any Ruby app server or framework.

Because Rack separates the layers of the stack more cleanly, it was way easier to hook the Heroku toolbar code into the right place. Take that lesson and generalize it, and you'll start to glimpse the significance of Rack.

Whew, I think my brain has finally returned from its liquified state after Railsconf. Last year the conference felt like a vacation, since all I did was attend. This year, with all the speaking and booth-manning and meetings, it was pretty grueling.

Despite that, it was still plenty of fun. For example, due to some capacity issues at the convention hall, my talk got moved into the keynote room. So I got to feel like a big shot for an hour up on the big stage with all the banners and lights. :)

Tim Goh posted an excellent blow-by-blow of my talk. Apparently he manually keyed in all the code from my slides during the presentation, which is damn impressive because I was going through them pretty quickly!

We answered about a zillion questions about Heroku at our booth (not to mention just people stopping us in the hallway). This involved lots of waving our arms around and drawing on our plexiglass wipeboard.

(A few more photos.)

Talking to so many people about what we're doing brought a few things into focus for me. The main one is that not many people really get what we're doing. Questions I answered over and over: No, we're not a web-based IDE. No, we're not a reseller of EC2. No, we're not a competitor of Engine Yard.

My best answer to the "what is Heroku?" question is as follows: Heroku is a automated deployment platform for Rails. "Automated" means you don't think about server stuff at all: just load up your code and go. "Deployment platform" means a place to run your app - which could be while you're developing on it, could be a staging/prototype/demo-for-the-client deployment, or could be a full production deployment.

Going forward, the guys and I are going to be thinking about ways to make our message clearer. I guess that's one of the challenges of using a blue ocean business strategy: when you're pioneering a completely new space, explanations are hard.

Thanks to the conference organizers for another great event; to everyone who came to my talk; to everyone who approached us about partnership opportunities; and to all the excellent speakers. And to all the new friends I made, I hope to see you again soon (maybe at Rubyfringe?).

Places you'll find me at Railsconf this year:

• Giving my talk about custom Nginx modules on Saturday afternoon. The talk has evolved quite a bit since I wrote the description, so expect some broader topics, like why I think HTTP is the critical enabling protocol in the era of Rails and cloud computing.
• Attending the Heroku product talk, in which we propose why you may never need to think about servers or hosting again. This is inconveniently scheduled immediately before my session talk, so I'll have to duck out partway through.
• Signing books along with the other recipe contributors to Mike Clark's Advanced Rails Recipes at the Powell's Books booth at the lunch break on Friday.
• Hanging around our booth, where I intend to hack on Heroku and my open source projects, listen in on Geoffrey Grosenbach's podcast interviews, and meet everyone that stops by. So... stop by! :)

During a presentation I gave last month, someone asked what Heroku plans to do about the many different versions of Rails and Ruby, not to mention choices on components like database (MySQL, PostgreSQL, SQLite) or app server (Mongrel, Thin, Ebb).

I answered that this is part of the benefit you get by using Heroku. (In corporate-speak, part of our "value-add.") We select the best components, from the server infrastructure and operating system up to the version of Rails and preinstalled gems and plugins. We make sure it all works together in a complete package, so that you can just dive in and start coding, without worrying about making a bunch of choices before you even begin.

Another company that works this way is Apple. Apple doesn't make motherboards or video chipsets or drives. What they do is select all of the best OEM components and put them together into a beautiful package where all the parts are guaranteed to work seamlessly together. You never worry about your monitor not supporting the sync rate your video card wants to use, or whether you'll have drivers for your DVD writer, like you would with a build-it-yourself PC.

A similar metaphor, but in the software world, is the makers of Linux distributions. This is actually quite close to what we're doing: bringing together lots of open source components into a unified bundle, where all the pieces have been tested as playing nicely together. Given the fast pace of software development and the huge variety of different pacakges, this is quite a challenge - and an ongoing one.

Todd Hoff did a writeup on my presentation, and he mentions the term "curator" to describe companies or people that fill the sort of role I've just described. I think that's a pretty good term - Apple, Linux distributions, and Heroku all fit into this category.

My current favorite distros are Debian on the server and Ubuntu on the desktop, and they both do a great job at this collector role. But I think it's worth looking at another player, Red Hat, to see how this role can be done both right and wrong.

A few years ago, Red Hat was the distro to beat. Red Hat 7, 8, and 9 were all stellar releases. Then they split their distro into two forks: Red Hat Enterprise and Fedora. This seemed like a logical step: provide a slow-moving, highly stable version for production server environments; and a more cutting-edge version for desktops and personal or staging servers.

But this turned out to be a disaster for their market share: nearly overnight, Red Hat went from the most popular distro by far to a distant third. (The move may have been good for their revenue or profitability, I can't speak to that.)

Understanding why this happens helps us understand the role of a good curator. Having a single distro meant that Red Hat had to balance pushing the technology envelope against keeping things stable. In those days they did a great job at finding the sweet spot. Many people complained when they introduced major new infrastructure like glibc2 or UTF8, but in my opinion their timing was exactly right. Sure, it caused some transitional pain for users - mostly proprietary binaries like Adobe Acrobat and Yahoo Messenger breaking - but the longer-term benefit of the technology being introduced, such as being able to easily internationalize all applications, were huge.

But once they split the distro into RHEL and Fedora, they no longer had to maintain this balance. Now RHEL is constantly out of date - causing users to seek out third-party packages or compile things themselves, which defeats the whole purpose of stability. And Fedora is bleeding edge instead of cutting edge, making it painful and unpleasant to work with for anything but pure tinkering.

You saw similar user pain when Apple switched from OS 9 to OS X, or from PPC to x86. Users cried about this stuff - a LOT. (As with UTF8 on Linux, Adobe apps seemed to be one of the centerpoints of user pain.) But once the initial shock wore off - which was pretty quickly - everyone was a lot better off. Can you imagine what Apple's place in the technology industry would be today if they had stuck with OS 9 and PPC?

The curator role is surprisingly challenging to do right. I hope we can learn from the lessons of these companies, and others that curate successfully, as we continue to develop Heroku.