Rubyfringe was great. Off-the-beaten-path topics, small size, tons of style, and countless small touches made it stand apart from the typical tech conference. (They included a Rubyfringe-branded condom in the swag, for gawdsakes.)

Quick summaries of some of the talks follow. Warning, these probably reflect my own disposition as much as that of each speaker.

• Dan Grigsby - Don't work for the man, be a programmer/entrepreneur instead. Treat each venture as an experiment and don't be shy about terminating the ones that don't work. You'll strike out a lot but eventually hit a home run.
• Yehuda Katz - Living on the edge is fun, but also dangerous. Finding a balance between the cutting edge and the bleeding edge is the trick.
• Luke Franci - There are methods for testing other than code-oriented ones like unit testing. RCov doesn't mean much. Use QA, usability tests, and code reviews for well-rounded coverage.
• Obie Fernandez - To be successful at consulting, watch the balance of power in the relationship with your clients. Most Rails freelancers are charging too little - he recommends no less than $100/hr.
• Matt Todd - Don't be afraid to dive in with both feet. Make mistakes, learn from them. Pick good problems to solve.
• Jeremy McAnally - Frameworks are getting fat. Frameworks should be specific. Don't use Rails (or any other framework) outside of its domain.
• Hampton Catlin - Javascript is a good general purpose language, but terrible in the browser. Add-on libraries like Prototype and JQuery are just band-aids on this problem. The client-side language should be tightly coupled to the DOM, like CSS. He proposes Jabl.
• Giles Bowkett - Fuck the man, fuck the mainstream. Programming is a tool to do your art. His art is music, his tool is Ruby, and the result is Arcaeopteryx. (Giles stole the show with this talk. He ran like twice as long as his alloted time but we were all so into it that no one minded.)
• Damien Katz - If you want to become the guy that gets paid to build cool things, take a risk: start by making something cool without any plan for how to get paid for it. (He did that, and now IBM pays him to work on CouchDB and contribute it to the Apache Foundation.) Also, Erlang is sweet.
• Reginald Braithwaite - It's too bad Ruby isn't more like Lisp or Haskell.
• Tom Preston-Warner - The scientific method rules, use it for coding (and life). Ruby 1.8 has weird memory leak bugs. Github now has a git-powered pastie site, gist.github.com. Also, Erlang is sweet.
• Blake Mizerany - Sinatra is a framework for fast, small web services. Routers are unnecessary obfuscation; treat a web resource url like you would a function name, and address it directly.

The overarching theme of the conference seemed to be that Ruby's steady march into the mainstream means it's losing its luster for us, the early adopters. Ruby was once fringe, but now it's not. We're now all in seek of the fringes of Ruby, and of software tech in general.

You'll note the use of "fringe" as an adjective. A unique culture seemed to emerge from the attendees over the weekend, and using "fringe" this way - as in, "Dude, that is so fringe" - was one trait of that culture.

It's a great term. The fringe refers to all the places where the weird, interesting, chaotic experimentation goes on. This is the spawning ground for tomorrow's new hotness, but the fringe never looks like much when looking at it from the mainstream. Why do I need a new database paradigm? SQL serves just fine, thank you. What about a new web framework, a new object relational mapper, a new transport protocol, a new language? Rails, ActiveRecord, HTTP, and Ruby also serve just fine. CouchDB, Sinatra, DataMapper, XMPP, and Erlang are on the fringe, along with countless other even less well-known projects. These things are not just weird in how they work, they're weird in that they solve problems that most people don't even know they have. That's the fringe.

"Fringe" is more descriptive than the more commonly used "cutting edge." Cutting edge implies a one-dimensional graph, as if tech is on a single well-charted path toward an ultimate destination. Put that way, who wouldn't want to be on the cutting edge, meaning you're further down that inevitable path?

But that's not how it is. The state of tech, charted over time, is an N-dimensional graph. The fringe is the ragged edges of that graph, the weird bits hanging off the edge. Weirdos with weird visions doing weird stuff that few besides them understand. 99% of this weird stuff never turns into anything. But some tiny fraction turns out to be a new direction for tech, the next big thing, the new hotness, a revolution. In this, it has more in common with biological evolution than with the design and planning we associate with the works of man.

Early adopters crave being in the fringe. We love the chaos, the freedom for wild experimentation, the cognitive challenge of trying to predict which bits of this massively heterogeneous mess may turn into something world-changing. It's not the best place to be from a practical standpoint: proven tech from the mainstream is what you want for getting "real" work done. But there's a satisfaction you can get in the fringe that can't be found anywhere else.

rush 0.4 released. Some changes:

• Rush::Box#processes returns a ProcessSet, for syntax like this:

processes.filter(:cmdline => /mongrel_rails/).kill

• Daemonize shell commands: bash 'some_daemon', :background => true

• Pass args to rush on the command line to execute a one-off, like this:

$ rush 'processes.filter(:command => "ruby")'

Details and rdocs.

The healthcare industry is starting to see benefits from being honest when they make mistakes. Denying that doctors and hospitals ever screw up has been the historical approach to avoiding malpractice suits; yet by being more honest, hospitals are seeing a decrease in such lawsuits.

If doctors can do it, why not software authors? Own up to your mistakes. People will trust you more in the long run.

Although I've been a believer in TDD/BDD for quite a while, rush was the first time I started a project and said, ok, THIS time I'm going to get really serious about it. My very first commit was a bunch of empty specs, and since then, I don't think I've committed any new feature or even a bugfix without an accompanying spec.

A few weeks into the project, I had made really substantial progress and had quite a lot of functionality. One day, while using it to access a remote machine to do some file operations, something very surprising happened: I typed a certain command and it didn't behave the way I expected it to.

I figured I must have a typo or something, but double-checking it, I realized that, no: my command was correct, it was the program which was behaving incorrectly. I had found, it seemed, a bug.

I felt a sudden sense of disorientation and panic. The program wasn't behaving as expected? What should I do? Log things? Run a debugger? Just squint at the code for a while and see if I could spot the problem? These methods seemed so...crude.

And yet, I realized, these techniques are the very ones I've been using my whole career, that I use every single day to get work done. I don't give them a second thought. Yet, after two weeks of doing pure BDD, the idea of spending time dealing with a bug seemed foreign and painful.

I ended up tracking down the problem and discovered that, despite my nearly 100% code coverage (as reported by RCov), I had a fork in a single-line conditional that was not speced. Upon realizing this, another feeling washed over me: a sense of mistrust in the project's codebase, because a fragment of one line of code did not have spec coverage. I wrote the spec, confirmed that it failed / exposed the bug, and then fixed the bug itself ten seconds later. Relief flooded over me: things were right with the world again.

Nearly all developers, myself included, spend most of our time in that state: not quite trusting that the code all works. I only had the chance to realize how stressful and unpleasant this state is because I lived outside it for a little while.

This experience also told me something else: after several weeks of intense development, this was the first time I had encountered a bug. I feel absolutely certain that that's never happened to me before.

You could argue that rush was not then (and is still not now) a very big or complicated program, but I don't think that's significant. First, by this time I was already using it to control a remote server (i.e., it was communicating commands over the network via rushd); that's the sort of thing that is highly prone to bugs. But more importantly, even very short and simple programs often have bugs lurking in them.

My whole life I've assumed that bugs are a given in software, and that trying to eliminate them completely is a waste of time. And I think I was correct, for the traditional, non-BDD approach. ("Code-driven development," I guess?)

But it may be that a very disciplined approach to BDD means the opportunity to have truly bug-free software. Just imagine what it would be like to have all the time you spend debugging code go into writing it instead. Granted, a lot of that goes into writing specs; but for me, writing specs is far more fun than debugging. It's programming, not sleuthing.

Bug-free software is a very bold claim, though it does require a somewhat narrow definition of the word "bug." A bug is a situation where the code has been specified to behave one way, and instead it behaves another. It does not include things that users or developers may like the software to do, but that it is not specified to do currently. It also doesn't include external depedencies (libraries, web services..) behaving in an unexpected manner - good software should try to recover gracefully from failures in external services, but doing so is a type of feature.

Look back at that definition of bug again: "...the code has been specified to behave one way..." BDD is the very act of writing executable specifications. In non-BDD, there is no specification; so by definition, software written that way can be said to either have an infinite number of bugs, or perhaps just no functionality that you can rely on.

Cloud computing is on everyone's minds, because it offers the promise of infinite horizontal scalability. But to achieve this, we have to change how we build applications.

One such change is how we use the filesystem. The filesystem is unix's database. "Everything is a file" has served us well for decades, and that concept will continue to be critical at the systems layer. But at the application layer, it's time to stop treating the filesystem as a catch-all dumping ground, and start treating the data we store there in a more structured way.

An app's main use of the filesystem is sourcefiles. What qualifies as a sourcefile? Your code, sure - Ruby, ERB, HTML, Javascript, CSS, specs/tests, rake tasks. But also, small static assets that are part of the application's interface, like public/images/top_left_gradient.png and public/robots.txt. If you check it into revision control, then it is probably a sourcefile.

Other than sourcefiles, what do we stick on the filesystem? PIDs and logfiles come to mind. Anything that it is in tmp or log. This stuff is not source, which is probably why it's in your .gitignore. In my opinion it should not be in your application's directory structure at all.

How about user-uploaded assets, like profile pictures? attachment_fu offers a filesystem backend, which shoves files into your public/ dir. But these are not source - it's application data. It has more in common with the contents of the database: data specific to a particular installation of the app. Putting this data into your source tree is confusing.

More significantly, it greatly complicates the problem of scaling.

The correct solution, in my opinion, is to forbid access to the source tree by the web app. Temporary files can be offered through Ruby's Tempfile interface, with the understanding that files thus created are not accessible beyond the lifetime of the request being served.

Logs are a whole other challenge. I'm not a big fan of logfiles; there are better solutions to the logging problem, which I'll write about some other time. In the meantime, logs should go outside the code tree, some sort of /var-style location which can be cycled or thrown away as needed. This location could be write-only for the app; it pushes things in, but it can't read them back or otherwise access it once written. A one-way channel, ala syslog.

As for attachments, asset stores are the correct solution. attachment_fu's :storage => :s3 backend, for example. Storing in the database is reasonable, though I've always found a lot of frustration in trying to store large binary data in the database. Apps on Heroku can use the :storage => :heroku attachment_fu backend.

As we continue to explore the next generation of application deployment, I think we're going to bump into a number of ways to structure apps differently in order to make them scalable. There will be some transitionary pain with these changes, because structure implies restrictions. Many PHP developers coming to Rails have complained about not being able to access sessions from models, or write SQL in your view. MVC creates restrictions, yes, but those very restrictions are what provides the structure. Coming from an unstructured environment, those restrictions may seem cumbersome or arbitrary; but once you're in the habit, you come to appreciate the structure they create.