The unix shell (bash) and remote login (ssh) are centerpieces of the server and app deployment process. While building Heroku, however, Orion and I became aware that these tools are pretty far out of step with modern, agile development practices.

I've wanted a Ruby-syntax replacement for the unix shell from almost the moment I began using Ruby. Whenever I can, I write shell scripts as Ruby scripts with lots of backticks. But the "everything is text" mechanism starts to show its age when you end up with Ruby code like this:

my_ip = `ifconfig | grep inet | grep -v 127.0.0.1 | grep -v inet6`.match(/inet ([\d.]+)/)[1]

Yergh. (If you've never had occasion to write code like this in the wild, just check out god's process lookup methods.)

What we really want - the modern way - is to query the unix system (filesystem, processes, network, services) as if they were a database. This avoids the fragility of text pipes, the complexities of firing up a complete new environment on each system call, and would allow unit tests of system-level code.

This is why I've created rush. It's a replacement for bash and ssh which uses Ruby syntax. More than that: it IS ruby. Imagine an irb shell in which you can do everything you can do at the unix command line, but without any backticks. That's the vision; what I've got so far is a good start in that direction.

I said it replaces ssh, so this isn't just a shell: you can use it to control an arbitrary number of remote boxes, using the exact same interface as you would locally. Copy a file, or grep through a logfile, or kill a process - whether the machine is remote or local, the interface is identical.

Unlike the character-based connection of ssh, the rush client connects to the rushd process on the remote server and passes discrete commands. This is very similar to connecting to a remote database. When you run a SQL query, it makes no difference to the programmer whether the connection is a remote box or a local one; the client handles this seamlessly. You can even connect to multiple databases from the same client. rush goes even a step further by allowing you to pass data seamlessly between any number of local and remote connections.

A quick example:

local = Rush::Box.new('localhost')
remote = Rush::Box.new('my.remote.server')
local['/etc/hosts'].copy_to remote['/etc/']

Check the rush website for more examples and to try it out.

One of the inspirations for rush as a shell was this preview of MSH, the Microsoft shell. I get the feeling that this is vaporware (though I don't really know, not being in the Microsoft world at all), but the concepts introduced in the preview really struck home. Treating data returned from shell commands - like file matches from grep or processes from ps - as discrete objecs, rather than text which can be parsed, is the obvious next evolution for shells.

There are some other deficiencies in the bash+ssh model:

• Consistency. Bash is a full-fledged programming language; more specifically, it's a DSL for managing a unix system. But it could also be considered a collection of smaller languages. Standard tools like cp, mv, ps, grep, sed, and sort all have their own unique syntax. You may combine several of these in a single command, which is a bit like mixing several different programming languages on one line. I've been using unix shells on a daily basis for well over a decade, and still I sometimes forget the syntax for a particular command. Compare this to Ruby, or any other modern scripting language, where just a few months of working with the language is enough to teach you 90% of the language's syntax.
• Quoting. Bash commands often have many layers of quoting. Consider:

  ssh remote "rm `grep '^class Thing' lib/* -l`"
  

  This has four layers of quoting: the bash command line on the client, the bash command line on the server, the backticks, and the regexp. This leads to both confusion (do I need one backslash or two to escape this quote character?) and is riddled with security holes.
• Quirks and limitations. Two that I frequently bump into are running out of space in the command line buffer space with backticks, such as:

  grep some_method `find . -name \*.rb`
  

  On a large project, you'll need to rewrite this with xargs:

  find . -name \*.rb | xargs grep some_method
  

  If the directory has filename with spaces in them, you have to use the null separator option on both find and xargs:

  find . -name \*.rb -print0 | xargs -0 grep some_method
  

  Ick. In rush, this would be:

  dir['**/*.rb'].search(/some_method/)
  

• Exceptions. Bash commands have three outputs: stdout, stderr, and the shell return value. Most of the time you're only interested in one and can ignore the others. But for more advanced uses, you need two, or perhaps all three. Explicitly checking for return values (or worse, pattern matching against stderr) is not a lot of fun. Exceptions are the modern way to handle errors.

Go give it a try, and then tell me what you think.

And now for one of my favorite bits of black magic from the unix poweruser's toolkit: ssh tunnels.

Like most good tricks, this one is simple. It lets you bounce TCP traffic through an ssh connection. This is handy in a variety of situations, but the one I've used it most often for is to access a website which is available only inside of a corporate LAN. If you've got external ssh access, you can set up a tunnel that will let you point your browser at a local port to access that site.

The syntax is:

ssh -L [local_port]:[site_you_want_to_reach]:[remote_port] [ssh_host] [command]

This can be a little confusing, but there's actually only two variables that matter. So first I'll fill in the defaults you'll probably always want to use:

ssh -L 9999:[site_you_want_to_reach]:80 [ssh_host] "sleep 9000"

Much better. Now we just need to know the site we want to go to, and the host we will tunnel it through. For example, let's say we want to view RubyInside, but tunneled through your remote server example.com:

ssh -L 9999:rubyinside.com:80 user@example.com "sleep 9000"

You can now browse to http://localhost:9999/ and see RubyInside. (This won't work if the remote site uses named virtual hosts. You can rememdy this by adding the hostname (like rubyinside.com) to your /etc/hosts as an alias for localhost, and then point your browser to http://rubyinside.com:9999/.)

Since RubyInside is public, this example is not actually useful - but now that you see how it works, let's look at how it can be used to view a Rails app running inside a LAN on someone's workstation. Let's say the mongrel is running on port 3000 and that the workstation is devbox.localnet, but devbox.localnet is only accessible from your ssh bouncepoint, which we'll call bouncepoint.example.com.

ssh -L 9999:devbox.localnet:3000 user@bouncepoint.example.com "sleep 9000"

ssh tunnels can also be used to bypass those silly content filters that some companies insist on installing on their local LANs. This trick requires that you be able to send outgoing TCP traffic, and you have a remote ssh server listening outside the LAN. (If port 22 is blocked, you'll need to find one that isn't, and then get your remote ssh server to listen on that port.) By tunneling all your web traffic through ssh, you can bypass any content filters - and also get a completely secure connection that is impossible to easedrop on.

For example, if you wanted to view the latest Penny Arcade comic, but found that the client site you're working on for the day blocks it, you might use your remote VPS host to set up a tunnel like this:

ssh -L 9999:penny-arcade.com:80 vpsuser@vpshost.com "sleep 9000"

One final note: if you're wondering what the sleep 9000 is for. ssh requires a command, and when the command is done executing, it exits. So you have to give a command that will stall. I use 9000 because it's plenty long and easy to type. (When you're done, just hitting Ctrl-C in the terminal that's running the tunnel will close it.) But if you wanted it to last indefintely, you could use while [ 1 ]; do sleep 100; done.