Adam @ Heroku
a tornado of razorblades

POST Payloads

Posted by Adam Wiggins on November 15, 2007 at 05:04 PM

I often find myself needing to make some sort of an HTTP request (ajax or rest) in which the data being passed by the call is a single uniform block of data. For those situations I've become fond of just passing the data as the payload of the POST, and bypassing the CGI protocol altogether.

For example, if you dig around in the javascript for the Heroku console, you'll see that it sends the command with this code:

   1  new Ajax.Request('/console/command',
   2    { method: 'post', postBody: command, ... } })

The fun part here is postBody: command. This just sends the command (which is presumably some Ruby code) as the raw POST data. Then in the controller, we can pick it up like this:

   1  class ConsoleController < ApplicationController
   2    def command
   3      cmd = request.raw_post
   4      ...
   5    end
   6  end

Clean, easy, and simple. I've found that CGI encoders/decoder can to get confused if you have too many layers of nested quotes and other characters, which is common when you're passing chunks of code around. (Part of this may be caused by different implementations of the escaping - in an ajax call, the encoding is done by Javascript, and the decoding by Ruby.) Passing it straight through as a block avoids this problem entirely.

Plus, you can test it at the command line really easily:

   1  curl http://localhost:3000/console/command -d "Widget.count"

I get the impression that this method may violate some W3C standard, or least some unwritten netiquette law - but so far I haven't found anyone expressing an opinion one way or the other. If you know of or can think of a problem with this method, please leave a comment.

Your first thought might be - what if I need to pass some additional data, such as an id? Well, you can put the CGI parameters into the URL, and leave the POST payload as your uniform data block. To take a completely made-up example:

   1  curl "http://localhost:3000/posts/create?author_id=1&publish=true" -d "Here's the payload."

One other thing you might want to do when transferring binary data is set the content-type of the POST, which is not only more correct, and also keeps Rails from logging the payload (not pretty with a large binary file). Try this:

   1  curl http://localhost:3000/image/upload --data-binary @public/images/rails.png -H 'Content-type: image/png'
Tags: http
Hierarchy: previous, next

Comments

There are 5 comments on this post. Post yours →

Jeremy Kemper

This is totally kosher (and to be encouraged IMO) though you may consider passing a Content-Type header indicating what you're sending, such as application/x-ruby or something.

It's definitely acceptable to throw stuff into the request body like that, as long as it's properly encoded.

To really avoid the encoding issues, you might want to use a multipart/form request, rather than putting the values into the x-www-form-urlencoded body. You can see the difference by making the two requests against a netcat listener in another terminal (nc -l localhost 2222):

curl -d "y" http://localhost:2222/ curl -F "x=y" http://localhost:2222/

but the latter is way harder to setup, particularly in JavaScript.

If you want to avoid including additional parameters in the URL, you can include them in the post body and parse the string yourself. Rails does some extra parsing jumping through hoops, but CGI.parse(rawpost) or rawpost.split("&").map { |v| CGI.unescape(v) } should also do the trick.

the method is very good, but can someone pass anything he want in to the url with this method?

Uh... isn't this a huge security hole? It seems than anyone who can craft a Post could send ruby code to do something bad.

"rake db:migrate VERSION=1" "Dir.entries.each { |f| File.delete(f) }"

But other than that, Heroku sounds really really cool!

Waiting for my invite, JB

@Blaine - Yeah, I've played with multipart for binary blobs (got another post brewing on this topic), but it's annoying because there's no good Ruby library for doing the call on the other end. If it's coming from a browser, of course, then no problem.

@JohnB - It's no more a security hole than anything else you can do on a login-protected site. The user's session is authorized on an ajax call, just like anything else.

Post a comment

Required fields in bold.