Polling for updates is bad. We’ve known this for about as long as computers have existed. So why are so many web-based services (SUP, RSS and Atom feeds, Twitter, etc.) based around polling?
The answer lies, first and foremost, in the asymmetry of HTTP. The web is split into two pieces: programs that make requests, and programs that handle them. It’s very rare to see a single program behaving as both a client and a server.
To fix the asymmetry, we need to be able to act as if being able to respond to HTTP requests was within easy reach of every program, so that we can notify interested parties of changes by simply sending an HTTP request to them. This is the core idea of web hooks.
We need to push the messy business of dealing with long-polling away from the core and toward the edge of the network, where it should be.
We need to let programs dynamically claim a piece of URL space using a plain old HTTP client and handle requests sent to URLs in that space using that same HTTP client.
Once that’s done suddenly asynchronous notification of events is within reach of any program that has access to an HTTP client library, and protocols and services layered on top of HTTP no longer have to contort themselves to deal with the asymmetry of HTTP. They can assume that all the world’s a server, and simply push and pull content to and from whereever they please.
Tunnel HTTP over HTTP, in a structured, controllable, securable way. Let programs claim part of URL space, and serve HTTP, all by using an ordinary HTTP client library.
Why hasn’t this been done before?
This isn’t a completely new idea, it seems, though it doesn’t seem to be widely used — yet.
At around the same time as I was writing ReverseHttp (i.e. May 2008),
Donovan Preston at Second Life was writing
up his take on the same idea, which he also called Reverse
“PTTH”). His idea is to use the HTTP 1.1
Upgrade header to switch
the direction of the protocol, which works well for non-browser
environments. He also has a Comet-based solution very similar to
mine, except that it uses JSON
objects for describing the HTTP requests and responses where mine uses
the actual HTTP message formats. Donovan Preston writes more about
the Second Life variant
and has, with Mark Lentczner, produced an
Upgrade-header based protocol.
Update: I’ve only just now (August 2012) discovered Eric N. Sit’s SB/MEng thesis from 2000, “Reverse HTTP tunneling for firewall traversal”. This is the earliest presentation of the idea that I’m aware of. I think it’s pretty neat that Sit was emphasising firewall traversal, which is one particular situation where enrolment is really important.
Download and links
Instructions for downloading the implementation are here
There’s lots of thinking from others along the same lines:
Up until mid-2010, there was a free demo ReverseHttp service running
reversehttp.net, but I have since taken it offline because the
traffic bill was getting too high. If anyone would like to sponsor or
host the publicly-available service, or would like advice on
configuring a service instance of their own for their own use, please
Please see here for more information.