Fine-grained tail of the Mechanical Turk

I'm constantly being told web services need to be coarse-grained if they're to be successful. And then along comes something like Amazon's Mechanical Turk, which seems to be interesting (and potentially very successful) precisely because it calls on very finely grained units of work. Obviously this coarse-grained vs fine-grained distinction isn't a simple black-and-white matter.

The Amazon Mechanical Turk is the latest addition to the Amazon Web Services portfolio. I've described it in more detail in my ZDNet blog (see Wetware as a service), but in essence what it does is provide a programmatic interface to the individual brainpower of Amazon's registered users. Developers who need a program to perform an action that computers don't do very well — for example, writing descriptions or evaluating images — can instead have their program hand off the task to the Mechanical Turk, pay a fee, and just wait for the answer to come back. The Amazon service takes care of advertising the task to users and paying them on successful completion.

The sort of tasks the Mechanical Turk performs are small tasks that may take only a few seconds for a human being to perform. What's unique about the Amazon service is that it puts those tasks in front of a huge pool of people 24 hours a day, any one of which might decide to complete the task. Humanity (or at least that part of it that holds an Amazon account) becomes the back-end resource that powers the service (indeed, there are ominous shades of The Matrix here, as one slashdot contributor memorably pointed out last week).

Having this huge pool — along with Amazon's payment processing system to commercialize it — makes it possible to parcel out extremely finely grained tasks in the expectation that they will mostly be performed within, say, a few days. That's quite enough for many tasks, especially when compared to the costs and timescales of conventional ways of getting this kind of work done, which might involve hiring temporary staff or redeploying existing resources.

Of course, the tasks themselves are finely grained but you might argue that the Amazon service itself is quite coarse-grained, in that it does take care of a lot of the logistics required to make its pool of human intelligence available in this way.

Another way of looking at this is to highlight the 'long tail' dimension of Amazon's service. It only works because of the way that it aggregates many small offers of work and puts them in front of an aggregated pool of many small potential offers of labor. Perhaps that's the reason why the fine-grained unit of service is so much more common in 'Web 2.0' applications than it is in conventional enterprise SOA applications (Google's aggregated offers of advertising across aggregated partner sites is another example). These Web 2.0 enterprises have long-tail economics at the heart of their business models, whereas traditional enterprises do not.

But perhaps it's a mistake for the latter to concentrate only on coarse-grained service-enabling of existing operations and processes in their SOAs if it might also be possible to build new revenue streams from hitherto undiscovered fine-grained long-tail opportunities within their internal or external relationships. Successfully monetizing these long-tail resources has all the appearances of being a highly disruptive innovation. Those are always the scariest and riskiest, but they are frequently also the most lucrative innovations.

posted by Phil Wainewright 5:05 PM (GMT) | comments | link