The virtual resource market comes a step closer

I first came across the concept of a virtual resource market at the Grid Today conference in 2004 when Steve Yatko (who later became my boss) made his keynote presentation on ‘Service Oriented Computing’. Apparently we were not alone in thinking about this convergence of grid computing (as we called it then) and economics, with JP Rangaswami and Sean Park discussing similar concepts back in 2002. A few years later I found myself presenting on the same topic at OGF20, and a short while later Steve, Vlad and I applied for a patent (which still remains pending).

It was only a couple of weeks ago that I was discussing how soon this would come to pass with a friendly cloud startup CEO at IGT2009, and he thought it was some way off. He can join Simon Wardley in the camp that thought this would happen but take longer. I personally was always optimistic that this would happen sooner than later, though I must confess to some recent confusion over Amazon’s capacity management for AWS – just what is ’spare’ capacity, and what happens when Amazon itself is busy (in the run up to Christmas)?

Today saw those questions answered as Amazon announced the introduction of spot instances, which allows Amazon to auction EC2 capacity (in addition to the existing on demand and reserved instance prices). Right now the market is closed[1], and people can’t sell their reserved instances to others, but it’s reasonable to expect that these things will come to pass. I’m not sure at this stage whether the VRM will become a reality first in the Amazon public cloud (and I guess I’m with Simon on thinking that there’s a great opportunity for them there) or in the private data centres of large enterprises (which is the problem we were looking at 5 years or more ago); but as of today we’re a whole lot closer to that vision.

[1] and there’s some speculation about transparency – do different people see different prices?