Originally post on the Intergen Blog

I’ve been spending a bunch of time of late thinking, working and talking about Cloud Computing platforms. There’s plenty of smoke and even a little fire too.

If you’ve been listening to the IT media for the past 12 months: Cloud Computing means many things to many different people but I tend to list 3 key attributes of most Cloud Platforms.

1. Elasticity
   The ability to turn capacity on and off as and when required
2. Infinite Scale (or the illusion thereof)
   The ability to scale up/out your solution as much as you want. It’s illusory in that Microsoft or Amazon do have a limited data centre resource, but it’s unlikely your application will be able to come even close to saturating it.
3. Pay for what you use
   In the cloud you only pay for the user accounts, or processor hours, or gigabytes of storage/traffic that you actually use. To a degree this goes along with elasticity- though the level of granularity is key.

From the various vendors there are Cloud Platform offerings with differing degrees of what I tend to think of as vertical integration.

• Software as a Service
  These are the likes of the Microsoft BPOS offerings or Salesforce.com. They’re fully packaged services- rock up with your credit card and you can have a seat of Salesforce or an Exchange mailbox. Your unit of scale is basically a user account- have as many user accounts as you want for as much scale as you want.
• Infrastructure as a Service
  At the other end of the spectrum are the big virtual machine clouds run by people like Amazon.com (EC2) or Rackspace (Mosso). Your unit of scale here is a Virtual Machine (complete with operating system). I also group things like Amazon S3 and Azure Storage in here. You can have as many virtual machines as you want in EC2- but, remember that scale doesn’t come for free, you’re still going to be looking after your operating system; keeping it patched and so forth.
• Platform as a Service
  This is the greyest of the categories. We’re talking here about platforms that provide you with some sort of abstraction atop multiple servers allowing you to scale out (elastically). The sorts of things that fall in here are Windows Azure Compute, Google Gears and Microsoft Live mesh. Arguably Salesforce, CRM Online and SharePoint Online fit in here to a degree too. The unit of scale here is a more abstract concept- these platforms generally require you to build a scale ready application from the outset; by taking this hit up front you get the benefits of not having to worry about things like the underlying operating system management.

So if that’s our taxonomy why do I say that elasticity is the key for early adopters?

Elasticity is the one thing that the cloud vendors can do that you can’t achieve in any other way. Being able to efficiently support users who want to turn things off an on all the time requires both scale and diversity. Scale, because you need to have a large enough customer base that a user turning off a hundred or so machines represents a mere fraction of your total capacity rather than the vast majority of it. Diversity, because you want to have different workloads with different peak demands running on your gear.

The parallels here with more traditional utilities should be obvious. The reason that the electricity system works is scale and diversity. Save for some very large industrial customers, who tend to have different contractual terms, dropping 90% of the demand for a given customer really doesn’t make a big difference to a utility. Their scale is such that individual customer load changes tend to wash each other out. Yes there are national changes in demand over the day but your power company turns capacity on and off in large (and relatively efficient to start/stop) generating units. One of the reasons that this model works is that the utility customers are a diverse bunch- if their entire customer base were residential customers, all of whom cooked roast lamb every night and all at 6pm exactly then the model wouldn’t work quite so well would it?

We’ve got a pretty nice data centre setup at Intergen. We can run applications very efficiently- we have a PUE of 1.62. We can run applications at great scale- need 1000 cores on a 12 month contract we can do that. We can keep you going 24/7/365- we’ve not had an outage since we moved to our new Terrace premises. What I can’t offer you though is 1000 cores for 4 hours one morning because you’ve got a big online sale happening- we simply don’t have the scale nor the customer diversity to make that happen. I’d hazard a guess that there isn’t an internal enterprise data centre in the world geared up for this sort of request. This is the sweet spot of the cloud vendors, particularly the Infrastructure/Platform as a Service people.

It basically comes down to your periodicity and differential of demand. If your application needs 100 cores during working hours and 10 outside working hours you’re a good candidate for the cloud but other options might suffice. If your application needs 10 cores for 360 days of the year and 100 cores for 5 days in that period then my submission is that cloud is your only sensible option here. Highly elastic workloads can only be cost effectively delivered through cloud computing.

So a post that came through on RSS this morning from Ryan Dunn got me motivated to jot down some notes on a topic I’ve bee thinking about for a while. How best to deploy into Windows Azure. I’ll caveat this post with the fact that at the moment the Windows Azure deployment story is fairly simplistic- I don’t think we’ll be deploying via the web portal come RTM.

So in Windows Azure you basically have two types of account, Storage and Compute. Within a compute account you effectively have  two sets of instances that are grouped behind two URLs. These are referred to as ‘Staging’ and ‘Production’. So you might have two URLs that look something like this:
http://foobar.cloudapp.net
http://<guid>.cloudapp.net

For all intents and purposes there is no difference between the instances that sit in behind these URLs. Basically the Azure Load Balancers are just pointing requests at the two different groups of servers. When you then push the ‘big blue button’ to do a production release you are basically just re-configuring the rules on the load balancer to swap the URLs around. This leads me to my point from the post title. In Windows Azure the idea of Staging really is much more akin to the military idea of a staging area. It’s the place where you marshal the troops ready to charge over the trenches.

The ‘Staging’ deployment in Windows Azure basically gives us an area to marshall our instances before we make them live on the production URL. It means that we can achieve a Zero downtime deployment model because the machines are warm when we start routing traffic to them. It also means that you might not be able to  (or probably shouldn’t)  treat the Staging Deployment like you might traditionally treat a staging server in a more traditional on premise deployment. In an on premise staging server you’ll usually take your application, deploy it into staging, test it (under limited load) then redeploy it into production- this is a very different model.

Looking at the image above you’ll see I have 1 instance in my Staging Deployment and 2 Instances in my production deployment.  If I do a swap of these then we’ll end up with 1 instance in production and 2 in staging- basically halving the capacity of our production system. This would be bad.

You should use your Staging deployment as a true staging area. You should configure it to run the same number of instances. You should give it the same configuration settings (where possible given code/data change vectors). In this way when you flick the switch and route traffic at those warm servers you’ll be ready to rock at full capacity.

What does this mean for your Windows Azure development lifecycle? Here’s how I’d configure my accounts for a real world scenario aiming to achieve zero downtime upgrades. I’m going to *gloss over* the considerations of upgrading your data layer and leave that for another blog post.

I’d configure myself with 3 storage accounts and 2 compute accounts for my application.
foobarDevStorage, foobarStageStorage, foobarProdStorage
foobarStage, foobarProd

I’d dev my project on my local box using the local computer fabric and local storage. In some situations I’d use the cloud for my Dev storage instead.

I’d treat my Stage accounts like a traditional Staging server. In other words I’d configure it with a couple of instances and deploy my package up there for pre-release testing. In reality I’d probably never use the Stage.Production Deployment on my staging server instead contenting myself with the http://<guid>.cloudapp.net URL. When I’m ready to do a production release I’d take the same package I used in Stage and deploy it into the Prod account. I’d use a Production config file and my Prod.Staging Deployment in the foobarProd account would be configured to run the full number of instances- i.e. if my production site runs on 30 instances my Prod.Staging deployment would need to be running 30 instances. I’d then to a quick^^ *smoke test* on the Prod.Staging deployment before finally swapping the Prod.Production and Prod.Staging deployments and basically routing traffic at my new deployment. I’d then suspend my Prod.Staging deployment because I’ll be paying to have those instances running.

That’s my current thinking anyway. All subject to change as Windows Azure changes over the coming month.

Now that’s the easy bit! Hot swapping the compute instances like this is easy because we are (or at least should be) building them using a stateless server pattern.

The harder part is achieving a hot swap upgrade when you need to change your data tier. Because the data tier is stateful we need to consider consistency of the data. This is a problem in on-premise models too and one that does not have an easy solution. Maybe the Azure team will come up with some ‘magic’ that helps here??? I’ve got some thoughts on how it might be done but none of them are particularly elegant or developed yet. keep your eye out for a post on it sometime in the future.

If this stuff appears useful to people (including the footnote below) I might bang up a sample and a web cast. Let me know in the comments.

P.S. Ryan: Looking forward to seeing that tool!
^^ In reality I’d probably want each instance to have serviced at least one request in order to have JIT’ed everything and spun up. This means, for a 30 instance deployment, that I’d really have to slam the server with some load and probably have some way (setting a flag in the Application.OnStart event?) of ensuring that I had hit 30 unique servers. Easiest way to generate the load would probably be with a Windows Azure Worker role

So at almost all the Windows Azure events I’ve run around the world recently we’ve always gotten into a discussion about hooking Windows Azure applications up to on-premise systems. Indeed in a recent demo I wrote for a CRM + Windows Azure session I showed how to wire up Windows Azure applications to your on-premise CRM server. Often you’ll be using the cloud for what I call forward processing.

You should use caution here. Your Windows Azure application has the ability to scale almost infinitely. Your on-premise application does not. If you wire your Windows Azure web role directly to your on-premise system, say by making a direct Web Service call, and then your site gets slash dotted you’ve basically built your self a massive cloud hosted Distributed Denial of Service engine!

What you need to ensure that you do is buffer all of your requests from the cloud to your on-premise system via a queue of some sort. You may choose to use a Windows Azure Storage Queue for this or you may choose to use the new Queue and Router capabilities in the .NET Service bus. This means that if your Azure Web Role suddenly gets a huge amount of traffic the requests to your back end system will be queued and processed only as fast as the on premise application is able to handle them- the last thing you want is for a simple client self service application in the cloud to bring down a core business application on premise.

Obviously doing this is going to require that the call to the backend system is capable of being made asynchronously. If it’s a one way operation then it’s easy, if it’s a two way operation then getting data back to your user is generally going to be a case of having your worker role right back to to an Azure Table or indeed having a notification queue for each user that your worker role can enqueue the result onto.

In summary: If you wire your Windows Azure application directly to your on-premise application you are dicing with danger.

So I’ve been head down and arse up this week; I’ve barely had enough time to get on top of my email inbox let alone my unread blog entries. Finally made some progress this evening at YVR and now on NZ83 en route YVR-AKL.

So while I was underwater a stink kicked up around a thing called the ‘Cloud Manifesto’. Meant to be being released Monday, it was leaked here (all 6 pages of it) a few days early. From my reading between the lines of the thinly veiled blog post by Steve Martin from Microsoft it looks like the document has been written, by an as yet undisclosed group, and is being farmed around a bunch of companies for them to ‘sign up’. There is a posting on the Cloud Computing Interop Forum (on Google Groups) by Reuven Cohen (who I’ve never heard of) who is the ‘Creator of the Enomalism Elastic Computing Platform’ (which I have to admit I’ve never heard of either). Reuven is also involved in Cloud Camp (which I have heard of). I tell you all this and encourage you to go and have a sniff around some of the links above and some of the press coverage so far so that you have some context before continuing.

http://www.techcrunchit.com/2009/03/26/out-of-order-20/ Steve Gillmor from TechCrunch

http://blogs.zdnet.com/BTL/?p=15341 – Amazon expressing their antipathy towards the whole thing

http://www.eweek.com/c/a/Cloud-Computing/Microsoft-Calls-for-Open-Cloud-Standards-538212/ Darryl Taft

I provide the above to give some background. Now for some of my thoughts.

The Emperors New Clothes*

So reading the document itself most of it feels so obvious it barely warrants saying- it’s PR puffery really. I’m going to cherry pick a few statements from the document to discuss.

“We believe that these core principles are rooted in the belief that cloud computing should be as open as all other IT technologies.  “

I hate to break it, but, other IT technologies aren’t that open. As I’ll set out below I’m a big believer that in most cases openness and interoperability are emergent aspects of technology and not something you should really set out to engineer.

“To reassure their customers, cloud providers must offer a high degree of transparency into their operations.”

So I just don’t know how realistic this is. Certainly Microsoft and even more so Google are highly protective of their data center operation details. A measure of security through obscurity is important here still I think. I would also disagree with the authors that moving data into a shared infrastructure necessarily exposes one to more potential for unauthorized exposure.

“Cloud providers must use and adopt existing standards wherever appropriate.  The IT industry has invested heavily in existing standards and standards organizations; there is no need to duplicate or reinvent them. “

So this is bang on target and to be honest the biggest thing that the Cloud Providers can do to ensure *useful* openness is to ensure they use existing standards wherever possible. But do we need this sort of industry bickering to state the bleeding obvious?

“Any community effort around the open cloud should be driven by customer needs, not merely the technical needs of cloud providers, and should be tested or verified against real customer requirements.  “

Would be interesting to know what level of customer involvement there was in the Manifesto document? Were folks like SmugMug (who are my favourite example of a great Cloud operated business) or enterprise customers (I know there is at least one airline using EC2 for some of their stuff) involved? It all feels like a bit much of a simplistic puff piece at the moment- though according to the CCIP group post ‘major players’ have been involved.

“Cloud computing standards organizations, advocacy groups, and communities should work together and stay coordinated, making sure that efforts do not conflict or overlap. “

Haven’t really gotten off to the best start in this regard to be honest.

This document is meant to begin the conversation, not define it.

I’m afraid I agree with a number of the commentators (both vested interests and ‘independent’ voices) at the top of this post. It was a pretty shitty way to ‘begin the conversation’.

Who’s Actually Behind It All

So a big question in my mind is who is actually driving this thing?

To be honest if it were just a group of bit players stroking and stoking their egos then I don’t actually think it would have generated the level of interest and posturing (from MS and Amazon to date) that we’ve seen.

Gillmor looks at the obvious candidates, IBM and Google (Amazon having declared they’re outside the tent). It seems inconceivable that Microsoft and Amazon would not be invited to be involved in this at the earliest opportunity. Even if it were the ‘Anything But Microsoft Brigade’ you’d still expect Amazon to be seated at the table- hell their model of a high scale application operator selling their dog food to others embodies for me much of what the cloud is about.

I have no idea how Cohen can, with a straight face, say:

“Given the nature of this document we have attempted to be
as inclusive as possible inviting most of the major names in technology to
participate in the initial draft.”

Given that this Manifesto appears to be largely driven by East Coast companies I can’t help but think that IBM have got their finger in this somehow. It’ll be somewhat ironic if this the case, as a key proponent of the standardization of Office Open XML (now ISO29500) I’ve had first hand knowledge of IBM’s ‘do as we say’; might this be their ‘not as we do’ moment?

I guess all will be revealed come Monday.

On The Substantive Matter- Or Why I Only Kind Of Give a Crap About Interop in the Cloud

Standards stifle innovation. There, I said it. Friends of mine will probably bitch at me for being so blunt, but, I genuinely believe that standardizing technologies is, in most cases, best left until they have reached a reasonable level of maturity.

Once something is standardized pace becomes glacially slow and the ability for innovators to recover a return on intellectual property becomes nigh on impossible. There are, of course, some business whom this model is de rigueur- I guess we’ve got to wait until Moday to find out more.

For the most part we’ll be building our applications using the same protocols and technologies that we’ve always done. At least for the stuff that spins my wheels, high scale applications, there’s not a lot that’s new in the cloud; it’s really just an interesting new way to deploy and dynamically scale the same architectures I’ve been working on since I was building SaaS apps in the Dot Bomb days.

The new stuff really comes down to the cloud fabric^ and the management thereof and frankly I think it’s too early to think about standards. Amazon scales at the unit of a virtual machine, Microsoft does the same but with sugar on the top to hide the fact that it’s really Windows at all. Let’s see what sort of models shake out best for customers before we try and anoint one king.

I Guess We Wait Until Monday

As someone looking to ship a pretty significant Cloud delivered app later this year what I really want is;

• For my Cloud provider of choice to ship their bits!
• For my travel schedule to slow down a bit- my Tripit stats are truly horrendous and I’m flying to Bangalore Monday morning.
• For the Manifesto authors, whoever and wherever they might be to read a little Hayek.

Let’s see was happens Monday.

*So worth nothing this wasn’t my turn of phrase. Used here first but I liked it so much I pinched it!
^Call it what you will, this is the term I use when talking about Windows Azure at least.

So I’ve been doing a bunch of presenting around the Cloud Recently and also starting to discuss it as a platform with a number of our customers.

I thought I’d put together a blog post on some of the key reading I’m getting people to do to come up to speed.

http://berkeleyclouds.blogspot.com/2009/02/above-clouds-released.html
Is an academic white paper out of Berkeley. Well worth a read and I’d probably start your journey here.

It’s also important, no matter which platform you choose, that you have a good understanding of how to build horizontally scalable systems and that you understand why and how to partition your stateful data tier.
I quite like the recent O’Reilly book Beautiful Architecture  for an overview of this with worked examples

Other stuff that’s worth reading include:

The Primer on Scale from Wikipedia is useful
http://en.wikipedia.org/wiki/Scale_out#Scale_vertically_vs._horizontally

Choosing Partition Keys
Some Thoughts on Choosing Partition Keys in Windows Azure's Table Storage

And Also from Dare: On ‘Trusting’ The Cloud
Asking should we trust the cloud is like asking should we trust horseless carriages

If STS’s, IP’s RP’s and other identity acronyms are Dutch to you then Vittorio’s article on Claims is well worth a read
Claims and Identity, On-Premise and Cloud Solutions

So you may find yourself having issues calling into the development fabric web roles. The error message will be something like ‘te server actively refused the connection’. There are some issues with IPv6. The development fabric will only listen on open host and I think what was happening is that the requests appear to be coming from the IPv6 address rather than the IPv4 one3.

Anyway, I disabled IPv6 and commented the IPv6 localhost entry from the *.hosts file. All worked after this.

So one of the cool things you can do in Windows Azure storage is just enter the URL of a public Blob in your browser and retrieve it via an HTTP GET. This makes sense as in REST, a GET and a URI will return the data at the URI. In the case of an Azure Blob this means the bytes stored returned with the appropriate content type in the header.

You can also do thinks like retrieve a list of the Blobs in a given container like this:

It would be nice to be able to do the same with Azure Table storage. A quick glance at the SDK leads one to think it’s as simple as

http://myaccount.table.core.windows.net/Tables

But this will fail on both the Cloud Storage and Local Storage. This is because

“The request must include the Authorization header and the Date header, as specified in Authentication Schemes.”

So that sounds simple enough- I just cracked open Fiddler thinking I’d just punch in the headers. But, these headers are actually fairly complex. Well, ms-x-Date is simple enough, but, the Authentication headers involve both canonicalization and signing of the request.

I’d love to give you a simple solution so you can just browse tables, but, I haven’t got one. I hope that at least this saves you from spending too much time trying.

So a question came up in the Azure training session today about what the timeout was before a ‘Dequeued’ message was returned to the quen unless it was ‘Removed’. Windows Azure is designed to ensure your messages will always be completed so it returns a dequeud message to a queu if it has not be permanently removed after a period of time.

I knew it had to be configurable… but had no idea exactly how.

The answer is here in the SDK

http://msdn.microsoft.com/en-us/library/dd179396.aspx

or, if you are using the StorageClient sample code from .NET (and if you are using .NET then you really should be!) then this post from Neill Kidd should help

http://blogs.msdn.com/neilkidd/archive/2008/12/07/poison-in-windows-azure.aspx

// Get message with 60 sec visibility
Message msg = queue.GetMessage(60);

This means to ‘Dequeue’ a message only place that message back on the queue if RemoveMessage has not been called within 60 seconds.

So based on the questions I have been receiving at the various Azure Services Platform events around the world I am starting to put together a list of blog posts to do as I get time. Please feel free to suggest other posts.

• What Fits Where
  I probably need to do a post that talks a bit about which bits of the Azure Services Platform fit into what sort of businesses and business models. Windows Azure won’t cure world hunger, but, it will be a great platform to use if you’re going to build the next Facebook. There’s not been a lot out of Microsoft with these sort of details yet, so, hopefully this post will go someway to answering these questions.
• Forward Caching in Windows Azure
  This post will talk about how to use the Local Storage in Windows Azure to cache data. Why you would do it. What data you should store in there. What data you should not store in there.
• What Being Stateless Means
  What does building a stateless Web Role or Worker Role mean? What are some common traps where people might slip into ‘statefulness’?
• Multithreading and Parallelization in Windows Azure Worker Roles
  When should you spawn multiple threads vs When you should create new instances. Will discuss things like performance bounds. CPU Bound vs IO Bound vs Network Bound.
• How To Implement A Plug-in Model In Windows Azure
  Because the Windows Azure deployment model is on a Package basis it’s not possible to add new DLLs once an application has initially being deployed into a Compute project. The solution is, of course, to load the DLLs remotely into Cloud Blob storage and then dynamically load the types using a plugin pattern.

Had plenty of conversations over the past few days in Mumbai with people about scale out strategies. About the difference between stateless and stateful portions of your application and about some ideas around choosing horizontal partitioning strategies.

Waddayaknow… I boot up RSS Bandit this morning and Dare has a great post on this.

Covers many of the things we discuss this week.

One of the things I focussed on a bunch this week is having some knowledge of the distribution of your data when choosing a good partition key.
You want a horizontal partitioning key (partition key in Azure Table Storage) that ideally DOESN’T fit a normal distribution.
Think of it like this. If you partition your data using say the first letter of the user name then you’re gonna end up with a whole bunch in your ‘S’ partition and not a lot in your ‘Z’ partition.

Anyway. A great post and well worth reading.

Building Scalable Databases: Pros and Cons of Various Database Sharding Schemes

So I am running an Azure Training course here in Mumbai today and people are asking about good blogs to read WRT the Azure Services Platform.

Here is the OPML export for my Azure folder from RSS Bandit.

If you are building applications with Windows Azure you will know that your Web Role and Worker Role instances need to be stateless. If you’ve been architecting high scale (read horizontally scaling server farms) then you’ll be used to this concept- you’ll know about the Sql Session state provider and so forth.

If you are using Azure you obviously will not be using the Sql provider. Instead you should look at the AspProvider example in the SDK. This uses Azure Blob and Table storage to store your session state.

So one of the funky things about the .NET Services (was Biztalk serivces) part of Windows Azure is the Connectivity Service.

This makes it easier for WCF to communicate bidirectionally through NATs.

It works by initially working through the cloud hosted relay and then attempting to establish a direct connection. The direct connection approach is pretty cool.

Obviously if it’s inside the same subnet they just talk direct, but, if it’s punching through NAT devices it requires some differnet approaches.

The approach that Clemens et al have taken here is the idea of NAT Port Prediction. Basically it involes ‘guessing which ports will be opened outbound, firing a packet at roughly the same time between both ends of the connection and ‘hopefully’ snapping a socket open.

High level details are here:
http://nutss.gforge.cis.cornell.edu/pub/imc05-tcpnat/

Quite a cool approach, Clemens assures us that it’s used a bunch already by the likes of Live Messenger and others.