Cloud "Everything As A Service" - the latest "abused by all, confuse all" phrase!

OK so there are many people abusing the word 'cloud' like crazy - sadly no new news there.

Personally I'm more aligned to the NIST definitions re cloud (http://csrc.nist.gov/groups/SNS/cloud-computing/)- now these aren't perfect, but rather a common reference point. Sadly even those are being 'interpreted' conveniently by many now...

Couple of things I'm fairly clear about in my mind :-

• Virtualisation /= Dynamic Data Centre /= Cloud
• Virtualisation is often used currently for 'tin stacking' with existing provision & operations processes, nothing wrong with that but a short term capex and facilities opex benefit only.
• Dynamic Data Centre is taking virtualisation to the next step, embracing automation and dynamic integration with other infrastructure support and management toolsets, this could be related to 'private cloud' if combined with a service model & opex accounting/recharging to the business customer
• Cloud includes a wider set of topics relating to ownership, finance, roles/responsibilities, managed services, commodity and scale that are typically not address by the above points
• Shared Infrastructure /= Cloud
• 'pool' would be better term for shared infrstructure, and yes these are a good idea in enterprises (should be the default option for everything in the infrastructure stack)
• But having an array with 15 different applications using it isn't a cloud, it's an array with multiple consumers, yes this forces a service mentality but that's hardly anything new...

I'm also of the view that a cloud service can be built without need for virtualisation, DDC or cloud optimised technologies, however in order to scale or be as cost optimised as possible clearly DDC & cloud optimised technologies are needed - so are open source, very different licensing models and very different processes but they are different topics.

Some of the questions I use as a litmus test for people telling me they are offering 'cloud services/technologies' is :-

• Are the actual purchase prices (not RRP etc, but actual buy prices) made available publicly for anybody to access?
• Are the prices the same for everybody (other than consumption based tiers)?
• Are the SLAs published publicly for anybody to access?
• Does the supplier publish a full TCO/ROI model for the customer to examine / adapt etc?
• What type of standards does this operate with / under?
• What's different to the past? (eg 'outsourcing', 'managed service', 'HP UDC', 'Egenera BladeFrame & PAM' etc)
• What is different between this and classic enterprise IT usage?
• Is the minimum duration of engagement hours, days, weeks, months or years?
• What is the 'startup latency' of the engagement?
• What are the metric elements for cost?
• What is the level of granularity of cost (consumption & change)?
• Do I need to meet/talk with a human in order to setup, purchase & use the technology or service?
• Is the technology or service fully controllable by a published API?

Clearly there are different questions / relevance between consuming cloud services, and utilising technologies in order to provide cloud services - but I find above an interesting starting point for both. Similarly there is no defined right or wrong answer to above, but the combined answers help frame my understanding (I have a very long list of other questions but that's for another day & blog entry).

Naturally consumption of cloud services tends to have a very different financial model & aspects that need to be accounted for when comparing (but that's a subject for a much bigger blog entry). Whilst internal company recharging can alter the appearance of financials for some it doesn't for the company as a whole (other than help to drive behaviours).

What I am getting really cheesed off with is people marketeering classic data-centre technologies as 'cloud' simply because they can be used as a component within a 'cloud service' - if I were to follow that analogy would I be able to buy 'cloud screws' from http://www.screwfix.com/?

Honestly, look its really simple, as usual quiet action & genuine cost reduction talks not hype, marketing or bluster - if you want to ride the 'cloud bubble' be clear how your technology genuinely relates to 'cloud' and in what context, or please I beg of you STHU and focus on what you're good at?

Cloud Backup & Android

Firstly a tip for Android users (and I'm a major fan of this platform for mobile devices) :-

If you change your Google account password, also make sure you change the password cached on your Android phone quickly afterwards - otherwise it appears that Google automatically decides that the failed login attempts from your phone (when it is auto syncing contacts, calender & email etc data) are a hacking attack and temporarily disabling your Google account... Clearly not good and needs some more thought from Google I think :(

Which leads me onto a related topic - cloud backup (which of course should really be cloud restore rather than cloud backup). In this context I'm really talking about the SaaS & PaaS definitions of cloud.

Firstly I should say that I'm greatly in favour of having multiple controlled & secured instances of data in several locations, my feeling is that for a lot of smaller organisations or individuals this simply doesn't occur. As such the technologies loosely referred to as 'cloud backup' could be invaluable to many people in easily & cost effectively enabling data persistence and recovery.

Now, before we dig into the real topic, there are lots of side-points to consider re security & availability in the cloud backup/recovery area, including :-

• Obviously the data needs to be protected, normally with encryption. My view is that this should have private keys supplied & owned by the user, and not by the backup/recovery provider. With the private keys similarly backed-up to a separate key escrow / backup & recovery provider.
• What SLAs does the provider have? (availability, accessibility, performance, integrity etc) and how, from where and how often are they measured & reported?
  
• How can you contact your provider should you have an issue? (a web form simply doesn't cut it)
  
• What guarantees do they offer to keep your data at their site available - are they a '2nd copy hoster' or do they treat your data with the same care as a master copy (eg do they do their own backups/replicas, can you treat their service as an archive rather than B&R etc?)
• What are the guarantees worth? what kind of financial penalties / compensation are available, how are they calculated & triggered and how do they compare re the value of the data?
• Is the provider somebody you'd trust with your banking details? As it's likely you'll either be giving them these, or all of the information behind them, in one form or another
• Cloud economics often rely on some form of content dedupe at the provider's end, you need to satisfy yourself that supplier's dedupe won't impact your security or encryption
  

But with the above in-mind, back to the real topic - the three real points I was wondering about here are a little bit different :-

• Should you backup your SaaS & PaaS cloud service data to your own local media (ie backing-up your part of the cloud)?

What happens to your data (your assets & value) when a service goes down, your account is deleted, the service is hacked, the company vanishes or... Can you backup your Google/Yahoo email to your local home NAS, can you backup your blog sites & social media pages to your local storage?

Irrespective of how it is done I'm increasingly of a belief that there is going to be a need for this. The first time it happens is often for some 'novelty' data which is irritating but little more, however as people rapidly move to cloud services that handle their data the risk & loss becomes higher...

Not saying stop using the SaaS services (different view re PaaS but that's another blog) as the prime system, but if the data is worth something (emotional, financial etc) then my view is that it should always be in two independent places, with one of those in the data owner's direct control.

So I'm wondering when the current generation of home NAS devices will start to include the ability to receive data from remote sites, or to have the ability to obtain that data automatically themselves?

• Can your cloud backup/recovery partner also backup your social media and SaaS services?

Moving on from the previous point, what I'm thinking about here is that rather than need to use local media, could your cloud backup partner (assuming they are different to your other PaaS/SaaS providers) also provide 'content aware' backups for your other internet data services such as blog sites, Facebook & MySpace sites, Twitter tweets/favourites/friends/followers, webmail and other PaaS / SaaS services etc?

• Could your cloud backup partner also move into providing a basic 'cloud DR' service?

It's a fairly simple step for a cloud partner to wrapper & automate the creation of an AWS EC2 image, load their backup/restore software onto that image and then allow the customer to restore their data 'as needed' to the EC2 image. Where in turn they can run the usual suite of common apps easily enough... Not earth changing but a simple enough value add that would provide transitory help for some situations...

Now I'm aware that some of the points above could be twisted into FUD, they certainly aren't intended as that (and I'll be more than grumpy if they do get used as FUD) - they are the questions I ask myself about my personal information storage (especially when an account gets disabled!).

In this topic (like many others) I certainly agree with some of the points that @StorageBod makes in his blog entry at http://storagebod.typepad.com/storagebods_blog/2009/08/information-haze.html re personal information being both of value and dispersed, with little current understanding from the public at large re the potential consequences...

NetApp - Cloud of Fog?

Right, so firstly despite what it may look like this isn't a NetApp bashing blog site or month (in fact I'm grumpy with all vendors - and NetApp are a strategic supplier to me), but I'm afraid I'm a bit lost re NetApp's marketing and 'cloud' and really need to ask a public question :-

"Can anybody give me any detail and substance to NetApp's cloud strategy, technology and deployments?"

I ask this genuinely, as despite asking the company directly for over 2yrs, and despite regularly looking myself through their blogs and whitepapers, I'm finding it really hard to locate any information re NetApp and IaaS, PaaS or SaaS usage or technologies (direct of indirect).

Oh of course I've heard the stories re large company X using ### many filers for search or webmail, and other such things - but frankly these strike me as very much a standard use of a technology coupled with significant discounts linked to volume & name etc. Not specifically addressing the changed requirements in a cloud (thinking IaaS) environment (ie individual assets may have less availability req, object protocols often needed, much greater connectivity reqs etc).

What I'm looking for info on is :-

1) A technology & commercial model that is more aligned at a web 2.0/3.0 business, with vast data scale, objects & files, geographical distribution, policy based mngt, compression & dedupe of data, adequate performance, adequate availability of a physical asset but v high availability of the information, and at a 7yr TCO price point substantially less than today's post discount per TB price

2) Any info re an object store capability (although Val Bercovici has already said he's not prepared to pre-announce anything right now)

3) Any info re a direct to market SaaS offering?

4) Any details, info or case studies re genuine cloud companies or web 2.0/3.0 companies that are using or planning to use NetApp in decent sized (ie multi-PB) cloud deployments

I'm wanting to like their 'cloud' technology but frankly speaking at the moment trying to find out about it is like trying to knit fog (sorry couldn't resist)

If any NetApp'er or anybody wants to point me towards some substance and fact then I'd be more than grateful :)

Objects & Metadata

As usual Dave Graham brings up some interesting and worthwhile topics in his blog post here http://flickerdown.com/2009/08/micro-burst-metadata/#comments

Now being an ex database programmer ('ex' of anything being of course the very worse and most dangerous type), and of course a storage curmudgeon, I have a passion for the topic of metadata and data. And being somebody having to deal with PBs of object data I naturally have some concerns and views here...

Now normally I agree with Dave on a lot of things - but I have to say I much prefer my scallops to be seared and served on black pudding nice and simply, letting the quality of the flavours shine.



That said I have to agree re his view of being able to segment metadata & object storage models into two areas - but do think there is a place (almost essential IMHO) for both models in the future storage.

We've seen this area tackled by a number of existing technologies re CAS and object stores (Caringo CFS gateway onto Castor object layer is good example) - but are only just starting to see the key new elements test these, namely vast scale (think EBs), geo-dispersal/distribution/replication, low cost.

I do also think it's worth exploring some of the possible types / layers of metadata, for me this breaks into :-

• System / Infrastructure metadata - the metadata mandated by the storage service subsystem for every application using the service and every object held within the storage service. System metadata is under the exclusive control of storage service subsystem, although can be referenced by applications & users. Examples such as object ID, creation data, security, hash/checksum, Storage service SLA attributes (resilience, performance etc) etc.
• Application metadata - This is the metadata associated with each object that is controlled and required by the application service(s) utilising the object. There may be multiple sets of application metadata for a single object, each only accessible by the approved application.
  
• Object metadata - context & descriptive attributes, object history, related objects, optional user extensible metadata
  

I would expect all 3 examples of these metadata to be linked with every object, with at least the 'system metadata' always held locally with the object. The 'application metadata' & 'object metadata' may reside in the storage system, the storage service, the application or any combination. (In this context I refer to the storage system as an object store, and the storage server as being object store + metadata store)

Some of the metadata relates to the application and infrastructure architecture (eg geo-location information re object distribution & replication) whilst some of the metadata are attribute fields used within the application itself.

Given the above, it should be clear that I certainly agree with an entry Dave made in his blog comments re :-

"interesting note on ownership to which I'd say that there has to be dual ownership, one from the system level (with immutable meta such as creation date, etc.) as well as mutable data (e.g. user generated meta). The meta db then needs to maintain and track 2 different levels. Policy can affect either, fwiw."

So some thoughts about where to locate metadata as it relates to the object :-

• As referenced above, I believe 'System metadata' must always reside with the object as it is used by the storage service for mngt, manipulation and control of the object itself, and ensure it's resilience & availability.
• As has been an issue with file-systems for some time, there is always an issue with fragmentation of the underlying persistency layer with vast size differences between objects and metadata when they are tightly coupled
  
• As a result of needing to traverse the persistency layer to establish the metadata, there are performance issues associated with metadata embedded within the object layer - move the metadata to a record based system and performance & accessibility can increase dramatically
• For certain classes of use (eg web 2 etc) it's often the metadata that is accessed, utilised & manipulated several orders of magitude more often than the objects themselves, thus the above improvements in performance and accessibility of metadata (thin SQL query etc) make major differences
  
• Clearly if the metadata and objects are held separately the metadata can be delivered to applications without needing to send the objects, similarly the metadata can be distributed separately / in-advance of the object. Thus having major advantages for application scaling and geo-distribution.
  
• With the split of persistecy location / methods this also allows for security layers to be handled differently for the metadata and the object.

This also brings into line a question area I've been working with for over 2 years with object stores - in that what features & functions should live in the application layer and what features and functions should live within the infrastructure (storage service) layer. What areas of metadata are actually data information in their own right, or embedded in the application logic, there appears to be no clear rules or guidelines.

If you like, this could be seen as an argument between IaaS & Paas - and for sure the only sensible answer for a company right now is IaaS, PaaS exposes far too much of the logic, taxonomy, behaviours, trends and metadata layers to the PaaS provider than is healthy.

There is also an additional interest point re metadata - as we move from the System metadata into the Application & Object metadata, should we consider privacy and encryption of the metadata itself? (assuming that the objects will always be protected appropriately) I could see how this will be a requirement in some multi-tenancy environments an for some metadata elements...

Lastly some more questions :-

1. How do you cover the topics of backup/recovery of the various metadata elements?
2. How do you cope with bulk import / export of the various levels of metadata and their logical / context relationships?
3. What standards will emerge for metadata schema definitions and attributes?
4. What standards will emerge for policy script language & descriptors that manipulate within the storage systems? (think how to describe an SLA in a programmatic language)
5. Can security authorisation & permission tokens exist and be enforced in a separate context and control domain to the identities?
   

Naturally in this we're not covering any of the 'internal' metadata used by the storage system to locate objects, to handle the multiple instances of the same object within a 'object storage service' (resilience, replication etc), to enable sharding / RSE encoding of objects etc that the storage system has to cope with.

Now I'm off for some lunch, I'm hungry and fancy some seafood for some reason ;)

Cheers

Ian

Related articles by Zemanta

• AppScale (byzantinereality.com)

Hello Dave - don't get misty eyed

Just a brief post to say hello and welcome to the cloud area to somebody that I have a great amount of time & respect for, so :-

"hello Dave Graham it's great to hear you're taking a key role in the cloud infrastructure arena, please keep to your great ways so far and don't get misty eyed or foggy over matters :)"

To hear from the man himself see here :-
http://flickerdown.com/2009/08/transition-to-the-cloud/