Dave Graham started it all off here with these 2 blog posts :-
http://flickerdown.com/2009/06/moving-from-block-to-cloud-emulex-e3s/
http://flickerdown.com/2009/06/emulex-e3s-fitment/
Chris Evans also riased points on here :-
http://thestoragearchitect.com/2009/06/19/cloud-computing-emulex-enterprise-elastic-storage-e3s/
Emulex have now posted a site for information re this at :-
http://www.emulex.com/solutions/cloud-storage.html
I raised some questions that I'm still not sure I've found the answers to :-
- How does the 'adapter' handle and treat mutable / changing blocks? (eg does it write new block object and retire the old or something more optimised (eg a mini delta block))
- What are the scale targets for the adapter (or groupings of adapters) re qty objects, capacity abstracted, latency, throughput & cache etc?
- What underlying cloud APIs are used? are these 'pluggable / changable'? and can multiple be used at once?
- What specific encryption & KMS system is used?
- How does the adapter work with authentication, authorisation & accounting / billing attributes that may need to be handled re cloud storage?
- What policy mngt framework is used to control the behaviours of the adapters? and how does this relate / compete / cooperate with other policy frameworks (eg Atmos's or Symm FAST etc)?
- Does the adapter maintain the checksum history of the cloud objects written in order to validate that their retrieval matches? if so where is this data stored and how is it protected / made resilient?
- What's the target price point?
- What's the availability date?
- Who will be the first array manufacturer to include this (or similar) as a BE card? and how will that affect technology capability licensing within that array?
- Could this be made to work with other block formats / devices (eg tape emulation)?
- Is this partnering with any other object storage formats (eg XAM, Caringo Castor, EMC Centera etc)?
Ian,
ReplyDeleteThanks for all the interest and questions. Since the E3S is very new, we took the time to get the answers right, which I hope will provide more clarity into our plans. As always, feel free to reach out to us via Twitter or share your thoughts on one of the Emulex blogs as we continue to participate with you.
First, the simplest answer for many of these questions is “it depends”. It depends primarily upon the storage partners that choose to go to market with the E3S technology. Emulex is an OEM company and our embedded solutions only go to market via our partners in the server and storage industry. So, many of the final implementation details will depend on how our OEM customer(s) choose to integrate E3S into their existing product offerings.
Second, E3S is still a product concept and proof of concept demo. Emulex is working with our OEM partners to validate feature, function and market acceptance of the technology. While this work is progressing nicely, we simply do not have concrete answers to many of the more specific implementation details yet.
That said, there are some fundamental design criteria in which we believe that are guiding our development of the E3S technology – seamless integration with existing IT infrastructure, transparent bridging of block data to the cloud and enterprise-class data security, availability and integrity are tops among these.
Based on these principles we can provide some answers or guidance on the questions posed below.
1. How does the 'adapter' handle and treat mutable / changing blocks? (eg does it write new block object and retire the old or something more optimised (eg a mini delta block))
ReplyDeletea. E3S is much more than a cloud adapter. E3S is a technology solution designed to seamlessly bridge enterprise data to cloud storage.
b. E3S technology is not intended to be in the data path, manipulating blocks of data in this way. Rather, E3S is a target device for appliances/applications of higher intelligence further upstream. These appliances or applications are responsible for handling changes and understanding what data goes where. E3S simply provides a block target and a bridge to cloud storage.
2. What are the scale targets for the adapter (or groupings of adapters) re qty objects, capacity abstracted, latency, throughput & cache etc?
a. A very difficult question to answer because there are so many variables. Each customer environment will be unique and factors like WAN bandwidth & latency, data types and data usage models will greatly influence the scalability of the solution. Because of this, E3S is designed to be customizable in all the right areas including cache size, extent size, compression, encryption etc. so that it easily fits in the broadest possible selection of customer environments.
3. What underlying cloud APIs are used? are these 'pluggable / changable'? and can multiple be used at once?
a. So far, the E3S concept demo has been customized for use with EMC Atmos, Amazon S3 and Nirvanix APIs as well as a private cloud mode. The cloud storage targets are customer configurable from the user interface and multiple targets can be chosen. In fact, as discussed in our whitepaper, the ability to slice and disperse data (IDA) to multiple cloud endpoints is critical to increasing the availability and integrity of the data sent to the cloud and a central feature to E3S.
4. What specific encryption & KMS system is used?
a. This is most certainly one of the features on the “it depends” list. Of course, we could align the key management strategy within E3S to match Emulex’s strategy on our encryption HBAs and will likely do so. However, since E3S must ultimately integrate within and OEM solution, our encryption and key management will be aligned with those OEMs’ plans.
5. How does the adapter work with authentication, authorisation & accounting / billing attributes that may need to be handled re cloud storage?
ReplyDeletea. Managing a single 3rd party cloud provider in these areas is fairly straightforward but managing multiple cloud storage endpoints with authorization, billing, etc. is likely more than any IT admin will wish to take on. Therefore, the E3S user interface will abstract many of these functions in a “set it and forget it model”. Authorization details and activity logs for all of the connected cloud targets will be centrally accessed from the E3S UI.
6. What policy mngt framework is used to control the behaviours of the adapters? and how does this relate / compete / cooperate with other policy frameworks (eg Atmos's or Symm FAST etc)?
a. Management of E3S features is fully self-contained within the application today. However, should our OEM partners wish to do so, the management of E3S can be integrated into upper-level management applications so as to eliminate the need for IT personnel to learn and maintain yet another UI.
7. Does the adapter maintain the checksum history of the cloud objects written in order to validate that their retrieval matches? if so where is this data stored and how is it protected / made resilient?
a. The E3S, as an enterprise technology, incorporates enterprise-level features. This includes strong integrity codes to protect all cloud-destined data, and optionally block-level integrity with T10 DIF. Along with these integrity features, the E3S is capable of greater resiliency through data reconstruction even in the face of lost or corrupted cloud data (think RAID over the clouds). The E3S strives to bring enterprise-class features to make cloud storage enterprise-friendly.
8. What's the target price point?
a. This was a proof of concept for the technology, so no pricing has been set
9. What's the availability date?
a. We are not able to comment on availability at this point.
10. Who will be the first array manufacturer to include this (or similar) as a BE card? and how will that affect technology capability licensing within that array?
a. We don’t know. The race is on! ☺
Additional Questions Added on August 9th:
ReplyDelete1. Could this be made to work with other block formats / devices (eg tape emulation)?
a. Certainly. Since the E3S technology presents a block interface target, any application that utilized block format can use E3S as a target.
2. Is this partnering with any other object storage formats (eg XAM, Caringo Castor, EMC Centera etc)?
a. Not at this time.
(Great questions deserve big responses, so do reach out any time. Hit us up at @Emulex)