We are living in a world where 86% of enterprise business are saying they will migrate all of their services into the cloud, but at the same time, 80% of the same enterprise businesses continue to spend on their on-prem infrastructure. What does this really mean? From my experience, I have found that businesses see the benefits of making the journey to the cloud, but for a multitude of reasons (skills, application requirements, bandwidth) they still maintain their on prem DCs. That’s why we have seen the emergence and growth of the term ‘hybrid-cloud architecture’, the best of both worlds.
Pure Storage have been tracking these trends and are bringing to market products and solutions that help their clients build bridges between the on-prem and cloud storage worlds. We are at a point, with the advancement of technology, that moving workloads between on-prem and the cloud is becoming more and more simplified. You can run your VMs on-prem or in the cloud and your experience will be pretty much the same from a management and operational perspective. However, this isn’t yet the case with storage. If you were looking at object storage your selection of vendors for an on-prem solution was vast, whilst historically your cloud options were limited (for instance AWS EBC). This brought with it its own challenges. These include the lack of features such as deduplication or compression, common in the on-prem world, as well as requiring different management tools all add another layer of complexity.
Pure Storage is trying to address some of these challenges by improving their current portfolio, whilst also introducing new products to answer the growing demands of the market. The storage industry, historically, has been very static and lacked innovation. Pure Storages’ simplistic and unique approach and their focus on continuous improvements has redefined the whole primary storage market. Previously Gartner reports separated flash and non-flash storage, mainly due to the price points deeming them worlds apart. However, Pure Storages’ competitive pricing has seen this gap close, forcing Gartner to now combine flash and non-flash in a comparative view. Talk about shaping the market!
So, what has changed in the past 10 years? Besides the fact we now have iPhone Xs instead of the iPhone 3G (and Arsenal still haven’t managed to win the Premier League), the price for flash storage has massively decreased and AWS has grown beyond all expectations. So, what is in store for the next 10 years in storage I wonder? In my opinion, it’s all eyes are on Pure Storage to lead the way.
With epic growth and innovation at the forefront, Pure Storage came to this year’s Pure Accelerate conference with two big announcements: The launch of the FlashArray//C and Cloud Block Store.
The FlashArray//C will be targeting the secondary storage market where Pure Storage have previously struggled to position themselves. As expected, Pure Storage will have a whole range of FlashArray//C based on size and throughput requirements. However their first model will be a FlashArray//C60 which will be available this year in three versions: 1.3PB, 3.2PB and 5.2PB. Pure Storage has decided on using QLC flash due to its competitive price point. QLC memory has challenges in terms of the limited number of writes and re-writes possible along with a higher latency of 2-4ms. The first challenge has been addressed by some of the innovative features Pure Storage has developed in the background, such as ‘in-line deduplication’ (not writing the data which has been already written and using a pointer instead) and also ‘garbage collection’ (data that is stored on the flash drive but will be removed and need to be restored). Higher latency with smaller throughput is not necessarily an issue but can potentially be risky for some workloads. In response, this is addressed by the parallel writes that FlashArray//C monitors in the background and aims to increase capacity. This means that a larger number of disks increases the space required by the on prem solution, and therefore increases the cost.
It will be interesting to see how the FlashArray//C will compare to the FlashArray//X from a commercial perspective. Will we see a wave of clients moving away from FlashArray//X to FlashArray//C? The second question is can FlashArray//C beat the price of secondary storage solutions such as NetApp? Possibly not, but an all-flash secondary storage brings other cost savings opportunities, when considering, rackspace, power, air conditioning and operational overheads. For the clients where the recovery speed is essential, an all-flash secondary storage solution could be a game changer.
The second product announced to launch was Cloud Block Store (CBS), which will address the challenges of a modern storage solution in the cloud. CBS architecture consists of multiple building blocks - controllers, virtual drives and storage. The controllers and virtual drives will be running on EC2 instances (C5n) whilst storage, for durability reasons, will be running on AWS S3. The controllers will be running in HA mode, as they do on traditional physical arrays and the virtual drives running on EC2 will act as flash drive equivalents. All EC2 instances will be running the cluster that is representative of the ‘storage shelf’ in the physical storage world. All the data is load balanced via the controller and sent to one of the virtual drives and then stored to S3. You will need a NAT gateway to allow all the EC2s to call-home and the VPC endpoint to connect the EC2s to S3 which keeps the traffic within the AWS network. All of this will be deployed by CloudFormation in AWS so you won’t have to spend time deploying this. It will be interesting to see how this type of holistic approach will shape the market in the future. Clients who already own FlashArray or FlashBlade can start using CBS without any additional licenses and start distributing their data between on-prem and cloud storage. One example that came to mind of a use case for this solution is the moving of data required in an AWS test environment to the cloud and back on-prem after testing has been carried out. PureStorage always put their new products through rigorous testing and Cloud Block Store has already been tested by 60 pre-GA clients and based on the conversations I was having at Accelerate, it performed very well. Cloud Base Store can run between different availability zones and supports ActiveCluster, similarly to physical arrays. However, the same latency requirement applies so don’t expect ActiveCluster between AZs in different regions!
Kubernetes and storage have always been a challenge. Even though Kubernetes offers consistent storage it never has any intension to be the go-to highly available and reliable storage solution. We have seen a few attempts in the past (like for instance FlexVolume) to try to address these issues. However we have recently seen big steps being made to allow other vendors and communities to plug in their solutions, speer headed by K8s storage integration and Container Storage Interface (CSI. We have seen storage vendors like AWS, Google Cloud and NetApp plugging their products into Kubernetes clusters, paving the way for other vendors to follow suit.
Pure Storage started to develop their first version of K8s plugin a year ago, but have only recently introduced their own Pure CSI connector called Pure Service Orchestrator (PSO). The benefit of PSO is not only highly available and consistent storage, but also the flexibility it provides with dynamic mounting of the volume in case of pod failure. This is bringing us closer to running workloads which has previously been a ‘no-go’ in the K8s world. PSO supports CSI 1.0 (apparently the community is already working on CSI 1.1 standard) and supports all Pure Storage FlashArray, FlashBlade and Cloud Block Store storage. A minimum spec supported version of Kubernetes is 1.1.4. PSO, which can be easily downloaded from Github without the need for extra licenses.
During the conference there was a number of other exciting announcements , such as the launching of Pure TestDrive. This new release will allow partners to showcase Pure Storage to their clients through Customer Solutions Labs. Here at Natilik, we are slightly ahead of the game as we use Pure Storage in our own DC environment and have a bespoke showcase built to illustrate to clients the power of Pure.
I can’t wait to test some of the new products like Cloud Block Store and Pure Service Orchestrator that I will review in a series of upcoming blogs. Pure Accelerate was great fun and if you think that storage is boring, I would suggest you attend next year, to see how exciting it can be and how PureStorage are pushing the industry into completely new territories!