Per IDC’s Copy Data Management Challenge, “65% of storage system capacity is used to store non-primary, inactive data”. In fact, Inactive data residing on flash is the single biggest threat to the ROI of data center modernization initiatives. As my friends and I get older we often find ourselves talking about “right-sizing”, or moving to a smaller, and less expensive piece real estate. In many cases life has evolved or perhaps needs have changed and we simply don’t want or need the cost of maintaining a larger residence. The benefits are obvious when it comes to savings in mortgage, utilities, and general upkeep. Well your data has a lifecycle as well, it starts being very active with high IO and the need for a high quality of service and low latency. However, as data ages it’s just not economical or necessary for it to reside in the “high rent” district which in most cases is an active flash tier.
Now data tiering through the lifecycle is not a new concept, but the options available to you have never been greater. Destinations such as lower cost/performance tiers of spinning disk can be an option. If your organization has deployed a private cloud, that might be an excellent destination to tier inactive data. For those who have adopted public cloud IaaS, that certainly is a low-cost destination as well. Let’s explore some of these options and solutions for managing the data lifecycle through the different options available. More importantly, let us look at some issues that should be considered before creating a data lifecycle residency plan with the goal of maximizing your current investments in on premise all flash arrays, and both private and public clouds.
Automated Storage Tiering
Deploying automated storage tiering is a good place to get started as the concept is familiar to most storage managers. For example, Hitachi Dynamic Tiering is software which allows you to create 3-tier pools within the array, and it allows you to enact policies which will automatically move data to a specified type of media pool once the pre-defined criteria has been met.
In a modern hybrid flash array like the Hitachi Vantara VSP G series, your pools can be defined based upon the type of storage media in the array. This is especially economic in the VSP G series because the array can be configured with SSDs, Hitachi Flash Modules (FMD), or hard disk drives. Essentially, high IO data starts residency on high performance SSD or FMD, and then dynamic tiering automatically moves it to low cost hard disk drives as it ages and becomes less active. The savings in storage real estate in terms of cost per GB can be well over 60%. But wait there’s more benefits to be had.
Integrated Cloud Tiering – Private Clouds and Object Storage
It’s no secret that migrating inactive data to the cloud can lead to a storage saving well over 70%. The benefit doesn’t stop there as a well-managed data lifecycle frees up top tier flash for higher priority active data. Many top financial institutions choose to tier inactive based data off flash tiers and onto lower cost private cloud object storage. In this way, they get the savings of moving this data into a low-cost tier and the there are no questions of data security and control behind the corporate firewall. In addition, if the data ever needs to be moved back to and active tier it can be done quickly and inexpensively without the data egress fees incurred by public cloud providers. In addition, Private cloud object storage like the Hitachi Content Platform (HCP), give your enterprise a “rent-controlled” residence with all the benefits of a public cloud and without concerns of security because you are in control of your data.
Cloud Tiering – Public Clouds
Public clouds like Amazon and Azure have changed the data residency landscape forever. They are an excellent “low-cost” neighborhood for inactive data. Companies of all sizes, from small to the largest enterprise, leverage the low cost and ‘limitless’ storage of public clouds as a target for inactive and archive data.
Potential Issues - Tiering Data to the Cloud
The concept of tiering to either public or private clouds is simple but executing a solution may not be as straightforward. Many storage vendors claim the ability to tier to the cloud, but when you look at their solution, you’ll often find that they are not transparent to applications and end-users, often requiring significant rework of your IT architecture and a lower / delayed ROI in data center modernization. These solutions add complexity to your environment and often add siloed management requirements. The bottom line is that very few vendors understand and offer an integrated solution of high performance flash, automated tiering, and a choice of migration to multiple cloud types. Regarding public clouds, it should be noted that a downside is that if the data is not quite inactive, let’s say “warm,” it can be very costly to pull it back from the public cloud due to the previously mentioned data egress fees. Not to mention it can take a very long time based on the type of service level agreement. For this reason, many tenants choose to only migrate backups and cold archive data to public clouds.
Hitachi Vantara Cloud-Connected Flash – 1 Recipe integrates all the right Ingredients
Cloud-Connected Flash is a solution from Hitachi Vantara which delivers a complete solution for file data lifecycle residency. Hitachi is a leading vendor in hybrid flash arrays, all-flash arrays, object storage and cloud.
The solution is an easy concept to understand, “Data should reside on its most economic space”. As illustrated in the graphic above. Active NAS data is created and maintained in a Hitachi Vantara VSP G or F series unified array. Within the array, data is dynamically tiered based on the “migration policy” between pools of SSD, FMD (Hitachi Flash Modules) and disk drives. As the file data ages, Hitachi data migration to cloud software moves the data, based on policy to your choice of clouds (Public Amazon, Azure, IBM Cloud or private HCP, Hitachi Content Platform). When migrating data to HCP, the data can also be pulled back into the top flash tiers if needed creating a highly flexible and dynamic data residency environment. But the value doesn’t stop at the savings in terms of cost of storage and maintenance. The Hitachi cloud connected flash solution can also include an analytics component to glean insights from your data lake which is comprised of both on premise and cloud data.
Cloud to Data Insights with Pentaho Analytics
Pentaho Analytics enables the analysis of your data both on premise and in public and private clouds. As an expert in IoT and analytics, Hitachi Vantara offers the customizable Pentaho platform to analyze data and create dash boards and reports in your cloud connected flash environment. The goal being to achieve better business outcomes by leveraging your Hitachi Vantara cloud-connected flash solution. Pentaho offers data integration, a highly flexible platform for blending, orchestrating, and analyzing data from virtually any source, effectively reaching across system, application and organizational boundaries.
- Run Pentaho on a variety of public or private cloud providers, including Amazon Web Services (AWS) and Microsoft Azure
- Leverage scale-out architecture to spread data integration and analytics workloads across many cloud servers
- Connect on premise systems with data in the cloud and blend a variety of cloud-based data sources together
- Seamlessly integrate and analyze leading Big Data sources in the Cloud, including Amazon Redshift and hosted Hadoop distributions
- Access and transform data from web services and enterprise SaaS applications
Learn More, Hitachi Vantara Can Help
Hitachi Vantara Cloud-Connected Flash solutions are highly flexible, and individual components can be deployed based on business needs. This enables customer to start with dynamic array tiering, then add cloud tiering and ultimately analytics. Please contact your Hitachi Representative or Hitachi Partner to learn how your organization can benefit from cloud-connected flash.