NEC Corporation of America

Virtualized Data Centers Need Storage Array Management Software to Complement vCenter

2011-11-30T11:00:00Z

As more organizations virtualize their server environment, they face two challenges: centrally managing both the growing number of their virtual machines (VMs) and the growing number of storage arrays that host these VMs. On the server virtualization side, VMware vCenter has emerged as a central console that first detects and then centrally manages VMware VMs across their environment. On the storage side, similar storage array management software like NEC Storage Manager is now available to complement VMware vCenter as it discovers NEC D, M and S Series storage arrays and then administers their advanced storage software features.

Many organizations are familiar with and likely use VMware vCenter to centrally manage the VMs in their environment. Using vCenter, administrators only need to log in to it to centrally manage VMs across the enterprise as opposed to having to log in to each ESX host.

However as organizations grow the number of ESX servers in their environment the number of external storage arrays is growing as well. As this occurs, organizations are looking to vCenter to manage them as well.

While this can be done, vCenter can at best be used in a limited fashion. Most storage arrays only integrate with vCenter at a rudimentary level. This limits vCenter's management abilities to doing only storage array detection, monitoring and maybe some basic storage array management tasks (LUN masking, creating volumes.)

This is inadequate for what organizations today require. They instead need a more robust management console to ensure that the advanced features found on today's storage arrays are centrally accessible and manageable so they can be effectively utilized.

This advanced storage software management functionality is what NEC Storage Manager provides for NEC storage arrays. In the same way that VMware vCenter centrally discovers, monitors and manages VMware VMs, NEC Storage Manager Suite complements vCenter by providing like functionality for NEC D, M and S Series storage arrays.

Once these arrays are under the control of NEC Storage Manager, administrators may then use its console to perform routine storage management tasks on any of them such as:

Creating volumes
Assigning volumes to specific hosts
Setting LUN security
Verifying multi-pathing is properly configured and check its status
Setting and monitoring storage array alerts

But more importantly organizations may use NEC Storage Manager to capitalize on the advanced storage software features available on NEC storage arrays. These include array-based snapshots, array-to-array replication and performance monitoring and management that have been available on NEC arrays for some time but are often underutilized or ineffectively used by enterprises.

While this is attributable to a number of factors, this underutilization may be coming to an end as NEC Storage Manager frees enterprises to tap into the latent value of its advanced storage array features. Among these features that are available on NEC storage arrays, there are three that enterprises will likely want to utilize in their virtualized environment:

Array-based snapshots. Server virtualization creates an opportune time for organizations to update their backup strategies and array-based snapshots are becoming the first line of defense for data protection. Using NEC Storage Manager, enterprises can centrally configure snapshots on any NEC storage array for any application server (virtualized or physical) that has data residing on them.

To ensure the snapshots are recoverable, NEC provides hooks into file systems and Microsoft SQL Server databases. Application consistent snapshots may then be scheduled using NEC Storage Manager to shorten backup windows and remove the overhead of backup from VMware ESX hosts.

Performance monitoring. Monitoring storage array performance is becoming an imperative to effectively pinpointing and troubleshooting virtualized applications that access multiple storage arrays. Using NEC's Performance Monitor feature, which is part of NEC Storage Manager, enterprises can proactively take steps to identify the source of the problem, quickly isolate disk drives that are "hot" or network ports that are congested and take action to resolve the issue.

Array-based replication. Whether the disaster is small or large, enterprises increasingly want the flexibility to recover applications more quickly at another site, especially as they virtualize their application and centralize their data on these arrays. Meeting this objective requires that data is regularly replicated offsite and the replication process managed appropriately.

Using NEC Storage Manager, enterprises can take advantage of the replication software available on NEC's storage arrays, centrally configure NEC's replication software and then monitor and manage this ongoing replication between arrays.

Enterprise administrators love how VMware vCenter centralizes the discovery, monitoring and management of VMware VMs. But as they grow accustomed to its functionality, they are beginning to look for its storage equivalent so they can also centrally manage their storage arrays in the same way.

NEC Storage Manager answers this call as it enables enterprises to move from a micro to a macro approach in managing their NEC storage arrays. By providing a central management console that discovers, monitors and manages NEC D, M and S series storage arrays, enterprises get more than single pane of glass from which to manage storage software features on NEC storage arrays. They can begin to confidently implement NEC's advanced performance monitoring, replication and snapshot features so they can manage and scale their backend NEC storage infrastructure as easily as their virtualized one.

New NEC M100 Delivers What Midrange Arrays Today Must Offer: VMware Integration with Reliability

2011-11-02T13:00:00Z

According to IDC, revenue from external disk storage systems totaled over $18 billion in 2010. But what that IDC number does not fully reflect is the growing impact that midrange arrays are having on organizations of all sizes and how well they are positioned to deliver the other key feature that organizations now want in their VMware environments: Reliability. Among the midrange arrays available, the new NEC M100 storage array is better positioned than most to deliver on these two features.

All midrange arrays share some common denominators in terms of feature functionality as these contribute to the attractive price points of midrange arrays relative to their high capacity and performance. The NEC M100 SAN storage array continues that tradition.

The M100 ships with the latest and greatest hardware components to provide the features that organizations expect any midrange array to deliver. Examples of the hardware that the M100 supports include:

8 Gb FC or 1/10 Gb Ethernet storage networking interfaces. On an NEC M100 organizations may choose from the two fastest available storage networking interfaces: 8 Gb FC and 1/10 Gb Ethernet. This gives them the flexibility to use either the FC or iSCSI protocol in their environment.
6 Gb SAS HDD interconnect. A high speed connection between storage controllers and backend HDDs is a prerequisite to achieving optimal performance. The M100's 6 Gb SAS interconnect assures organizations that the M100 will deliver a high level of performance.
Solid state drives (SSDs). SSDs are a game changer as they dramatically accelerate application performance with read times that are as much as 20X or faster than HDDs. The NEC M100 is on top of this trend with its support for both 100 and 400 GB SSD drives.
High capacity drives. Some shops just require lots of storage capacity. To flexibly meet the needs of these organizations, the M100 supports up to 192 TBs using 2 TB 7200 RPM SAS drives.
Mix of disk drive types. Most organizations have a mix of applications with some that need high performance while others need high capacity. By supporting a mix of SSDs and HDDs, the M100 provides the drives that organizations need to meet their varying application requirements.

Yet in delivering these features, the NEC M100 simply matches what many other enterprise midrange arrays offer. To distinguish itself the NEC M100 does more than perform or scale. It provides the features that organizations need to confidently store data on it and then access it when they need it.

The latest and greatest features on a midrange array are only interesting to organizations if they can count on them to work reliably. In virtualized environments reliability has become even more important since unreliable hardware can impact multiple VMs. To prevent this from occurring, the M100 provides a number of features to ensure optimal reliability.

Active-Active controllers. This high end storage array feature on the M100 is available on less than half of the midrange arrays on the market. It ensures that should an M100 storage controller fail, attached hosts experience little or no interruption in service. Further, this controller configuration ensures that when a periodic 30 minute microcode update needs to occur, it can be done dynamically without application impact.
Delivers 99.999% uptime. Availability is a necessity in virtualized environments as any midrange array outage has the potential to impact the entire environment. To meet this requirement, the M100 is fully hardware redundant so should any component fail another is immediately available to replace it. The NEC M100 also provides a number of alerting options to warn organizations of potential problems so they can be quickly resolved and problems or downtime averted.
No impact recoveries from HDD failures. As more VMs access the M100, even doing a rebuild of an HDD failure may cause an undesirable performance hit to applications. The NEC M100's new SuperPhoenix technology addresses this concern. Prior to an HDD rebuild, it determines the extent of the HDD failure or if an HDD rebuild is even necessary. If it is, the HDD is rebuilt in such a way that application performance impact is controlled and minimized.
Solves silent data corruption in HDDs. All HDDs experience some data corruption over time that other midrange array systems may not detect. Using its SuperPhoenix technology, the M100 constantly monitors HDDs. If an HDD error is detected, the HDD is disabled while the M100 determines whether the HDD should be permanently replaced by running a battery of tests against it. By making this determination, the M100 ensures the integrity of the data without unnecessarily requiring a HDD replacement if it is a fixable error.

In addition to reliability, a growing concern for more organizations is the looming rate spike in their utility bills. Over half of state utilities are seeking permission to pass along hundreds of millions of dollars in rate increases in excess of 10%. This behooves organizations to select midrange arrays that are operationally efficient.

The NEC M100 addresses this emerging concern in at least three ways by supporting:

The use of 2.5" HDDs which require less power and space to operate than its 3.5" predecessor.
Controller processors that run at a thermal design power of 30W. This design is typical of processors from the 1990's which only consume as much power as a 100W bulb.
Autonomous switching to a low power mode. This demands less resources to the point that it even powers off RAID groups that are unused.

This combination of using smaller drives, more efficient processors and autonomously managing its power usage contributes to the M100 consuming less power and generating less heat than other midrange arrays while still delivering on the midrange array features that organizations need to remain competitive.

Finally, no midrange array may be considered a "complete" solution without virtualizing both its storage resources and integrating with VMware. The NEC M100 delivers on both accounts.
Using the NEC M100's dynamic storage pooling feature organizations can mix and match storage in almost any manner they want. These pools can use different HDD types, support the introduction of SSDs into the pool and can even use HDDs that reside in different groups.

The NEC M100 also keeps pace with enterprise VMware adoption by updating its VAAI integration. The M100 now supports VMware's Assisted Hardware Locking, Boot from SAN, Block Zeroing and Full Copy features so that the M100 may be managed holistically as part of an organization's virtualized deployment.

Organizations today are all about building virtualized data centers but that demands they deploy a storage infrastructure which is cost-effective, efficient, highly available, reliable and manageable. The NEC M100 answers that call.

Its Active-Active controllers, SuperPhoenix technology, dynamic storage pooling and VMware VAAI integration provide more than the latest and greatest technology that organizations want in their virtualized environments. The NEC M100 delivers them in such a way that they can be confidently, efficiently and reliably implemented.

Genesis Hosting Uses a Proof of Concept to Reap the Tangible and Intangible Value of Building an In-house Reference Architecture

2011-10-18T11:30:00Z

Virtualization is sweeping through data centers of all sizes and, as it does, it introduces levels of complexity that organizations are ill-equipped to handle. To mitigate this, reference architectures are emerging as a technique to standardize which hardware and software are deployed, under what circumstances, and how it is managed.

As reference architectures proliferate organizations must make a choice: acquire a pre-packaged configuration or build one that matches their specific requirements. While not an easy decision, early evidence suggests that building a reference architecture costs significantly less and better meets their needs than acquiring a pre-packaged solution.

Reference architectures provide data centers a proven template solution consisting of specified hardware and software to helps ensure the physical infrastructure that hosts virtualized applications is deployed, managed and scaled in a predictable manner. This is why hardware providers are taking their existing solutions and offering them as a pre-packaged reference architecture.

However shortcomings with this approach exist as these architectures use products which:

Are not optimized for sharing.
Are difficult to integrate as each has its own management console and API
Are built on the assumption that they will be used by one client
Create silos of compute power and storage capacity

These factors create the exact opposite of what organizations want: an inflexible data center. A pre-packaged reference architecture may require a large up-front investment for their hardware and software that become more expensive to scale over time. Further, organizations limit themselves to a specific provider that must certify all of the components used in its pre-packaged solution.

This limitation of using a single provider continues to inhibit organizations as they look to implement readily available, cost-effective technologies like CPU, memory, network and storage. These are needed to handle the data and storage growth that virtualized environments create as well as to take advantage of compression and deduplication technologies that optimize storage capacity.

However using a pre-packaged reference architecture these technologies may not be available as soon as an organization needs them and then when they are available, come at a price premium that puts them out of reach.

To avoid these drawbacks of pre-packaged reference architectures more organizations are looking to build their own. These offer the predictability and scalability that organizations want but without the pre-packaged lock-in or price premium. Further, building a reference architecture frees organizations to capitalize on readily available memory, processing and storage technologies more cost-effectively.

The caveat to organizations building their own reference architecture is that it is only economical and practical to do if it can be effectively managed. An ad hoc deployment of these technologies without a plan to manage them creates challenges that are even greater than the pre-packaged approach.

A build-your-own strategy therefore demands that organizations introduce the right mix of technologies into their data center infrastructure. To do so, organizations must thoughtfully deploy such a solution by beginning with a proof of concept. Genesis Hosting is one example of a company that has already done so.

Cloud hosting provider Genesis Hosting looked at both pre-packaged and in-house reference architectures as a means to build out its data center infrastructure. In examining them Genesis found that an in-house reference architectures provided it with the flexibility it needed, significant technology advantages over pre-packaged reference architectures and a more predictable pricing model.

Genesis early-on recognized the pitfalls of ad hoc so all new new hardware and/or software it considers for use in its data center must satisfy two criteria. They must demonstrate a technology advantage over competitive products and provide well-documented APIs.

Cutting edge technology is important to Genesis Hosting as it seeks to maintain its competitive edge but so is managing its infrastructure. So all solutions that Genesis considers for use must offer APIs that its management software can call before it qualifies as a candidate for use in its data center infrastructure. However assuming hardware and/or software meet these two criteria, Genesis then subjects it to a proof of concept.

It was NEC's servers, Ethernet network switches and storage coupled with their APIs that recently led Genesis to do a proof of concept. NEC hardware was tested with the following:

Blackball Search-in Software
Nexenta storage software
QLogic FC switches
VMware vSphere

To safely test this mix of hardware and software, Genesis put it into a pseudo-production environment, virtualized the hardware and ran simulated production workloads against it. During this testing period Genesis also verified that its own management software could call the NEC's hardware APIs to test the depth of their capabilities and feature functionality.

During this proof of concept Genesis measured how well NEC responded to Genesis's requests for information about their products and technology. In this particular test, Genesis wanted access to the NEC product managers to gauge how candid they were in discussing what NEC's equipment could and could not do.

While Genesis Hosting's CEO Eric Miller was pleased with the performance, stability and robustness of the NEC hardware and software, NEC's candor about what it could and could not deliver were equally pleasing to him. Genesis wants a reliable platform for hosting its customers' applications and prefers a known quantity. Miller says, "NEC's proven features and known limitations are better than promised features and untested capabilities of other solutions."

It was what Genesis learned about the performance of NEC's hardware coupled with the ability to access and manage its hardware from the proof of concept that led Genesis to ready NEC for use in its production environment.

Genesis Hosting's high interest in the robustness of NEC's equipment was directly tied to how it allocated hardware resources in its data centers. Genesis Hosting creates a virtual data center (VDC) within its physical data center for each of its clients. Each VDC can access and use any physical resource in the data center so long as the underlying hardware is both virtualized and addressable by Genesis Hosting's management software.

To use NEC in this environment, Genesis Hosting's configured its management software to call NEC's equipment so each VDC would remain:

Self-Managing. Any Genesis Hosting client can access its VDC at any time and independently provision physical resources or run applications without needing the help of Genesis
Scalable. As its clients' VDCs consume physical resources and need more, Genesis can non-disruptively provide it to them.
Shared resources. Genesis shares its physical resources between its clients' VDCs by using the APIs to monitor resource utilization and dynamically allocating them where they are most needed
Secure. Genesis uses NEC's storage software functions to ensure VDCs do not trespass on what belongs to another VDC

Organizations want the physical infrastructure that supports their ever growing virtual environments to provide the performance, predictability and stability they need. But to get these attributes organizations do not have to feel obligated to pay a premium for pre-packaged reference architectures.

As Genesis Hosting found when it completed its recent proof of concept, NEC's technology and products provide what it needs to continue to readily grow its reference architecture. While it had to take the appropriate precautions by first putting NEC through a proof of concept, Genesis found that it could confidently build a viable reference architecture at a fraction of the price of pre-configured solutions and take this solution from testing into production without posing any risk to its clients' VDCs.

To read more about how Genesis Hosting leveraged NEC's hardware to first do a proof of concept and then build a reference architecture, click here.

67% Performance Boost and WORM Functionality Highlight Latest NEC HYDRAstor Software Release

2009-09-15T13:20:00Z

The NEC HYDRAstor grid storage platform is best described as one that can evolve and grow on an as-needed basis while introducing newer technologies as they become available. Today's announcement continues to reinforce this compelling value proposition that the HYDRAstor platform delivers. In today's software release NEC improves HYDRAstor's performance, adds in-flight encryption to its replication feature and extends its support of application-aware deduplication to Tivoli Storage Manager and EMC NetWorker all at no extra charge to HYDRAstor users.

Ever since the HYDRAstor was first released, its grid storage architecture has served as an example of how easy a storage system can be to manage and grow assuming it is designed with today's user requirements in mind. Today's new release is no exception to that as the new software extends or improves upon HYDRAstor's capabilities in three main areas:

1. Up to 67% performance boost. This performance boost is based solely on doing a software upgrade for the HYDRAstor with no requirements for additional hardware or new license fees. Prior to this release each HYDRAstor Accelerator Node provided up to 300 MB/sec. By simply upgrading the HYDRAstor software, performance will jump up to 500 MB/sec per Accelerator Node or 1000 MB/sec for a standard two Accelerator Node configuration. This translates into faster backups as each node is able to process the data inline quicker.

What is also of note is that these new performance numbers are published for general applications and do not assume a scenario that is only applicable for a very specific application with a rare configuration as other storage vendors tend to do. This means more users can achieve the published performance numbers, and users with high duplicate data rates in their backup streams could see even higher performance numbers that what NEC is publishing.

2. New encryption and chargeback capabilities. One of the tenets of NEC's design of the HYDRAstor is that core software functional improvements that most customers need should be made available to both existing and new HYDRAstor customers at no additional charge. Today's release continues that trend as it has added in-flight encryption of replicated data as well as a new storage chargeback capability.

Simplified in-flight encryption of data. More enterprise organizations are looking to protect their remote offices as well as replicate data in their main data center to a secondary site. But as they do so they want to ensure that the data is encrypted while in-flight so it cannot be used for nefarious purposes in the event it is intercepted or compromised.
To accommodate this request that NEC has received from many of its customers, NEC has added an encryption feature to the HYDRAstor software base, which first encrypts data before it goes over the wire and then decrypts it on the other side so it can be read. This inline encryption can be enabled or disabled at a file system granularity and is available as a free enhancement for all Accelerator Nodes that have licensed HYDRAstor's RepliGrid WAN-optimized replication.
Storage charge back enforcement with file system-based quotas. Storage charge back was another feature requested by NEC's customers that were seeking to designate different quotas for different file systems or applications so they could manage the entire HYDRAstor storage pool. This new feature should help organizations better respond to today's tough economic conditions by first helping them understand the storage consumption of individual applications, departments or business units, and then charging them accordingly.

3. Enhancement of existing HYDRAstor functionality:

Increased flexibility in resiliency depth settings. The HYDRAstor has always offered the flexibility to increase or decrease the resiliency of its system. Now organizations can dial the resiliency level up or down at the file system level. Using this enhanced feature, users can assign a particular application to a specific file system and based upon the importance of the application and its data, adjust its protection up or down.
This feature becomes very handy as many scratch, reference, or test data can now be assigned to HYDRAstor file systems that have a lower level of resiliency and lower overhead while the data associated with critical application data can be dialed up to meet their higher protection requirements.
Capacity on Demand. More organizations are looking to deploy disk-based backup into remote offices but need to justify the cost. To accommodate these organizations, NEC is providing new smaller models that can start at lower levels of capacity and price but still get all of NEC's software features. NEC is also making it possible to expand the logical storage capacity of the HYDRAstor system at a later time via a software license, without requiring the addition of any hardware components.
Extending application-aware deduplication. NEC announced application-aware deduplication in June 2009. This feature interrogates data as it comes in from a particular application (such as backup or archive software) and determines whether or not the data is a duplicate after it filters away metadata that the application has injected into the data payload.
In its first release of application-aware deduplication, HYDRAstor supported CommVault® Simpana® and Symantec NetBackup, but this release adds application-aware support for Tivoli Storage Manager (TSM) and EMC NetWorker.

Another significant feature that NEC added with this software release is HYDRAlock WORM (Write Once Read Many) support, which can be licensed separately. With HYDRAstor's implementation of WORM, users can configure individual file systems with different policies for different applications to automatically secure files so they cannot be deleted or altered. Similar to the licensing model used for RepliGrid, HYDRAstor's WAN-optimized replication, the new HYDRAlock WORM functionality can be licensed on a per Accelerator Node basis so the customer only pays for the part of the system that needs it.

HYDRAstor's beauty is that organizations can scale it as they need, get the features they need as they need them (through software updates such as are found in this release) and not have to worry about some painful, pending storage migration because of HYDRAstor's grid storage architecture. Economically and safely keeping information on disk for longer periods of time without the hassles associated with storage system upgrades are now the keys for organizations to getting more value and reducing their spending on storage, and NEC continues to deliver on these key areas.

As organizations continue to manage data, it does not make much sense for them to continually spend monies for features and enhancements that should be part of a product's natural evolution. The NEC HYDRAstor is part of this new trend. It adds new features and enhancements such as application-aware deduplication, more granular setting for data resiliency, reductions in up-front storage costs, in-flight encryption, and performance boost free of charge.

These new features further position the NEC HYDRAstor as the platform of choice for archive and backup data by providing more value, more capabilities, and more efficiency while still keeping organizational costs associated with managing the data under control.

Deciphering Application Metadata is Data Deduplication's Next Frontier

2009-09-11T18:00:00Z

Dedupe is an easy concept to grasp. At its most basic level it reduces storage requirements and touts the improvement in backup and recovery times. It seems as if it is a "win-win" scenario and, for the most part, it is. But let's not lose sight of the fact that dedupe is still in its infancy and is being continually fine-tuned and changed. This should keep us from becoming lackadaisical in our perception of this technology and how it is still in its early stages.

ADMAD: Application-Driven Metadata Aware De-duplication Archival Storage System, a paper co-written by multiple authors from Tsinghua University in Beijing, China, and the University of Minnesota, reinforces this belief. It states in regards to dedupe that, "There are still research challenges in current approaches and storage systems, such as: how to chunk the files more efficiently and better leverage potential similarity and identity among dedicated applications."

We would encourage many of our readers to go ahead and read the above mentioned paper. It specifically proposes Application-Driven Metadata Aware De-duplication (ADMAD) as an option to interrogate metadata information at the application layer to identify and dynamically define meaningful data chunks to maximize the effect of the dedupe process.

The problems that introducing application awareness into deduplication solves can be succinctly summarized. Every backup application inserts its own metadata into the backup data stream so that it knows how to manage the data. But this can result in problems later on if the target device ingesting the data does not understand this metadata as it deduplicates the data.

The first time the target chops up the data may be completely different than the second time if the deduplicating target cannot decipher it. It is quite conceivable that even though the incoming data in the second stream has not changed, the application sending the chunk of data to the target may tag it differently just because of how the application chooses to manipulate the data. In other words, the data is the same but the tag is different. Yet because this tag now resides inside the chunk of data instead of where the target is looking for it, the data will not be optimally deduplicated.

Deduplicating storage systems like the NEC HYDRAstor may not be doing everything as prescribed by ADMAD, but they are making progress in this important area of incorporating application awareness within deduplication algorithms. About three months ago, NEC announced the addition of application awareness deduplication into its HYDRAstor grid storage platform. The HYDRAstor can now optimize deduplicated data more efficiently for specific applications by analyzing incoming data streams and filtering application level metadata to eliminate the negative impact that metadata has on data deduplication ratios.

HYDRAstor, after an application type has been associated with a file system, will automatically filter the metadata away from the backup data payload which lets its data deduplication process work exclusively on the incoming data. Since different applications have their own metadata that they insert into the data stream to manage 'their' data, once that metadata is split out with corresponding offsets it is stored independently to ensure the data is reconstituted in its original form during reads and helps insure much higher deduplication rates can be achieved.

With its initial implementation of application-aware deduplication, HYDRAstor deployed solutions for CommVault® Simpana® and Symantec NetBackup with early results showing impressive gains in data deduplication ratios for both of these applications.

NEC has seen a 2-3x improvement in data deduplication ratios for CommVault for a full weekly backup cycle consisting of one full backup, followed by six incremental backups, and then another full backup. When implemented within a customer's environment the CommVault solution actually resulted in over 4x improvements in the deduplication ratio, when compared to deduplicating data generically/agnostically, over a span of 4 weeks which represented the customer's retention period.

Realize that your mileage may vary as data patterns in organizations vary. However the ADMAD study suggest that gains of 50% or more in data deduplication ratios can be achieved when stripping out metadata as compared to using application-agnostic deduplication approaches. The fact that the HYDRAstor showed such notable gains reinforces the conclusions of the ADMAD study and it stands to reason that organizations should expect to see similar improvements in their deduplication ratios when using the HYDRAstor in their environments in conjunction with either of these two backup software platforms.

The inclusion of application-aware data deduplication in the HYDRAstor is a smart move on NEC's part. Not only is it a competitive differentiator but current and new HYDRAstor customers get this feature for free as part of the base code that is installed on their HYDRAstor. Such gains in data reduction ratios coupled with its increased ability to show more value only make it logical for NEC to look to expand the number of backup software applications that will support this feature going forward as it should prove to be a factor that organizations look at when deciding between NEC and a competing product.

A 3-Step Plan to Delivering Backup and Recoveries with Higher Success Rates and Less IT Oversight

2009-07-16T10:00:00Z

582,226. That's the number of individuals laid off at America's largest companies as of July 10, 2009, according to the Layoff Tracker on the Forbes website since November 1, 2008. And, depending on what date you are reading this blog, that number has probably gone up. But what this number can not quantify is how layoffs are forcing the individuals left behind within these Fortune 500 organizations to cope with increasing workloads and nowhere is that problem more acute than within IT.

Despite these workforce reductions, most studies still show storage growth continuing at a robust 30% or higher through 2009. Even though a recent report from IDC forecasted a 3.1% decline in storage spending from 2008 to 2009, IDC still expects those systems sold to result in a net increase of storage capacity by 44% versus normal year-over-year capacity increases of 50 - 60% (even though it may not translate into more systems to manage since storage density increases accordingly). Another survey by Applied Research that was conducted at about the same time that IDC report was released contacted 400 IT professionals and it found that in their organizations most (over 90%) anticipate that their storage budgets for the next 6 months to 2 years will stay the same or even increase.

What this means from an IT perspective is that even as the number of people available to manage this data is reduced, there is now more data to manage, protect and recover than before. To cope with this growth and protect and recover their organization's data using fewer people, IT can take the following steps to resolve this:

Bring in a disk-based storage system that supports deduplication. Disk-based backup and recovery improves success rates to 98% or higher versus 60 to 80% success rates using tape and, and by reducing the required capacity for the same amount of data through deduplication, the storage system can scale to handle this continuous data growth.
Purchase resilient, reliable disk-based deduplicating systems. Using commodity disk storage systems as backup targets may be appear economical on the front end but as backup storage capacities grow, so do their costs with some estimating that management and operation costs often come in at several times the acquisition cost of these systems. Organizations need to purchase resilient, disk-based deduplication backup solutions that automatically handle tasks like disk drive failures with little or no intervention required on the part of IT staff to fix these types of common hardware issues.
Obtain scalable disk-based backup solutions that can start small and grow large. Most organizations are dealing with budgets that are flat right now but are forecasted to grow in the future. The last thing organizations want to do is buy a disk-based backup solution with available budgetary funds that only meets their short term needs but then does not scale to meet their future needs. Just because budgets are limited does not mean organizations cannot put a solution in place that meets their needs now and into the future.

NEC HYDRAstor is one solution that fits this list of criteria by providing organizations the building blocks that they need to meet their disk-based backup needs now and into the future. Because HYDRAstor is based on a grid architecture, organizations can start small and then scale by adding either more Accelerator Nodes for higher performance or Storage Nodes for greater capacity at a future time as their demands dictate and budgets permit without needing to replace the entire system.

Further, because the HYDRAstor distributes deduplicated data across multiple nodes, in the event of a disk failure or even the failure of an entire node, data is protected with and always available with minimal or no intervention required by the administrator. Because the system is also future evolving, newer, denser and faster components can plug in directly into the existing grid and the system can takes care of distributing the I/O load and account for any differences across components. This allows organizations to maximize investment protection by extending the life of hardware purchases.

The loss of over 500,000+ jobs in Fortune 500 organization in the last eight plus months is a pretty telling statistic about the current state of the economy. Yet in the face of this, organizations still have to deliver on their most critical functions of which backup and recovery clearly belongs. To meet that objective, organizations need new technologies that improve efficiency and reliability, scale to meet future needs and take less time to manage and support. The NEC HYDRAstor is clearly one technology that should be on the short list of products that organizations need to consider for this purpose.

There are Better Options to Disk-based Archival and Backup than Using the "Off" Button

2009-05-07T10:00:00Z

Recently Kelly Polanski (another DCIG analyst) and I had a rather lengthy discussion about the value of keeping archive and backup data on disk versus tape long term. We were both in agreement that using disk in some form as an initial backup target makes sense in most environments but as we started to debate the merits of keeping data on disk versus tape long term, the issue can get more cloudy. While DCIG has previously argued that eDiscovery is becoming a more compelling reason to keep archive and/or backup data on disk long term, the concerns we had centered on the fact that some disk-based archival and backup storage systems can become as problematic as tape.

Kelly is talking to some end-users and resellers where disk storage systems are starting to lose some of their shine when used in archiving and backup capacities. When I pressed her further, she indicated that these accounts started using disk for archive, backup or both which many times solved their initial problems around backup by shortening backup windows and creating more successful backups. But they are now encountering new challenges that are just as serious as the ones they used to have with tape. Key issues include:

Inadequately size for capacity. This may force the organization to introduce another storage system or appliance to archive more data so now they need to manage multiple devices.
No way to extend the deduplication benefits from the first appliance. Many archiving and backup storage systems now deduplicate data but if a second appliance is introduced, organizations lose whatever benefits of deduplication that the first appliance delivered since the new appliance needs to start deduplicating data from scratch.
Inadequately sized for performance. To rehydrate large amounts of data for either recovery or eDiscovery, searches could put inordinate loads on the appliance that results in unacceptable slow recovery or search times as well as concurrent normal backup or archive access.
Some archiving storage systems can only be accessed via proprietary network APIs. Only applications that support the proprietary network APIs of the storage system can access or retrieve data from the archiving system.
No upgrade path. Once the data is on the storage system, how do you upgrade the system or migrate the data off the system to a new one? Kelly told me that some companies she is dealing with have simply stopped trying because it is too expensive to either migrate the data or upgrade the system. She is aware of at least one company that simply turned their archiving storage system off because it was full. Now they only power it on only when some business need dictates.

The last situation is almost comical if it was not true and you can be sure that if one company is doing this, others probably have already done so or are thinking about it. Granted, I do not know all of the circumstances surrounding this event but it goes to the point that deploying disk-based storage systems for either archive or backup requires more forethought than most companies are giving it

In the near-term, disk almost always expedites archives, backups, eDiscovery searches and recoveries but as the days turn into weeks, weeks into months and months into years, can the system really adapt its capacity and performance to meet the growing data volumes that are sure to come? Obviously many systems cannot.

Organizations who are already encountering problems with their disk-based storage systems - or who are looking to avoid these types of episodes - are advised to look at storage systems such as the NEC HYDRAstor that is based on a grid storage architecture. It can scale either capacity or performance independently since users can add new Accelerator nodes for performance or Storage nodes for capacity. It can also eliminate the upgrade and deduplication problems that other storage systems present since all HYDRAstor nodes are part of one logical configuration. All nodes integrate into a single shared pool of storage and deduplication is distributed and processed globally to guarantee a single copy across the entire grid. New nodes are dynamically added or even used to replace aging, out-of-date technology. Older nodes can be retired, and the system automatically transitions the data onto the remaining nodes.

I'm sure that the conversation that I had with Kelly is not an isolated incident and should illustrate that organizations need to think more strategically about the disk storage systems that they use for their archive and backup needs. Using almost any disk system in lieu of tape for archive and backup can clearly often solve some immediate archival or backup problems but they can create larger, more complicated issues down the road. It is for these reasons that organizations should consider new storage systems such as the NEC HYDRAstor that meet today's needs but can also scale to meet tomorrow's needs as well.

The Cost of NOT Keeping Archival and Backup Data on Disk

2009-02-24T11:00:00Z

Over the last few months DCIG has spent fair amount of time researching and documenting specific reasons why tape will not die. Green IT is the one reason we most often hear cited for retaining tape, though new disk-based deduplication and replication technologies coupled with new disk storage system designs that are based on grid storage architectures can offset some of those concerns. So before organizations think that after 30, 90 or 180 days that they should immediately move their archival and backup data, deduplicated or otherwise, from disk to tape just to save money, there are certain intangible savings from an eDiscovery perspective that keeping data on disk provides that are not always feasible on tape.

Immutability, portability and remote recoveries along with capacity and power savings are reasons sometimes used to justify copying some archival and backup data to tape. But if organizations focus solely on these reasons, they are failing to account for the new needs that today's more litigious environment is creating to access and search these data stores. Specifically, as organizations respond to Electronic Data Discovery (EDD) requests during periods of litigation, they may find that the costs of recovering and searching data on tape to do eDiscovery offsets whatever cost savings that tape initially provided.

According to a recent article by George Socha and Tom Gelbman, their 2008 Socha-Gelbmann survey that surveyed the state of eDiscovery in 2007 found that commercial expenditures on EDD topped $2.7 billion in 2007. Further, they forecast expenditures on eDiscovery will grow by 21% in 2008, 20% in 2009 and 15% in 2010 which equates to a $4.5 billion market by 2010.

However whether or not your organization participates in that trend will likely depend on how you store your data. Charles Skamser, VP of Sales and Marketing for Trial Solutions, makes a point in a post on his ediscoveryconsulting blog that 2006 changes to the FRCP (Federal Rules of Civil Procedure) create new requirements for organizations to utilize new eDiscovery technologies but how much you pay will likely vary according to the sort of media you store you data on. The cost to simply index 2 million files stored to tape can run as high as $75,000 while the cost to access, index and search archived and backup data stored to disk can be a fraction of that.

This is where keeping archived and backup data online on disk as opposed to on tape can result in dramatic cost savings for organizations. If data on tape needs to be restored so it can be accessed and searched to satisfy the requirements of the eDiscovery request, this could take hours, days or even weeks to complete. This does not take into account the extra storage capacity that might be required to recover all of this data and/or the time required to index it and search it.

So if organizations spend all of this time just retrieving the data, it leaves them little or no time to assess it. Since the FRCP only gives organizations 30 days to respond to an EDD request, they may find themselves unable to produce the data or, if they produce the data, unable to fully understand the nature of the information they just shared and the implications to their organization.

Compounding the problem, failure to provide the information could result in the presiding judge issuing the dreaded "death instructions". When a judge issues these instructions to a jury, he or she is essentially communicating to them that your organization has something to hide and should be viewed in the most negative light. So when one starts to add up the costs of retrieving the data from tape, the shortened length of time or inability to assess the recovered data before sharing it and the possibility that you may lose the case simply because you can not access and produce the data quickly enough, suddenly using tapes for "Green IT" does not look so appealing any more.

These new eDiscovery demands that are being driven by updates to laws like the FRCP make it clear that aging backup and archival data stores are not as static as in the past. As more large organizations look to balance how they use disk and tape in their environment, moving some data off to tape certainly makes sense for some of the reasons I mentioned above. But for organizations to assume they can move all of their archived and backup data off to tape after a set period of time may expose them to increased eDiscovery costs and even lost lawsuits because they produce data too slowly or not at all.

Organizations are right to weigh the costs of keeping data on disk or tape but, as they do so, they also need to factor in today's more litigious environment and the cost of not having instant access to these data stores. New disk-based deduplicating grid storage systems like the NEC HYDRAstor make these decisions between disk and tape a little easier to make. While the HYDRAstor can cost-effectively keep data on disk long term, it also provides an option for organizations to copy data off to tape without impacting day-to-day backups or recoveries. Further, since the HYDRAstor uses standard network file sharing protocols, organizations can potentially use new eDiscovery tools to access and search stored archival and backup data on the HYDRAstor without first having to recover the data.

Tape has its place in today's data centers but organizations need to consider more than just cooling, heating and power when deciding how to best manage their data. We live in the information age and the inability to access and search data is no longer a luxury that most organizations have. Changes to laws like the FRCP and new ones being promised by the current Congress should make every organization realize that whatever savings tape may provide may be more than offset by an organization's inability to respond to legal eDiscovery requests and the subsequent court losses that may result.

Simplifying the Mind-Boggling Exercise that Pricing a Deduplication Solution Can Become

2009-02-02T19:00:00Z

Almost 3 years ago now, Robin Harris over at Storagemojo.com starting posting the list prices for different vendor's products so customers have at least a starting point when comparing product prices. Though I suspect the list prices associated with these vendors' offerings have changed since he originally posted some of them, what I specifically found remarkable is how difficult it is to ascertain what a deduplication solution will cost for an organization. The difficulty in pricing deduplication solutions had less to do with making sure you getting deduplication than making sure you include in your configuration all of the options that your environment needs, such as failover, NAS or VTL interfaces, data retention periods or replication, to effectively compare different solutions.

Pricing deduplication solutions becomes especially difficult in enterprise organizations as they look to introduce deduplicating solutions into their environment for the first time. They may not always have all of the information that they need to properly size an appliance so they end up having to make an educated guess as to exactly how much data they are going to back up, how long they will retain it and what the throughput rates during backup windows are going to be.

This was the case recently for Memorial Care Medical Center in Long Beach, CA, as reported by SearchDataBackup.com. Memorial Care discovered after the fact that it had implemented an undersized deduplication solution. Nightly it was backing up 20 TB of data but the deduplicating solution only had 13 TB of raw capacity so after only a month, it ran out of space. When this occurred, it elected to purchase another deduplication solution which added both more cost and complexity into their environment since it now had multiple backup targets and more devices to manage.

But Memorial Care Medical Center's situation is not unique. Doing all of the research necessary to put together an appropriately sized solution is an arduous task and, even then, circumstances can change. I wrote an article a couple of years ago where the Minneapolis law firm of Winthrop and Weinstine had done its homework, forecasted for growth and selected an appropriate solution. But an unexpected court case generated 1.5 TB of new backup data so the law firm, after only a month, had to start looking for a deduplication solution with more capacity.

In order for enterprise organizations to avoid these situations as they price and compare different deduplication solutions, they should take the following steps:

First, factor everything into the price of the deduplication solution. Organizations need to make sure the solution has all of the features that they are going to need and use. Examples include compression, data resiliency, deduplication, failover, number of Ethernet ports, replication, sufficient capacity and performance scalability. As these aforementioned organizations found out, overlooking or failing to account for even one of these features can result in an organization needing to change out or purchase another solution much more quickly than they anticipated.
Second, how much time will it take to manage? Organizations may assume that if they run out of capacity, it is no big deal since they can just add another one and keep going. That sounds good on the surface but the practical ramifications are that they need to go through the whole capacity sizing exercise again. Additionally, the ongoing management of multiple appliances becomes more difficult since administrators need to configure and direct specific backup streams to specific appliances. This may then require they constantly manually balance the load between them so they do not overload specific appliances with too much backup data plus administrators are less efficient since they have to manage separate appliances.
Third, how are upgrades and maintenance handled? Again, this will not show up in a quote but if administrators need to regularly spend time applying upgrades and managing the growth of the appliances over their life, that adds to the total cost of ownership plus takes away time that the administrator could be doing tasks that add more tangible value to the organization.

In this respect, the NEC HYDRAstor is unique in that it helps organizations avoid most of the time-consuming, information-gathering sizing steps that other products normally require organizations perform. Yes, organizations still need to determine if they want a feature like replication, but HYDRAstor includes all other fundamental options as standard and minimizes the need to make many of the other upfront decisions that implementing deduplication normally requires.

Since the HYDRAstor is based on a grid storage architecture, organizations can dynamically add more performance or capacity in the form of Accelerator or Storage Nodes at any time. This allows organizations to start with a configuration and price point that matches their initial requirements and which they can then grow as quickly or as slowly as they want.

Most users find understanding and documenting one's backup environment complicated enough without having to go through the exercise of trying to gather all of this information to make an educated guess about which deduplication appliance to buy. The NEC HYDRAstor eliminates the need to do this guesswork in the first place plus it shortens the time that organizations need to price a configuration, eliminates much of the time associated with managing it after it is deployed and takes the risk out of purchasing a solution that will not grow as their environment grows. So while NEC has not quite done away with price lists and feature options, it does make pricing a deduplication solution a manageable exercise for enterprise organizations so they can more quickly and confidently arrive at a price quote that they can trust and rely upon without a bunch of hidden "gotcha"s waiting for them after deployment.

Storage Fundamentals are the Basis for New Storage Innovation

2009-01-27T19:00:00Z

Innovation within the data center seems to be on the lips of IT managers, vendors, and analysts alike. Innovation, it is said, will pull us through this economic downturn even as organizations experience cutbacks in budgets, staff and just general doom and gloom. These innovations include maturing technologies such as virtualization, grid computing and deduplication coupled with management initiatives like consolidation, outsourcing and reduced expansion. These ensure organizations can continue to cut costs and stay on budget while creating more efficient data centers that are ready for whatever tomorrow brings.

In a recent SearchStorageAsia article, "Gartner Predicts 2009: Innovation despite economic downturn", Robin Simpson, Gartner Research Director, makes it clear that organizations will need to realize that taking hold of innovation is what will bring about improved efficiencies and reduced costs. In the article, he says, "Technology continues to advance at a rapid pace but ... organizations that postpone essential technology deployments do so at their peril".

According to analyst firm IDC, data storage is expected to grow at more than 50 percent per year between 2008 and 2010. This puts enormous pressures on organizations to select the proper storage architecture that helps stay within budget while providing the ability to keep up with the expected dynamic and explosive growth. But what we fear is that many data centers are jumping on the innovation / feature bandwagon without much thought and forgetting about the fundamentals of that storage architecture.

Availability, reliability, and performance are still king in enterprise computing and especially in enterprise storage. In fact, without them, we might as well be doing our computing on an abacus. But since these three features are so tightly coupled together, they should be of high concern when deploying a storage solution - innovative or otherwise.

In this respect, NEC is taking no chances with its HYDRAstor storage system. While it introduces a grid storage that lays the groundwork for future expansion and eliminates many concerns organizations have about expansion and growth, it does not require organizations to sacrifice their concerns about these computing fundamentals either.

As a fully distributed platform, the HYDRAstor grid architecture is not limited by any single central resource and thus offers, through the simple addition of Accelerator Nodes or Storage Nodes, scalability in performance or capacity that screams of high availability, performance and reliability. This ability to add new nodes into an existing HYDRAstor system without provisioning, forklift upgrades, downtime or data loss allows the HYDRAstor to be more scalable, reduce total cost of ownership (TCO), and extend the life of older technology. This enables IT to quickly and effectively leverage this platform in support of changing business requirements.

But the beauty of the HYDRAstor architecture is that it still behaves as a single autonomous, self-configuring and self-healing system. It can automatically and non-disruptively load-balance across nodes to optimize performance. Adding to the protection of data and extending on the architectural availability and reliability features is the HYDRAstor's Distributed Resilient Data technology (DRD) which enables users to set protection levels according to business data criticality. For example, setting the HYDRAstor's resiliency level to three will protect against the failure of three drives or three nodes concurrently.

While technological advances are imperative, the traditional fundamentals of availability, performance and reliability that enterprise organizations demand cannot be ignored. The NEC HYDRAstor grid architecture delivers on these essentials while laying the foundation for what organizations will need to be successful in the data center of tomorrow. The stress of today's economy is adding to the pressure of organizations to innovate but, as Gartner's Simpson points out, those who fail to innovate even in these tough times do so at their own peril. The NEC HYDRAstor provides organizations a mechanism to innovate in their storage infrastructure without sacrificing the fundamentals of what made data centers successful in the first place.

Deduplication Guarantees do not Guarantee Money in the Bank

2009-01-19T11:00:00Z

Are deduplication guarantees really something you can take to the bank? As more companies look towards using disk in general as a backup target and deduplicating systems specifically, deduplication guarantees are emerging as a way to influence users' decision to deploy deduplicating systems. But in these tightening economic times, deduplication guarantees do not necessarily guarantee money in the bank and may shift your attention away from more critical evaluation criteria such as system reliability, scalability, and performance.

On the surface, deduplication guarantees of 20:1 or higher may sound like a good idea. Assuming recent surveys are accurate, most users have not yet implemented deduplication in production but plan to do so over the next couple of years. The lure of a guarantee is that it can take some of the uncertainty out of making the decision to deduplicate backup data by providing some re-assurance to the user that if the system does not deduplicate as promised, the vendor compensates them for the shortfall.

But when one takes a deeper look at the deduplication guarantees currently available, holes in the terms of these guarantees become more evident. For instance:

The guarantees are only applicable to specific applications that satisfy certain conditions. For instance, unless companies plan to backup and deduplicate data from Exchange or Oracle a specified number of times per week, the deduplication guarantee may not apply.
The vendor is the final arbitrator as to if the conditions of the guarantee are met. You need to use the vendor's software to measure the results and then the vendor interprets the results. This is not to imply the vendor will lie but it does raise some questions as to the validity of the results.
Remuneration is vague. If the vendor fails to meet the terms of the guarantee, the compensation may consist of additional hardware/software or services from the vendor. Yet in reading through the terms of the various guarantees on the market, the associated corrective actions are vague. For instance, if you need more storage capacity because the guarantee was 20:1 and you only achieved 15:1, how does that translate into software, hardware, or services that the vendor will provide?

So if deduplication guarantees are not the answer, what is? A better approach is to deal with vendors that are willing to bring their solutions into your environments and then let you test them to see how they actually perform under real-world conditions. NEC offers companies that option with its HYDRAstor. NEC will provide qualified companies the opportunity to run tests with an appropriately sized, production ready HYDRAstor. The HYDRAstor is sized to match the needs of that company's environment that the company can deploy, run for the needed period of time, get actual deduplication results and then make an informed decision about how to best proceed with the NEC HYDRAstor since they can see the real cost savings that deduplication provides. This approach will also allow you to evaluate more critical aspects of the system such as reliability, scalability, and performance, to ensure the system can address both current and future requirements.

Deduplication guarantees may provide some short term assurances but are not money in the bank as they are for now still too narrow in terms of the benefits they offer. Even if they offer a full return policy, very few organizations can afford the disruption and churn of replacing a production system due to failure to deliver on a deduplication guarantee. While there may eventually be a time and a place for deduplication guarantees, the conditions that current deduplication guarantees attempt to satisfy are not reflective of the reality of most customer environments. Instead, companies are better served by choosing solutions such as the NEC HYDRAstor that give companies the opportunity to prove the ROI while avoiding the need for them to make a leap of faith that a deduplication guarantee is intended to provide.

Data Migration and User Account Management: Grid Storage Tackles the Hidden Issues of Storage System Management

2008-12-03T11:00:00Z

Having managed multiple types of storage systems from multiple different storage vendors, there are two flaws that are common across many vendors' storage systems: the inability to transparently migrate data to subsequent generations of their own hardware and the inability to share administrative permissions with other like storage systems from that vendor. How acute this problem is depends on how many storage systems a company manages and how often it replaces them. However any administrator that is responsible for managing five, ten or more storage systems in today's enterprise corporations understands exactly what I am talking about.

As administrators know all too well, the task of managing storage systems (firmware upgrades, user management, storage allocations, etc.) eventually supplants the task of managing the application data residing on those storage systems. Yet managing multiple storage systems is now part of the enterprise storage management equation. Because of this, companies need to factor in how they will manage multiple storage systems from one console as well as migrate data from old systems to new systems to upgrade performance or storage capacity.

To be fair, there is some progress in the area of storage system administration. Companies can now manage multiple storage systems from one console so the same administrator or user account is present across all of the storage systems. In essence, storage system vendors are borrowing the concept of network directory services such as Microsoft Active Directory or LDAP and even starting to grant their storage systems access to these network services to simplify the task of user account management on storage systems.

However the bigger problem of data migration remains the Achilles Heel of storage system managment. If a storage system is fully depreciated, its technology is antiquated or the system is at its limits in performance or storage capacity, there is no easy way for a company to migrate data from an existing storage system to a new one without going through lengthy and painful data migrations.

Further, with disk becoming the primary target for archiving and backup, an interesting phenomenon is occurring - companies are discovering that they do not want to spend more money or time managing these systems. Essentially, they want all of the benefits of using disk for archive or backup without all of the associated downsides because they no longer have the time, patience or resources to deal with managing it (i.e. administration, data migrations and inability to scale and upgrade existing systems).

It is exactly these sorts of constraints that storage systems such as the NEC HYDRAstor that are based on a grid storage architecture address. In previous blog entries I have described how the HYDRAstor can scale storage capacity or performance independently through the use of its Accelerator and Storage Nodes. What is equally as powerful is that the HYDRAstor retains information about the data on the system as new nodes are added or old ones removed. This solves the problem of retaining user account information as the HYDRAstor grows and evolves as well as eliminates the need for companies to create new user accounts.

Even more importantly, the HYDRAstor automatically migrates data from old storage nodes to new ones and then evenly balances the storage capacity across its storage nodes. This automatic migration feature eliminates the normal pain that companies experience planning for outages and how to best migrate data so it laid out appropriately on the new system as well as just the task of actually migrating the data and monitoring its progress. Instead the HYDRAstor takes this task on in a non-disruptive manner for the life of the system.

The reality of grid storage is that companies can change their focus from managing the storage system to once again managing their business. Grid storage provides companies the new type of storage system that takes care of itself by self-managing its administration, growth and data migrations for as long as the business uses the system. Storage systems based on grid storage lay the foundation to become a company's one system forever that allows companies to take advantage of continual advances in capacity and performance, take the pain out of data migrations and preserve user accounts as the system grows and evolves.

Why NAS is now practical for Disk-based Backup in Enterprise Environments

2008-10-22T18:00:00Z

NAS is sometimes viewed as a challenge by enterprise shops if their intent is to use it as a target for disk-based backup. Two reasons often cited is that there is only a finite amount of storage capacity available on NAS and backup software does not handle out-of-space conditions on file systems very well. This causes failures in backup jobs as well as performance bottlenecks when multiple backup jobs are occurring . The use of grid storage architectures in products like the NEC HYDRAstor are helping to put some of these concerns to rest and making NAS a more practical option for use as a target for disk-based backup in enterprise shops.

Here are the typical concerns that enterprise shops have about using NAS as a backup target and how the HYDRAstor addresses them:

Limited file system capacity. Backup software and NAS systems are not always the best of friends when it comes to disk-based backup. While some backup software has the ability to manage out-of-space conditions on file systems, not every product does, so jobs can hang or abort unless an administrator manually intervenes. The HYDRAstor averts these situations by presenting to the backup software a file system that is essentially of infinite size (256 PB) and can be easily extended on the fly if/when such a need does become a reality. Not only is a 256 PB file system large enough to give even large enterprise shops sufficient headroom to meet their backup storage capacity needs into the foreseeable future, the HYDRAstor also deduplicates and compresses the data as it is stored to further alleviate storage capacity concerns.

Inability to scale storage capacity. The HYDRAstor does not just support one 256 PB file system but as many as administrators want to create. However it does not require that companies purchase all of that storage capacity at once but enables them to add it over time to the HYDRAstor configuration. In essence, the HYDRAstor is an extreme form of thin provisioning as it thinly provisions a 256 PB file system so it uses only as much storage capacity as backup data is stored to it. I call it extreme because the capacity in the HYDRAstor is available to any file system that needs it at any time across the entire HYDRAstor grid. Similarly, if the HYDRAstor itself starts to run out of storage capacity, it alerts the administrator so he/she can add more Storage Nodes to the HYDRAstor configuration. When additional capacity is added to the HYDRAstor grid, it becomes available for use to any of the file systems that need it without the need to allocate capacity to specific file systems.

Inability to scale performance. Everything about backups is performance intensive, especially when using disk as a backup target. The HYDRAstor must reserve some compute overhead for the file system itself; it needs to ingest multiple backup jobs at the same time; it needs to deduplicate and compress the incoming backup jobs; and, finally, it may need to recover and replicate data while these other functions are executing. Since all of this is occurring in an enterprise environment, it is conceivable that a NAS system may have inadequate performance to go around. The HYDRAstor's grid storage architecture addresses these concerns as well. Just as it offers Storage Nodes to scale storage capacity as companies need it, Accelerator Nodes that can also scale performance so companies can expand performance when they need it. Using this 2-tier grid architecture comprised of Accelerator and Storage Nodes, companies can linearly scale the resources of the HYDRAstor to maintain performance as the system grows.

Enterprises are justified in their skepticism about using NAS as a disk-based target for backup. However the second release of HYDRAstor continues to alleviate some of these concerns. Not only is it a second generation product (which always helps to give enterprise shops more peace of mind) but it continues to make monitoring, managing and scaling capacity, performance or both on the HYDRAstor relatively pain free and simplistic events. And while features such as its 256 PB file systems have been part of HYDRAstor since its first release, when companies factor in its improved performance and higher capacity features coupled with what else the HYDRAstor already has to offer, HYDRAstor begins to present a compelling case for using NAS at the enterprise level for disk-based backup.

The Introduction of the 256 Petabyte File System Into Disk-Based Backup

2008-10-09T18:00:00Z

When I recently attended VMworld 2008, I had the opportunity to get a closer look at NEC's latest HYDRAstor release, the HS8-2000, and some of its features. Of course at a trade show all you generally have the time and opportunity to do is take a quick look at some of the product's hardware and software features. But in this case there was a feature on the HYDRAstor that struck me just from the short time I spent evaluating it: the ability to create a 256 petabyte (PB) or larger file system.

For those of you unfamiliar with the HYDRAstor, it is not a virtual tape library (VTL) but a networked attached storage (NAS) system that is primarily intended for use as a disk-based backup target. Configured as NAS, it presents a file system (CIFS or NFS) to backup servers that use it as a disk-based backup target. Now normally using NAS as a disk-based backup target may be a concern in enterprise shops for one major reason: file systems have limited capacity that creates an upper limit in terms of how large they can grow.

While VTLs also have upper limits in terms of storage capacity, VTLs appear as physical tape libraries to the backup software. Since backup software knows how to manage physical tape libraries by recognizing out of space conditions on a tape cartridge, it also knows how to recognize out-of-space conditions on virtual tape cartridges on a VTL. So when a virtual tape cartridge fills up, the backup software understands that it needs to start writing to the next virtual tape cartridge, which avoids the out of space condition that can cause backup jobs to hang or fail.

However if a file system fills up backup software may suspend operations or hang since it does not know how to manage out-of-space conditions on file systems. As a result, backup jobs may fail or pause until a backup administrator manually intervenes by either deleting some of the data from the existing file system or re-directing the backup job to a new file system target with more capacity.

HYDRAstor's support of a 256 PB file system helps to negate this specific concern about using NAS as a disk-based target. By providing a disk-based target that can contain 256 PB or more of data, companies can have confidence that directing backup jobs to a HYDRAstor file system will not result in backup jobs hanging because of an out-of-space condition.

Of course, just because administrators can create and present a 256 PB file system as a backup target to corporate backup servers does not mean that the HYDRAstor actually contains 256 PB (or more) of storage capacity. (If fact, I am quite sure the 256 PB file system I created on the HYDRAstor while at VMworld had nowhere near 256 PB of storage capacity behind it.) But because HYDRAstor supports the creation of multiple such file systems, it increases the overall virtual capacity supported on the HYDRAstor (at least from a file system perspective), from petabytes to Exabytes or even larger.

In this respect, the HYDRAstor is like some other storage systems in that it also uses its own form of thin provisioning to present a file system to the backup host so it appears to have more storage capacity than it actually has. The main differentiator is that HYDRAstor does not require any actual provisioning of resources. Using its grid architecture, all capacity across the entire HYDRAstor is available for all file systems and applications as a shared pool of storage. This improves the efficiency of the utilization of the available capacity by the file systems and applications that need it while permitting the addition of more capacity over time.

HYDRAstor's support of a 256 PB (or larger) file system addresses a concern of enterprise shops that often goes unaddressed by current disk-based backup systems that use NAS - unlimited storage capacity. By giving each and every file system on the HYDRAstor the ability to grow to 256 PB and beyond, companies no longer need to worry about managing and extending file systems manually. Instead they only need to set this up at the beginning when they configure the file systems, regularly monitor the HYDRAstor and how close it is getting to capacity (as opposed to individual file systems) and then add new Storage Nodes to the HYDRAstor when it starts to run short on capacity. HYDRAstor then takes it from there and automatically assigns the capacity on as-needed basis to specific file systems which can then be extended on-the-fly to whatever size file system is needed.

Remote Office Data Protection Rolled into new HYDRAstor HS8-2000

2008-09-22T10:00:00Z

The ease in which HYDRAstor's underlying grid storage architecture gives companies to migrate to higher capacity and faster performing hardware found in its new HS8-2000 make it easy to overlook some of its other new features. Part of the reason I devoted the last blog entry to HYDRAstor's self-evolving architecture is because I usually have to do just the opposite: educate readers about the advantages of upgrading to a new product so they can justify the pain of going through the migration. In HYDRAstor's case, it is so painless to upgrade and migrate to the new HS8-2000 release that it is almost easy to overlook its new features.

Part of the reason companies might miss these features is that they are more specifically targeted at enterprises either looking to bring a HYDRAstor in house for the first time or for those looking to extend data protection to remote offices or data centers. The two new features of the HYDRAstor that address these concerns include:

Rack-less models of the HS8-2000. Part of the impetus for introducing a rack-less model of the HYDRAstor was two-fold. First, it provides companies a low-cost option to introduce the HYDRAstor into their backup environment, especially if they already have their own standardized rack hardware. The other reason is it meets the less stringent requirements for the protection of data in remote and branch offices. While the grid storage architecture of the HYDRAstor is well-suited for corporations that need disk-based solutions that are scalable from both a capacity and performance perspective, the entry level solution of the previous generation HS8-1000 was a bit of overkill for some remote offices. The HS8-2002S rack-less model that is now part of the HS8-2000 series gives companies the option to start with a smaller HYDRAstor configuration (1 Accelerator Node, 2 Storage Nodes) for their remote offices with the option to upgrade to the larger HS8-2004S. Many-to-one replication. As companies deploy more HYDRAstor grids in other data centers or in remote offices, companies also want the option to replicate that data back to one remote site. While HYDRAstor already supported asynchronous replication with the HS8-1000, the HS8-1000 only supported one-to-one replication. The HS8-2000's introduction of new, entry level models for remote offices makes it a logical time to enhance the replication capabilities found on the HS8-2000 so one central HYDRAstor can receive data transmitted from one or more HYDRAstor grids located in remote offices. Since companies can control not only when data is replicated between HYDRAstor grids but also what data is replicated by selecting which filesystems they want replicated on each HYDRAstor, companies can minimize network bandwidth between sites as well as minimize the capacity requirements at the DR site.

To date much of NEC's focus with the HYDRAstor was on providing enterprise caliber disk-based data protection that could easily and effortlessly scale either capacity or performance. That mission was accomplished in the HS8-1000. Now by adding rack-less models along with many-to-one replication to the new HS8-2000, enterprise companies can more economically extend HYDRAstor's reach outside of the data center and into remote offices while bringing their data back to the home office. In so doing, companies can bring HYDRAstor's benefits to these locations without introducing some of the data management headaches that tend to accompany other disk-based backup solutions.

New HYDRAstor HS8-2000 Demonstrates the Power of a Self-Evolving Platform

2008-09-09T18:00:00Z

A self-evolving platform is one of the promises behind products like the NEC HYDRAstor that are based on grid storage architectures. Grid storage architectures automatically take over data migrations during technology refreshes which eliminates the need for application downtime or for companies to do forklift upgrades. Yet up to this point it was difficult to establish the validity of that promise for the NEC HYDRAstor since its HS8-1000 series was still in its first release.

Today's announcement of the HS8-2000 series begins to answer some of these questions as it starts to demonstrate the power of self-evolution that grid storage architectures deliver. Normally companies cannot immediately take advantage of new hardware features found in new product releases. The HS8-2000 is a good example of hardware technology enhancements that can be immediately leveraged to enhance existing solutions.

Accelerator Nodes of the previous generation HYDRAstor HS8-1000 supported throughput speeds of 100 MB/sec while its Storage Nodes supported 37.5 TB of effective capacity (based on 20:1 deduplication/compression ratios and 25% Distributed Resilient Data [DRD] protection overhead). Contrast this with Accelerator Nodes on the new HS8-2000 that now support 300 MB/sec of throughput (a 3x improvement) with Storage Nodes that offer up to 157.5 TB of effective capacity (a 4x increase) as well as increases the overall scalability of the HYDRAstor to over 20 PBs in capacity and 16,500 MB/sec in a 165 node grid.

Of course, these types of hardware improvements found in the HS8-2000 are expected with any new product release. Yet for most companies to take advantage of these new features typically requires months of planning, lengthy procurement cycles and then painful migrations from the old products to the new ones. Having personally been involved with upgrade cycles that took this long, it was not uncommon that the next generation of products were sometimes announced and released before my company got the current generation of the product installed. The HS8-2000's grid storage architecture helps to shrink these upgrade cycles since it minimizes the length of time to perform these tasks as well as the need to manually perform them.

Since the nodes of the older HS8-1000 and the new HS8-2000 are fully compatible within the grid storage architecture, companies can start to deploy the HS8-2000's new technology while continuing to use their HS8-1000 nodes. Used in this manner, the existing HYDRAstor instance automatically begins to use the new nodes of the HS8-2000 in conjunction with existing HS8-1000 nodes while dynamically balancing the workload and capacity across both existing and new nodes.

Another desirable feature of this configuration is when companies are ready to retire or decommission older HS8-1000 nodes so they can be re-used elsewhere in the organization, they can do so non-disruptively. Upon user request, the HYDRAstor software automatically migrates data from any older HS8-1000 Storage Node onto the remaining Storage Nodes in the grid and then decommissions the older Storage Node so it can be removed from the HYDRAstor instance. In the case of new HS8-2000 Accelerator Nodes, they also are simply added to the existing HYDRAstor instance and can immediately be used to extend the overall throughput of the grid.

Technology upgrades and their associated data migrations can easily consume one-third or more of the time that companies devote to managing their IT infrastructure. In addition, they are highly prone to user error (and potential data loss) and require extensive planning and execution time. HYDRAstor's self-evolving grid storage architecture gives companies the opportunity to bring this to an end.

While the new hardware features on the HS8-2000 are compelling, the real story is that companies can take advantage of this technology now without breaking the budgets, the backs of their IT staff, or current business processes. In so doing, the HYDRAstor enables companies to extend the life of existing infrastructure or seamlessly upgrade from existing to new nodes. HYDRAstor automatically leverages new nodes to take advantage of higher capacity and performing hardware that is both more efficient and economical than existing technology while paying less in terms of time and money in the process.

Throw away the Crystal Ball; HYDRAstor Scales Capacity and Performance Dynamically

2008-08-29T18:00:00Z

Replication and deduplication are features that are fast becoming necessities when disk libraries are introduced into enterprise IT backup environments. But as I brought out in a previous blog entry, introducing multiple functions into disk libraries intended for enterprise caliber backup environments typically has some unpleasant trade-offs. A primary concern in enterprise IT shops is how large (or small) to initially configure the solution so companies neither overspend on oversized hardware nor purchase undersized hardware that cannot scale to meet their future requirements, so they need some way to forecast how their IT environment is going to look going forward.

Unfortunately most IT staff is not real good at reading crystal balls so the flexibility to independently scale individual components in enterprise disk-based backup solution is imperative. This is especially true since specific features within the disk library - deduplication, replication and storage capacity - each place different performance demands upon it, depending on how the environment grows.

It is because of these new demands that companies should consider disk libraries that are based on grid storage architectures like the NEC HYDRAstor that can independently scale capacity or performance to meet uncertain future demands. Specifically, here is how these requirements may stress a disk library and how the NEC HYDRAstor equips companies to dynamically respond to these needs.

Deduplication coupled with high backup throughput rates place a premium on CPU and memory. Deduplication algorithms need to analyze incoming blocks of data, index them, ascertain their uniqueness when compared to other blocks of data, and then store new unique blocks of data. During peak backup times in enterprise IT shops, these systems could experience throughput levels in the hundreds of MBs per second. However companies may not have a good sense of how fast backups will actually execute once they move from tape to disk. Also, the number of backup jobs may increase as backup jobs from business unit and departmental servers are directed to the new enterprise IT disk library. Using the NEC HYDRAstor, companies can scale performance (CPU and memory) by adding more Accelerator Nodes to their existing HYDRAstor at any time. Companies can add these new nodes to account for unexpected growth in backup jobs and deduplication overhead. New and old nodes then act as one integrated system.

Not all backup data deduplicate equally, so the need for more storage capacity may outweigh the need for more performance. How well corporate data deduplicates is always a best guess prior to implementing a deduplicating disk library. It could be as high as 500:1 (don't count on it), as low as 2:1 or anywhere in between depending on application/data type, retention period and specific backup policites. While industry pundits generally put the deduplication ratios around 20:1, the only ratio that matters is the one your company actually achieves. It is when you are the anomaly that it matters. So if your data stores climb abnormally high because of low deduplication ratios or unexpected data growth, can the storage capacity of your disk library scale at all, scale independently of the performance component of it or scale linearly so performance is not impacted? Using the NEC HYDRAstor, companies have the option to add essentially unlimited amounts of storage capacity (known as Storage Nodes) independently of Accelerator Nodes.

Replication may have no impact or it could exacerbate an already problematic situation. How replication impacts the deduplicating disk library really depends on how and when a company uses replication. If data is replicated when no backups are occurring (during the day), the replication feature may have no impact at all on the disk library. However if companies try to replicate data at night or at the same time they are doing backups and deduplicating data, how much extra capacity do they need to add to the disk library now or in the future to support this feature? Using the NEC HYDRAstor, companies can address this question when and if it arises by dynamically adding Accelerator Nodes (ANs) that can even be specifically designated to just replicate data offsite. On individual ANs, administrators can also set up replication at the file system level so only critical file systems that require it are replicated.

Appropriately sizing disk libraries is a primary concern for enterprise companies, yet IT departments are often forced to rely upon crystal balls to forecast how their environments will grow. Selecting the NEC HYDRAstor as their disk-based target addresses this concern, because they can add what they need (performance or capacity) when they need it. In so doing, the NEC HYDRAstor removes the upfront gyrations and crystal ball gazing normally associated with trying to size disk libraries, while giving companies the freedom and flexibility to scale their disk library in whatever direction their environment dictates.

Prerequisites for Introducing All-in-One Computing into Enterprise IT

2008-08-22T10:00:00Z

The "all-in-one" concept is one of the hottest trends in consumer technologies. Just looking at the gadgets and devices that I use on a day-to-day basis in my office, I am hard pressed to find one that does not perform multiple tasks. My office phone supports two lines, has a separate voice message box for each line and tracks all of my incoming and outgoing calls. My printer is not just a printer. It prints, copies, scans and faxes. Then, of course, there is my Blackberry which acts as a cell phone, email client, web browser, calculator, personal organizer (contacts/phone book) and a host of other functions that I have not even had time to figure out yet.

So why is it that this all-in-one concept is slow to catch on in enterprise IT shops? There is any number of reasons. Part of the problem is that these shops are often content to dedicate specific devices to performing specific functions rather than push the boundaries a bit.

However a bigger part of the dilemma is that enterprise IT shops inherently have much more at stake if a device that performs multiple functions fails. When my Blackberry fails, it just inconveniences me. When a device that performs multiple tasks fails in a business, it may disrupt multiple business processes. This can result in missed service level agreements and even incur financial penalties or lost revenue.

Anyone who has ever worked in production IT environments knows that is a sure fire way to get oneself fired. While that alone is not sufficient reason for enterprise IT organizations not to explore enterprise technologies that promise "all-in-one" functionality, it certainly gives IT staff in these organizations pause in implementing them.

Therefore to move all-in-one technologies beyond just consumer-oriented Blackberries into enterprise IT environments requires that these technologies deliver on some key concepts that keep IT managers up at night. These include:

The ability to sustain multiple, concurrent points of failure. Most computer hardware that is designed for a single purpose, whether it is a storage array, server, or network switch, has redundant everything - power supplies, disk drives, controllers, etc. Yet if two of these items fail at the same time in the unit, the whole unit fails and those who say it can't happen are lying through their teeth as I have seen it happen. Granted, it was always extreme circumstances but the impact was devastating. Now imagine if multiple processes hinge upon the availability of one specific piece of hardware. Companies will want every assurance that device can remain available all of the time.

Can measure and forecast how much of the hardware resources each function on the device will use. My Blackberry has all of the horsepower, memory and storage I need and, even when it doesn't, I run down to my local Verizon store, buy the item that I need, power my Blackberry off, insert the additional memory or storage and Voila!, I'm ready to go. Enterprise IT shops have no such luxury. They need to forecast, monitor and pro-actively respond with the right amount of memory, processing power or capacity before there is a problem. Then they need to justify why they need what they need, procure the needed equipment, plan the upgrade and non-disruptively upgrade the device so no application outages are incurred. Oh yeah, and then pray to God that they procured the right amount of memory, CPU or storage and that it solves the problem they said it would without causing any new problems.

Can scale to permit the introduction of the appropriate resources. Knowing what memory, CPU or storage you need to add to correct the problem is only part of the equation. That part of the equation only works if the device can scale to support it. That becomes tricky. If companies overbuy and never use the scalable feature, they find themselves paying for features they never use. Conversely, if they size their hardware too small, they find themselves having to take huge write-offs before the equipment is fully depreciated and needing to buy new hardware that is appropriately sized. Even in circumstances where they can buy the appropriate level of hardware, companies still need the flexibility within the device to dedicate the new hardware resource to the software function that needs it. Otherwise, the performance or capacity enhancement will be negligible.

Meeting any one of these three requirements is a tall order. Meeting all three sounds impossible. However technologies with these types of capabilities are now starting to show up in the enterprise data protection space and specifically in the form of grid storage. In forthcoming blog entries, I'll take a closer look at how the NEC HYDRAstor and its grid storage architecture is giving enterprise IT a realistic chance of bringing devices that provide the all-in-one functionality they want without the typical trade-offs that most other systems at the enterprise level expect.

Disk-based Backup Brings an End to Elevator Joy Rides

2008-07-29T10:00:00Z

Anyone who thinks tape is still the right primary target for backup only needs to watch a video on NEC's website that includes a testimonial from Orlando, FL, based TLC Engineering. In this testimonial, TLC shares some of its experiences using tape as its primary target for backup and recovery and the hassles associated with it. The situations that the individuals on the video describe are almost comical but, from past experience, I know that TLC's experiences are more common than not. For instance, the recovery procedures that TLC's network administrators had to go through in order to restore files from tape could include one or more of the following steps:

If at home, the network administrator had to drive in to work
Once there, they had to to to their workstation and log into the backup software to locate the file to recover and which tape the file was on
Retrieve the tape
Take the elevator to the floor where the tape drive was located (their data center spanned multiple floors in the same building) and insert the tape into the tape drive
Take the elevator back down to their office
Recover the file
Notify the end-user the file was restored and wait until the end-user verified that he or she could read the file
Get on the elevator again, retrieve the tape and then return it to its original location

Obviously every restore wasn't this complicated or required all of these steps but apparently all restores at TLC routinely called for elevator rides and the network administrator to be at work to perform the restore. This process took valuable time away from the network administrator who had better things to do than taking elevator joy rides to recover files while end-user work had to stop until the needed files were recovered. Bottom line, this approach was time consuming and a hassle for everyone involved.

So it is no surprise that TLC opted to go with disk backup in general and the NEC HYDRAstor specifically as its new backup target and source for data recoveries. In previous blog entries, I have discussed some of the inherent advantages that the NEC HYDRAstor provides to include scalability, ease of management, performance and high deduplication ratios that contribute to companies like TLC selecting the HYDRAstor. That is all well and good but there are also some intangible benefits of using the HYDRAstor that go well beyond all of the financial and technical reasons companies use to justify its deployment.

One of HYDRAstor's larger benefits is its file server based, grid storage architecture that contributes to eliminating many of the hassles associated with managing the backup infrastructure. Companies can simply attach the HYDRAstor to their existing LAN and then direct backup jobs to the disk target it presents. Since HYDRAstor is based on a grid storage architecture, it takes the guesswork out of how much initial performance or capacity companies need to buy for their short and long term needs. Instead companies like TLC can buy what they think they need and, if they need more storage capacity or performance, they can incrementally add what they need without the need to replace the exiting solution.

The other end result is a more productive, efficient workforce. By putting in a solution that scales to unlimited (theoretically) storage capacity or performance, it expedites backups and restores and eliminates the need to manage tapes so backups and restores occur successfully and more quickly. In the case of TLC, network administrators could now log in from home and restore files without needing to come in to work and take elevator joy rides to load and unload tapes. Meanwhile end-users who needed the files no longer have to wait indeterminable amounts of hours for the restores to complete.

As TLC discovered, creating an environment where hassle-free backups and restores can occur is possible. Yes, companies still need to give some forethought to what is the right solution for their backup environment and how best to configure it. But technologies like the NEC HYDRAstor are fundamentally changing how companies backup and restore their data and, in the process, making their IT staff more effective and efficient in their day-to-day jobs.

No RAID, No Problem: How Data is Protected on the NEC HYDRAstor

2008-07-16T10:00:00Z

In recent weeks and months, I have more than once raised the question about the safety and integrity of corporate data that is stored on systems that use RAID, especially SATA-based storage systems. The reason this problem is becoming more acute with each passing day is three-fold:

Deduplicated data. When you deduplicate data, only one chunk of data is stored that 10s, 100s or even potentially 1000s of files may have in common. The problem emerges during the restore since all of these files need access to the same chunk of data in order to restore the file. If that chunk of data in any way becomes corrupt or unreadable, the impact of the corruption becomes far reaching and may lead to the inability to recover many files.
Bit Error Rates. This is a problem that shows up on RAID systems when a bit of data on a disk drive becomes unreadable. It occurs so infrequently that it not really considered a problem in small environments, but as corporate data stores grow into the 10s or 100s of TBs, the likelihood of this error appearing turns from a remote possibility into almost a certainty. Reaching these multi-TB thresholds is quite possible for companies who need to keep their archived and backup data stores for years or that archive or back up data for hundreds or even thousands of applications or servers. Typical data growth rates of 50%+ per year further contribute to the likelihood of companies reaching these higher storage capacities.
Disk Drive Rebuild Times. SATA disk drives are already 1 TB in size and 1.5 TB disk drives were announced just last week. The problem that emerges is rebuild times should a disk drive fail. Using traditional RAID, storage systems need to reconstruct every block of the failed disk drive to a spare disk drive during the rebuild. Recovering a failed 1 TB drive will minimally take hours if not days (up to a week if some stories are to be believed). It's conceivable that another drive could fail in that RAID set causing the whole RAID array to fail. While RAID 6 partially alleviates the problem with protection against two concurrent drive failures in the same RAID set, it is likely only a band-aid, not a long term fix.

However my intent is not to leave readers hanging or fretting as to what storage systems they can select that take this problem into account. The NEC HYDRAstor is one such product that has taken steps to address this issue. HYDRAstor includes a feature called Distributed Resilient Data™ (DRD) that is able to offer more protection than RAID 5 or RAID 6 without their rebuild performance drawbacks. Because HYDRAstor is based on a grid storage architecture, it can by default survive the failure of not only multiple disk drives but also multiple Storage Nodes. The default setting is 3 disk drives or 3 Storage Nodes if multiple nodes are present (based on the video on the HYDRAstor web site, it looks like a company needs at least 12 nodes for a company to have assurance it can recover from the failure of 3 different nodes).

Here is how DRD works:

As data is ingested by the HYDRAstor, it breaks up an incoming chunk of data into 9 fragments plus also creates 3 additional fragments that act as parity fragments for a total of 12 fragments
At this point another HYDRAstor feature called DynamicStor™ kicks into gear. DynamicStor distributes these fragments across the disk drives and/or nodes
In the event of a disk and/or node failure occurs, DynamicStor can reconstruct the data from the fragments on the remaining disk drives and/or nodes

Companies with growing data stores (and what company does not have growing data stores?) need to come to terms with the fact that traditional RAID technologies are no longer sufficient to protect their data long term. Features like DynamicStor and DRD on the NEC HYDRAstor take into account the growing capacity of SATA disk drives and the new risks that higher capacity storage systems create while providing companies new disaster recovery options. In so doing, HYDRAstor provides companies a means to safely and confidently store and protect their backup data on SATA disk drives.

Have an Ulterior Motive When Implementing Disk-Based Backup

2008-07-08T10:00:00Z

In a recent blog entry, I took a look at the complimentary roles that replication and deduplication have in delivering global deduplication as part of an overall enterprise wide data protection strategy. However that is just part of the total global deduplication and enterprise data protection picture. No matter what solution a company implements or how well it replicates and deduplicates data, data stores still grow and the hardware on which the solution runs still ages.

On the surface, these don't sound like a big deal. But companies need to keep in mind that introducing disk along with deduplication and replication into the backup process does not immediately solve all problems associated with backup. The danger with using disk-based backup is that companies can still encounter performance and capacity limitations. Therefore an ulterior motive companies should have as a part of implementing disk-based backup and data protection is to eliminate all hassles associated with managing backup both short and long term - not just the immediate pain of backup and recovery.

Almost any disk-based solution - deduplicating or otherwise - is going to expedite backups and recoveries. Sure, some solutions may deduplicate better or do it faster but at the end of the day most companies are at the point that putting in place any disk-based system that supports replication and deduplication is better than dealing with the current backup pain.

However what companies often fail to account for is how fast their backup data stores grow when they start backing up data to disk. More than once I've talked to system administrators in companies where "undisclosed" or "hidden" departmental application servers start to come out of the woodwork once department managers hear that corporate IT backup processes actually work.

The dirty little secret in enterprise companies is that it is almost common place for departments within companies to hide their application servers from corporate IT because they know how badly broken the centralized corporate IT backup processes are. However once word leaks out that corporate IT has backup fixed, suddenly everyone wants to unload their departmental backup jobs on corporate IT. The end result is that backup data rates tend to grow much faster than what corporate IT initially forecast.

Frankly, this is the new challenge that corporate IT faces today: Put in place an infrastructure that can cost-effectively and easily scale on a whim (i.e. - without any planning). This is especially applicable when dealing with backup data because there are so many unknowns. Corporate IT generally already knows about existing applications and their forecasted data growth. But when it comes to determining how much data they will receive and/or need to transmit to secondary data centers and remote and branch offices (ROBOs) is at best an educated guess. Yet if companies underestimate, they can reach performance and capacity thresholds much more quickly and need to pay a lot to upgrade. Conversely if they grossly overestimate, they can pay way too much up-front.

In order to avoid these mismatches between secondary data flow and associated infrastructure capability, companies need to take a hard look at disk-based deduplication products that are based on grid storage architecture such as the NEC HYDRAstor. The NEC HYDRAstor provides both Accelerator Nodes for performance and Storage Nodes for capacity so companies can independently and economically scale performance, capacity or both, to keep up with their specific backup needs. The big benefit of grid storage architectures is that companies can start small and then scale accordingly.

Disk-based backup is bringing a lot of relief to a lot of organization but companies need to look beyond just the promise of the immediate relief that disk provides. While solving today's corporate backup problems are important, corporate IT also needs to keep their eye on the ball to ensure they are implementing IT infrastructures that cost-effectively scale while keeping management costs down to a minimum. Products such as NEC HYDRAstor that use grid-based storage architectures help companies address their immediate backup pain while also putting them on solid footing going forward.

The Juxtaposition of Deduplication and Replication Needed for Global Data Protection

2008-06-27T12:30:00Z

The juxtaposition of deduplication and replication in disk-based backup appliances is a powerful combination that companies can use to protect backed up data across data centers as well as data backed up at remote and branch offices (ROBOs). Yet where deduplication ends and replication starts can get a little confusing in grid storage architectures such as is supported by the NEC HYDRAstor that features global deduplication capabilities.

To understand how this works, let's first take a look at this from the perspective of a company that has data centers and ROBOs in different geographic locations. In this scenario, the company may want the flexibility to leverage its primary data center to recover the data backed up at any of its secondary data centers or ROBOs.

To deliver on this ideal, the company would first need to install a HYDRAstor grid at each of its data centers or ROBOs that acts independently of the HYDRAstors at the other locations. Each local HYDRAstor would then deduplicate all data at that site to address that site's need to shorten backup windows and provide fast recoveries of data at that site.

HYDRAstor's optional RepliGrid feature then enables the company to protect data from any of its secondary data centers and ROBOs at the primary site. Using this feature, each HYDRAstor at a remote site would replicate its deduplicated data asynchronously back to the HYDRAstor at the main data center on a regularly scheduled basis. The data replication process occurs as follows:

The HYDRAstor at the remote location provides a list of hash keys for its changed or new data to the HYDRAstor at the primary site.
The HYDRAstor at the primary site receives the list of hash keys and removes keys it already has from the list
The reduced list of hash keys detailing what deduplicated data is needed at the primary site is transmitted back to the HYDRAstor at the remote site
The HYDRAstor at the remote site sends the deduplicated data associated with the hash keys in the reduced list back to the HYDRAstor at the main site

The distinct business benefits that the HYDRAstor RepliGrid feature offers are two-fold:

Reduces Storage Costs. The amount of deduplicated data stored on the HYDRAstor at the main data center is minimized. By performing global deduplication across all local data and remote data, only net new unique deduplicated data from each remote site is stored on the main HYDRAstor.
Reduces Network Bandwidth Costs. Since the main HYDRAstor aggregates data across all of the sites, there is a good possibility that the deduplicated data already exists at the main site. By only transmitting net new unique deduplicated data, it minimizes the amount of data transmitted and hence the size of the network pipes needed to transmit the data.

However, offering deduplication and replication is only part of the enterprise data protection picture. As companies look to deduplicate data across their entire enterprise and replicate between multiple sites, spikes in performance and capacity may necessitate the companies to have the flexibility to cost-effectively and easily scale these components of the HYDRAstor architecture. I'll examine how the HYDRAstor accomplishes those tasks in a forthcoming blog entry.

Disk-Based Backup Brings Replication Into End User Data Protection Conversations

2008-06-13T18:40:00Z

At recent storage conferences (Storage Decisions, Storage Networking World, etc.) replication has emerged as a hot topic of discussion among end-users. In talking with these different users and listening in on a number of end-user panel discussions, there are a number of factors that they attribute to their increased interest in using replication as part of their company's overall disk-based data protection strategy:

The need to move data offsite after their data is backed up to disk. Should some type of disaster strike their production site (floods, hurricanes, tornados, power outages, etc.) they need to have some means to recover their data offsite. This generally means using either removable media (tape, optical, etc.) or replication to move it to another location.
Companies want to use removable media as a last resort to recover data as they view it as problematic to manage and recover from. If companies do store data on removable media, they prefer to view it as data's final resting place and not their primary or even secondary means of performing a data recovery.
Broadband network connections continue to remain affordable. The capacity and cost of a broadband connection is determined by the amount of data that a location needs to transmit. Remote and branch offices can obtain DSL (digital subscriber lines) that provide connection speeds at up to 512 Kbps for under $100/month while T1 lines (1.5 Mbps) run around $1000/month and T3 lines (45 Mbps) are about $10,000.
Using deduplication makes replication more affordable. Rather than replicating all of the data created by each backup, using deduplication companies only need to transmit net new blocks of data. This reduces the total amount of data that companies need to send which translates into the need for smaller, lower cost network pipes for replication.

Of course, implementing replication in large enterprises as part of their overall disk-based data protection plan becomes more complicated. While large enterprises are always looking for solutions that are easy to deploy and implement into their environments, these are only some of their considerations when selecting a solution. Large enterprises may have offices and data centers that span the globe, multiple backup software products, different data retention policies and numerous network links of varying capacities. As a result, they will want a solution that they can install that matches the size and cost constraints of these different enterprise environments.

Enterprises also need to identify products that provide architectures that meet current and future needs. Features such as global deduplication and more granular control over replication, such as what data is replicated and when it is replicated, are almost prerequisites in enterprise shops. These companies also need to think outside the box to take into account new architectures such as grid storage that provide new enterprise data recovery options. For instance, using grid storage, companies can start to think about creating storage grids that replicate data across geographic distances such that companies can recover data from any of the sites.

End-users now readily recognize that replication is becoming as integral to their corporate data protection plan as data protection software and hard disk storage systems. However enterprise corporations need to think more broadly about how they implement replication as part of their disk-based data protection strategy.

Deduplication and replication alone are not enough as large enterprises require disk-based backup solutions that scale in a manner that is very different than what is generally available on the market. These new solutions require new architectures such as grid storage that scale in such a way that they remain cost-effective short and long term. The grid storage architecture the makes up NEC HYDRAstor provides the type of scalability and cost effectiveness to complement the deduplication and replication features that large enterprise companies are seeking. In an upcoming blog entry, I'll take a closer look at how the HYDRAstor delivers on these features.

Anderson Center for Autism Drives Storage Costs Down to 70¢/GB Using the NEC HYDRAstor

2008-05-08T17:44:52Z

One of DCIG's objectives in blogging is to document over time how companies are using different vendors' products, the ways in which they are using the product, successes they are having and specific challenges that they are beginning to face. Greg Paulk, the IT Director for the Anderson Center for Autism, represents the first individual that DCIG has had the opportunity to do this with.

I first met Mr. Paulk at the Fall 2007 Storage Decisions conference in New York City and interviewed him shortly thereafter for a blog entry that appeared back in October 2007. However six months have passed since that interview so I followed up with Mr. Paulk to get an update on how his installation of the NEC HYDRAstor was performing, since Paulk was still using a beta version of the NEC HYDRAstor software when we last spoke.

Paulk revealed that he is now in full production with the production code loaded on the NEC HYDRAstor. However he is still using the same hardware configuration (two Accelerator Nodes and four Storage Nodes) that he started out using due to the high deduplication ratio that he is achieving with the HYDRAstor.

Last fall he was achieving a 17:1 deduplication ratio and hoped to eventually achieve a 35:1 ratio. Six months later, his deduplication ratio is now approximately 39:1 which has mitigated his need to buy additional capacity and has driven his cost/GB down to approximately 70¢/GB. "It's like getting 390 TB for the price of 10 TBs," says Paulk.

He also has not found it necessary to add more Accelerator Nodes into his HYDRAstor configuration. Though he has nearly doubled the number of servers he is backing up on a nightly basis (from 13 to 21 servers), he is achieving about 3 Gbps of throughput across his two Accelerator Nodes.

I then asked him, "What are the biggest benefits that you have experienced since you started using the HYDRAstor?" There were four benefits he cited:

First, it worked as advertised. The installation was easy (it took 68 minutes), and it has done everything he has needed it to do.
Second, it requires very little management overhead. He has one individual assigned to manage the HYDRAstor and, since it functions as one logical configuration, it takes very little time to manage.
Third, no backups have failed since he introduced the HYDRAstor, and it works 90% faster than when he was using tape.
Fourth, he has found it has reduced his stress level. Aside from alleviating his backup concerns, the HYDRAstor provides him a solid foundation that he can use to build for the future. He no longer has the traditional worries of how he will manage, upgrade or migrate data to new storage systems, because the HYDRAstor accounts for all of these concerns with its grid storage architecture.

In the next few months, Paulk plans to archive about one million documents to the HYDRAstor, which will consume about another 1.4 TBs of storage. What he is curious to discover is how they will impact his deduplication ratio. These one million documents are currently paper documents that need to be scanned so he wonders how they will impact the level of deduplication that he is seeing with his backup data once they are stored on the HYDRAstor. My plan is to catch up and speak with Mr. Paulk again this fall so DCIG can share some more of his story and experiences at that time.

NEC HYDRAstor Restores Focus on Optimized Data Management

2008-04-30T11:00:00Z

NEC's Vice President of Advanced Storage Products, Karen Dutch, recently brought out some salient points about storage management in her Spring 2008 SNW presentation, "Defining Storage Solutions in the Data Center 2.0". Specifically, she described the features that new storage architectures should deliver in order to keep storage management manageable as storage growth in organizations continues. Of course, the not-so-subtle message is that NEC's HYDRAstor delivers on these new features. Here's how I see the HYDRAstor doing so.

Self managing. NEC's HYDRAstor architecture supports self-management through the dynamic addition of nodes (servers) that offer either more capacity or performance. As more Accelerator or Storage Nodes are added to the HYDRAstor grid architecture, it non-disruptively redistributes data across old and new nodes to optimize performance and maximize data resiliency. This eliminates the normal processes of provisioning, sizing and data migrations that administrators have to perform, while alleviating the management overhead and costs associated with archive and backup processes.
Data mobility. Data mobility comes more prominently into play when new Storage Nodes are added into the HYDRAstor grid storage architecture as well as during technology refreshes of Storage Nodes. As new Storage Nodes are added, the HYDRAstor re-balances data across existing and new Storage Nodes to simplify data management. As existing nodes are retired, data from an existing node is automatically migrated to a new node and, if companies have multiple sites, HYDRAstor supports the movement of data to alternative sites to create an enhanced level of data resiliency.
Non-disruptive evolution. The HYDRAstor grid storage architecture addresses one of the most problematic aspects of storage management today: technology refreshes. As current storage systems age, usually the only option companies have is to purchase an entirely new storage system and then use either host or network based data migration tools to move to new storage controller architectures. Since HYDRAstor is based on a grid architecture, it can transparently evolve to newer technology simply through the introduction of new Accelerator or Storage Nodes based on the latest and greatest hardware technology. The HYDRAstor adds these new nodes into the grid while older nodes are marked for decommissioning and non-disruptively taken out of service.
Scalability without trade-offs. A key problem with current storage system architectures is that you generally have to pick between performance, capacity and cost when scaling the architecture. Since HYDRAstor's Accelerator and Storage Nodes are based on industry-standard, off-the-shelf hardware, the typical hardware costs associated with proprietary storage hardware architectures are avoided. Since HYDRAstor also decouples performance (Accelerator Nodes) and capacity (Storage Nodes), users can scale one or both to accommodate whichever direction their storage environment grows.
Enhanced, flexible resiliency. The HYDRAstor accounts for the growing possibility that today's RAID data protection architectures are insufficient when deduplication is used across 100s or 1000s of TBs of capacity. Administrators can define the level of data redundancy that is appropriate to their site and the HYDRAstor will dynamically distribute the data across the nodes to deliver the desired level of resiliency.
Integrated data management services. The HYDRAstor comes with two sets of integrated data management services. The base level of services includes the automated management of the data, more efficient storage (deduplication) and enhanced data resiliency. Advanced services like replication, WORM, security, classification and search are other features that users can optionally license from NEC.
Industry standard interface support. The HYDRAstor presents an industry-standard NFS and/or CIFS interface, so any Linux, Windows or UNIX server can archive data to it or, alternatively, any backup software can treat it as a disk cache and store data on it.

The most compelling benefit of the NEC HYDRAstor grid storage architecture is that companies who adopt this architecture can start to take their focus off managing the storage infrastructure and focus more squarely on the data they are entrusted with protecting and managing. Today's businesses live and die by how well they manage their data and the management of data is still tied too closely to how well the hardware is managed. By meeting these new storage solution features of the Data Center 2.0, NEC HYDRAstor restores the focus of administrators back to optimal data management and protection.