• Creating volumes
• Assigning volumes to specific hosts
• Setting LUN security
• Verifying multi-pathing is properly configured and check its status
• Setting and monitoring storage array alerts

• Array-based snapshots. Server virtualization creates an opportune time for organizations to update their backup strategies and array-based snapshots are becoming the first line of defense for data protection. Using NEC Storage Manager, enterprises can centrally configure snapshots on any NEC storage array for any application server (virtualized or physical) that has data residing on them.

To ensure the snapshots are recoverable, NEC provides hooks into file systems and Microsoft SQL Server databases. Application consistent snapshots may then be scheduled using NEC Storage Manager to shorten backup windows and remove the overhead of backup from VMware ESX hosts.

• Performance monitoring.  Monitoring storage array performance is becoming an imperative to effectively pinpointing and troubleshooting virtualized applications that access multiple storage arrays. Using NEC's Performance Monitor feature, which is part of NEC Storage Manager, enterprises can proactively take steps to identify the source of the problem, quickly isolate disk drives that are "hot" or network ports that are congested and take action to resolve the issue.

• Array-based replication. Whether the disaster is small or large, enterprises increasingly want the flexibility to recover applications more quickly at another site, especially as they virtualize their application and centralize their data on these arrays. Meeting this objective requires that data is regularly replicated offsite and the replication process managed appropriately.

Using NEC Storage Manager, enterprises can take advantage of the replication software available on NEC's storage arrays, centrally configure NEC's replication software and then monitor and manage this ongoing replication between arrays.

• 8 Gb FC or 1/10 Gb Ethernet storage networking interfaces. On an NEC M100 organizations may choose from the two fastest available storage networking interfaces: 8 Gb FC and 1/10 Gb Ethernet. This gives them the flexibility to use either the FC or iSCSI protocol in their environment.
• 6 Gb SAS HDD interconnect. A high speed connection between storage controllers and backend HDDs is a prerequisite to achieving optimal performance. The M100's 6 Gb SAS interconnect assures organizations that the M100 will deliver a high level of performance.
• Solid state drives (SSDs). SSDs are a game changer as they dramatically accelerate application performance with read times that are as much as 20X or faster than HDDs. The NEC M100 is on top of this trend with its support for both 100 and 400 GB SSD drives.
• High capacity drives. Some shops just require lots of storage capacity. To flexibly meet the needs of these organizations, the M100 supports up to 192 TBs using 2 TB 7200 RPM SAS drives.
• Mix of disk drive types. Most organizations have a mix of applications with some that need high performance while others need high capacity. By supporting a mix of SSDs and HDDs, the M100 provides the drives that organizations need to meet their varying application requirements.

• Active-Active controllers.  This high end storage array feature on the M100 is available on less than half of the midrange arrays on the market. It ensures that should an M100 storage controller fail, attached hosts experience little or no interruption in service. Further, this controller configuration ensures that when a periodic 30 minute microcode update needs to occur, it can be done dynamically without application impact.
• Delivers 99.999% uptime. Availability is a necessity in virtualized environments as any midrange array outage has the potential to impact the entire environment. To meet this requirement, the M100 is fully hardware redundant so should any component fail another is immediately available to replace it. The NEC M100 also provides a number of alerting options to warn organizations of potential problems so they can be quickly resolved and problems or downtime averted.
• No impact recoveries from HDD failures. As more VMs access the M100, even doing a rebuild of an HDD failure may cause an undesirable performance hit to applications. The NEC M100's new SuperPhoenix technology addresses this concern. Prior to an HDD rebuild, it determines the extent of the HDD failure or if an HDD rebuild is even necessary. If it is, the HDD is rebuilt in such a way that application performance impact is controlled and minimized.
• Solves silent data corruption in HDDs. All HDDs experience some data corruption over time that other midrange array systems may not detect. Using its SuperPhoenix technology, the M100 constantly monitors HDDs. If an HDD error is detected, the HDD is disabled while the M100 determines whether the HDD should be permanently replaced by running a battery of tests against it. By making this determination, the M100 ensures the integrity of the data without unnecessarily requiring a HDD replacement if it is a fixable error.

1. The use of 2.5" HDDs which require less power and space to operate than its 3.5" predecessor.  
2. Controller processors that run at a thermal design power of 30W.  This design is typical of processors from the 1990's which only consume as much power as a 100W bulb.
3. Autonomous switching to a low power mode. This demands less resources to the point that it even powers off RAID groups that are unused.

• Are not optimized for sharing.
• Are difficult to integrate as each has its own management console and API
• Are built on the assumption that they will be used by one client
• Create silos of compute power and storage capacity

• Blackball Search-in Software
• Nexenta storage software
• QLogic FC switches
• VMware vSphere

• Self-Managing. Any Genesis Hosting client can access its VDC at any time and independently provision physical resources or run applications without needing the help of Genesis
• Scalable. As its clients' VDCs consume physical resources and need more, Genesis can non-disruptively provide it to them.
• Shared resources. Genesis shares its physical resources between its clients' VDCs by using the APIs to monitor resource utilization and dynamically allocating them where they are most needed
• Secure. Genesis uses NEC's storage software functions to ensure VDCs do not trespass on what belongs to another VDC

1.  Up to 67% performance boost. This performance boost is based solely on doing a software upgrade for the HYDRAstor with no requirements for additional hardware or new license fees. Prior to this release each HYDRAstor Accelerator Node provided up to 300 MB/sec. By simply upgrading the HYDRAstor software, performance will jump up to 500 MB/sec per Accelerator Node or 1000 MB/sec for a standard two Accelerator Node configuration. This translates into faster backups as each node is able to process the data inline quicker.

What is also of note is that these new performance numbers are published for general applications and do not assume a scenario that is only applicable for a very specific application with a rare configuration as other storage vendors tend to do. This means more users can achieve the published performance numbers, and users with high duplicate data rates in their backup streams could see even higher performance numbers that what NEC is publishing.

2.  New encryption and chargeback capabilities. One of the tenets of NEC's design of the HYDRAstor is that core software functional improvements that most customers need should be made available to both existing and new HYDRAstor customers at no additional charge. Today's release continues that trend as it has added in-flight encryption of replicated data as well as a new storage chargeback capability.

• Simplified in-flight encryption of data. More enterprise organizations are looking to protect their remote offices as well as replicate data in their main data center to a secondary site. But as they do so they want to ensure that the data is encrypted while in-flight so it cannot be used for nefarious purposes in the event it is intercepted or compromised.
  

To accommodate this request that NEC has received from many of its customers, NEC has added an encryption feature to the HYDRAstor software base, which first encrypts data before it goes over the wire and then decrypts it on the other side so it can be read. This inline encryption can be enabled or disabled at a file system granularity and is available as a free enhancement for all Accelerator Nodes that have licensed HYDRAstor's RepliGrid WAN-optimized replication.

• Storage charge back enforcement with file system-based quotas. Storage charge back was another feature requested by NEC's customers that were seeking to designate different quotas for different file systems or applications so they could manage the entire HYDRAstor storage pool. This new feature should help organizations better respond to today's tough economic conditions by first helping them understand the storage consumption of individual applications, departments or business units, and then charging them accordingly.

3.  Enhancement of existing HYDRAstor functionality:

• Increased flexibility in resiliency depth settings. The HYDRAstor has always offered the flexibility to increase or decrease the resiliency of its system. Now organizations can dial the resiliency level up or down at the file system level. Using this enhanced feature, users can assign a particular application to a specific file system and based upon the importance of the application and its data, adjust its protection up or down.

This feature becomes very handy as many scratch, reference, or test data can now be assigned to HYDRAstor file systems that have a lower level of resiliency and lower overhead while the data associated with critical application data can be dialed up to meet their higher protection requirements.

• Capacity on Demand. More organizations are looking to deploy disk-based backup into remote offices but need to justify the cost. To accommodate these organizations, NEC is providing new smaller models that can start at lower levels of capacity and price but still get all of NEC's software features. NEC is also making it possible to expand the logical storage capacity of the HYDRAstor system at a later time via a software license, without requiring the addition of any hardware components.
• Extending application-aware deduplication. NEC announced application-aware deduplication in June 2009. This feature interrogates data as it comes in from a particular application (such as backup or archive software) and determines whether or not the data is a duplicate after it filters away metadata that the application has injected into the data payload.

In its first release of application-aware deduplication, HYDRAstor supported CommVault® Simpana® and Symantec NetBackup, but this release adds application-aware support for Tivoli Storage Manager (TSM) and EMC NetWorker.

• Bring in a disk-based storage system that supports deduplication. Disk-based backup and recovery improves success rates to 98% or higher versus 60 to 80% success rates using tape and, and by reducing the required capacity for the same amount of data through deduplication, the storage system can scale to handle this continuous data growth.
• Purchase resilient, reliable disk-based deduplicating systems. Using commodity disk storage systems as backup targets may be appear economical on the front end but as backup storage capacities grow, so do their costs with some estimating that management and operation costs often come in at several times the acquisition cost of these systems.  Organizations need to purchase resilient, disk-based deduplication backup solutions that automatically handle tasks like disk drive failures with little or no intervention required on the part of IT staff to fix these types of common hardware issues.
• Obtain scalable disk-based backup solutions that can start small and grow large. Most organizations are dealing with budgets that are flat right now but are forecasted to grow in the future. The last thing organizations want to do is buy a disk-based backup solution with available budgetary funds that only meets their short term needs but then does not scale to meet their future needs. Just because budgets are limited does not mean organizations cannot put a solution in place that meets their needs now and into the future.

• Inadequately size for capacity. This may force the organization to introduce another storage system or appliance to archive more data so now they need to manage multiple devices.
• No way to extend the deduplication benefits from the first appliance. Many archiving and backup storage systems now deduplicate data but if a second appliance is introduced, organizations lose whatever benefits of deduplication that the first appliance delivered since the new appliance needs to start deduplicating data from scratch.
• Inadequately sized for performance. To rehydrate large amounts of data for either recovery or eDiscovery, searches could put inordinate loads on the appliance that results in unacceptable slow recovery or search times as well as concurrent normal backup or archive access.
• Some archiving storage systems can only be accessed via proprietary network APIs.  Only applications that support the proprietary network APIs of the storage system can access or retrieve data from the archiving system.
• No upgrade path. Once the data is on the storage system, how do you upgrade the system or migrate the data off the system to a new one?  Kelly told me that some companies she is dealing with have simply stopped trying because it is too expensive to either migrate the data or upgrade the system. She is aware of at least one company that simply turned their archiving storage system off because it was full. Now they only power it on only when some business need dictates.

• First, factor everything into the price of the deduplication solution. Organizations need to make sure the solution has all of the features that they are going to need and use. Examples include compression, data resiliency, deduplication, failover, number of Ethernet ports, replication, sufficient capacity and performance scalability. As these aforementioned organizations found out, overlooking or failing to account for even one of these features can result in an organization needing to change out or purchase another solution much more quickly than they anticipated.
• Second, how much time will it take to manage? Organizations may assume that if they run out of capacity, it is no big deal since they can just add another one and keep going. That sounds good on the surface but the practical ramifications are that they need to go through the whole capacity sizing exercise again. Additionally, the ongoing management of multiple appliances becomes more difficult since administrators need to configure and direct specific backup streams to specific appliances. This may then require they constantly manually balance the load between them so they do not overload specific appliances with too much backup data plus administrators are less efficient since they have to manage separate appliances.
• Third, how are upgrades and maintenance handled?  Again, this will not show up in a quote but if administrators need to regularly spend time applying upgrades and managing the growth of the appliances over their life, that adds to the total cost of ownership plus takes away time that the administrator could be doing tasks that add more tangible value to the organization.

• The guarantees are only applicable to specific applications that satisfy certain conditions. For instance, unless companies plan to backup and deduplicate data from Exchange or Oracle a specified number of times per week, the deduplication guarantee may not apply.
• The vendor is the final arbitrator as to if the conditions of the guarantee are met. You need to use the vendor's software to measure the results and then the vendor interprets the results. This is not to imply the vendor will lie but it does raise some questions as to the validity of the results.
• Remuneration is vague. If the vendor fails to meet the terms of the guarantee, the compensation may consist of additional hardware/software or services from the vendor. Yet in reading through the terms of the various guarantees on the market, the associated corrective actions are vague. For instance, if you need more storage capacity because the guarantee was 20:1 and you only achieved 15:1, how does that translate into software, hardware, or services that the vendor will provide?

As administrators know all too well, the task of managing storage systems (firmware upgrades, user management, storage allocations, etc.) eventually supplants the task of managing the application data residing on those storage systems. Yet managing multiple storage systems is now part of the enterprise storage management equation. Because of this, companies need to factor in how they will manage multiple storage systems from one console as well as migrate data from old systems to new systems to upgrade performance or storage capacity.

To be fair, there is some progress in the area of storage system administration. Companies can now manage multiple storage systems from one console so the same administrator or user account is present across all of the storage systems. In essence, storage system vendors are borrowing the concept of network directory services such as Microsoft Active Directory or LDAP and even starting to grant their storage systems access to these network services to simplify the task of user account management on storage systems.

However the bigger problem of data migration remains the Achilles Heel of storage system managment. If a storage system is fully depreciated, its technology is antiquated or the system is at its limits in performance or storage capacity, there is no easy way for a company to migrate data from an existing storage system to a new one without going through lengthy and painful data migrations.

Further, with disk becoming the primary target for archiving and backup, an interesting phenomenon is occurring - companies are discovering that they do not want to spend more money or time managing these systems. Essentially, they want all of the benefits of using disk for archive or backup without all of the associated downsides because they no longer have the time, patience or resources to deal with managing it (i.e. administration, data migrations and inability to scale and upgrade existing systems).

It is exactly these sorts of constraints that storage systems such as the NEC HYDRAstor that are based on a grid storage architecture address. In previous blog entries I have described how the HYDRAstor can scale storage capacity or performance independently through the use of its Accelerator and Storage Nodes. What is equally as powerful is that the HYDRAstor retains information about the data on the system as new nodes are added or old ones removed. This solves the problem of retaining user account information as the HYDRAstor grows and evolves as well as eliminates the need for companies to create new user accounts.

Even more importantly, the HYDRAstor automatically migrates data from old storage nodes to new ones and then evenly balances the storage capacity across its storage nodes. This automatic migration feature eliminates the normal pain that companies experience planning for outages and how to best migrate data so it laid out appropriately on the new system as well as just the task of actually migrating the data and monitoring its progress. Instead the HYDRAstor takes this task on in a non-disruptive manner for the life of the system.

The reality of grid storage is that companies can change their focus from managing the storage system to once again managing their business. Grid storage provides companies the new type of storage system that takes care of itself by self-managing its administration, growth and data migrations for as long as the business uses the system. Storage systems based on grid storage lay the foundation to become a company's one system forever that allows companies to take advantage of continual advances in capacity and performance, take the pain out of data migrations and preserve user accounts as the system grows and evolves.

Here are the typical concerns that enterprise shops have about using NAS as a backup target and how the HYDRAstor addresses them:

• Limited file system capacity. Backup software and NAS systems are not always the best of friends when it comes to disk-based backup. While some backup software has the ability to manage out-of-space conditions on file systems, not every product does, so jobs can hang or abort unless an administrator manually intervenes. The HYDRAstor averts these situations by presenting to the backup software a file system that is essentially of infinite size (256 PB) and can be easily extended on the fly if/when such a need does become a reality. Not only is a 256 PB file system large enough to give even large enterprise shops sufficient headroom to meet their backup storage capacity needs into the foreseeable future, the HYDRAstor also deduplicates and compresses the data as it is stored to further alleviate storage capacity concerns.
• Inability to scale storage capacity. The HYDRAstor does not just support one 256 PB file system but as many as administrators want to create. However it does not require that companies purchase all of that storage capacity at once but enables them to add it over time to the HYDRAstor configuration. In essence, the HYDRAstor is an extreme form of thin provisioning as it thinly provisions a 256 PB file system so it uses only as much storage capacity as backup data is stored to it. I call it extreme because the capacity in the HYDRAstor is available to any file system that needs it at any time across the entire HYDRAstor grid. Similarly, if the HYDRAstor itself starts to run out of storage capacity, it alerts the administrator so he/she can add more Storage Nodes to the HYDRAstor configuration. When additional capacity is added to the HYDRAstor grid, it becomes available for use to any of the file systems that need it without the need to allocate capacity to specific file systems.
• Inability to scale performance. Everything about backups is performance intensive, especially when using disk as a backup target. The HYDRAstor must reserve some compute overhead for the file system itself; it needs to ingest multiple backup jobs at the same time; it needs to deduplicate and compress the incoming backup jobs; and, finally, it may need to recover and replicate data while these other functions are executing. Since all of this is occurring in an enterprise environment, it is conceivable that a NAS system may have inadequate performance to go around. The HYDRAstor's grid storage architecture addresses these concerns as well. Just as it offers Storage Nodes to scale storage capacity as companies need it, Accelerator Nodes that can also scale performance so companies can expand performance when they need it. Using this 2-tier grid architecture comprised of Accelerator and Storage Nodes, companies can linearly scale the resources of the HYDRAstor to maintain performance as the system grows.

Enterprises are justified in their skepticism about using NAS as a disk-based target for backup. However the second release of HYDRAstor continues to alleviate some of these concerns. Not only is it a second generation product (which always helps to give enterprise shops more peace of mind) but it continues to make monitoring, managing and scaling capacity, performance or both on the HYDRAstor relatively pain free and simplistic events. And while features such as its 256 PB file systems have been part of HYDRAstor since its first release, when companies factor in its improved performance and higher capacity features coupled with what else the HYDRAstor already has to offer, HYDRAstor begins to present a compelling case for using NAS at the enterprise level for disk-based backup.

For those of you unfamiliar with the HYDRAstor, it is not a virtual tape library (VTL) but a networked attached storage (NAS) system that is primarily intended for use as a disk-based backup target. Configured as NAS, it presents a file system (CIFS or NFS) to backup servers that use it as a disk-based backup target. Now normally using NAS as a disk-based backup target may be a concern in enterprise shops for one major reason: file systems have limited capacity that creates an upper limit in terms of how large they can grow.

While VTLs also have upper limits in terms of storage capacity, VTLs appear as physical tape libraries to the backup software. Since backup software knows how to manage physical tape libraries by recognizing out of space conditions on a tape cartridge, it also knows how to recognize out-of-space conditions on virtual tape cartridges on a VTL. So when a virtual tape cartridge fills up, the backup software understands that it needs to start writing to the next virtual tape cartridge, which avoids the out of space condition that can cause backup jobs to hang or fail.

However if a file system fills up backup software may suspend operations or hang since it does not know how to manage out-of-space conditions on file systems. As a result, backup jobs may fail or pause until a backup administrator manually intervenes by either deleting some of the data from the existing file system or re-directing the backup job to a new file system target with more capacity.

HYDRAstor's support of a 256 PB file system helps to negate this specific concern about using NAS as a disk-based target. By providing a disk-based target that can contain 256 PB or more of data, companies can have confidence that directing backup jobs to a HYDRAstor file system will not result in backup jobs hanging because of an out-of-space condition.

Of course, just because administrators can create and present a 256 PB file system as a backup target to corporate backup servers does not mean that the HYDRAstor actually contains 256 PB (or more) of storage capacity. (If fact, I am quite sure the 256 PB file system I created on the HYDRAstor while at VMworld had nowhere near 256 PB of storage capacity behind it.) But because HYDRAstor supports the creation of multiple such file systems, it increases the overall virtual capacity supported on the HYDRAstor (at least from a file system perspective), from petabytes to Exabytes or even larger.

In this respect, the HYDRAstor is like some other storage systems in that it also uses its own form of thin provisioning to present a file system to the backup host so it appears to have more storage capacity than it actually has. The main differentiator is that HYDRAstor does not require any actual provisioning of resources. Using its grid architecture, all capacity across the entire HYDRAstor is available for all file systems and applications as a shared pool of storage. This improves the efficiency of the utilization of the available capacity by the file systems and applications that need it while permitting the addition of more capacity over time.

HYDRAstor's support of a 256 PB (or larger) file system addresses a concern of enterprise shops that often goes unaddressed by current disk-based backup systems that use NAS - unlimited storage capacity. By giving each and every file system on the HYDRAstor the ability to grow to 256 PB and beyond, companies no longer need to worry about managing and extending file systems manually. Instead they only need to set this up at the beginning when they configure the file systems, regularly monitor the HYDRAstor and how close it is getting to capacity (as opposed to individual file systems) and then add new Storage Nodes to the HYDRAstor when it starts to run short on capacity. HYDRAstor then takes it from there and automatically assigns the capacity on as-needed basis to specific file systems which can then be extended on-the-fly to whatever size file system is needed.

Part of the reason companies might miss these features is that they are more specifically targeted at enterprises either looking to bring a HYDRAstor in house for the first time or for those looking to extend data protection to remote offices or data centers. The two new features of the HYDRAstor that address these concerns include:

• Rack-less models of the HS8-2000. Part of the impetus for introducing a rack-less model of the HYDRAstor was two-fold. First, it provides companies a low-cost option to introduce the HYDRAstor into their backup environment, especially if they already have their own standardized rack hardware. The other reason is it meets the less stringent requirements for the protection of data in remote and branch offices. While the grid storage architecture of the HYDRAstor is well-suited for corporations that need disk-based solutions that are scalable from both a capacity and performance perspective, the entry level solution of the previous generation HS8-1000 was a bit of overkill for some remote offices. The HS8-2002S rack-less model that is now part of the HS8-2000 series gives companies the option to start with a smaller HYDRAstor configuration (1 Accelerator Node, 2 Storage Nodes) for their remote offices with the option to upgrade to the larger HS8-2004S. Many-to-one replication. As companies deploy more HYDRAstor grids in other data centers or in remote offices, companies also want the option to replicate that data back to one remote site. While HYDRAstor already supported asynchronous replication with the HS8-1000, the HS8-1000 only supported one-to-one replication. The HS8-2000's introduction of new, entry level models for remote offices makes it a logical time to enhance the replication capabilities found on the HS8-2000 so one central HYDRAstor can receive data transmitted from one or more HYDRAstor grids located in remote offices. Since companies can control not only when data is replicated between HYDRAstor grids but also what data is replicated by selecting which filesystems they want replicated on each HYDRAstor, companies can minimize network bandwidth between sites as well as minimize the capacity requirements at the DR site.

To date much of NEC's focus with the HYDRAstor was on providing enterprise caliber disk-based data protection that could easily and effortlessly scale either capacity or performance. That mission was accomplished in the HS8-1000. Now by adding rack-less models along with many-to-one replication to the new HS8-2000, enterprise companies can more economically extend HYDRAstor's reach outside of the data center and into remote offices while bringing their data back to the home office. In so doing, companies can bring HYDRAstor's benefits to these locations without introducing some of the data management headaches that tend to accompany other disk-based backup solutions.