Monday, December 6, 2010

Limitations of The HDD

The primary issue with mechanical hard disk drives (HDD) is their ability to deliver acceptable performance. The rotational speed or RPM of the HDD has been stuck at 15K RPM since the late 1990s. In that timeframe processing power, memory capacity and networks have all increased in performance ten fold. The only performance improvement seen in individual HDDs has been as a result of capacity increases, more specifically, density increases. Higher recording substrate (disk platter) density means more data can be read per rotation. This increase still pales in comparison to the increase in demand for performance in this same time period.

HDD Speed Increases in the “Modern Era”

HDD Performance Workarounds Create More Problems Than They Fix

Since 2000 the storage I/O demands have not remained static despite the fact that HDD technology has. With the advent of shared storage and then server virtualization the problem has become worse. Less I/O demanding applications are now consolidated onto a single server, via virtualization, all accessing the same storage. The result is that most servers in the environment now have higher storage I/O needs and the typical workarounds have been a multi-step recipe for disaster.

First, very large RAID groups were built leveraging dozens of HDDs, increasing exposure to a double-drive failure. This led to the invention of RAID 6, which incurred an additional write penalty but enabled IT organizations to survive the double-drive failure risk. Of course, the chance of a failure in a third drive while a rebuild is going on remains. Additionally, as the capacities of HDDs increase so does the time required to rebuild them, and the time the system is vulnerable to another drive failure.

A large population of drives only helps performance while there is sufficient queue depth to saturate all the available drives. Once storage request queue depth has been exhausted then response time known as latency must be addressed or the application can be re-written to generate more disk I/O. Reducing latency can only be done via a faster drive, which if the system is using 15K RPM drives does not exist. The only option then is to use the faster portion of the drive exclusively, the outer edge of the platter. This method, called “short stroking”, does improve response time and reduces latency, but at the cost of 2/3 of the disk capacity of the drives.

If additional performance is needed there are no other options with HDD technology. Applications then have to be re-written to deal with the limitations and to increase queue depth. Database sharding is an excellent example of this practice. Re-writing the application leads to additional server purchases and extremely complex code re-writes.

In either case the data center is left with hundreds of drives, potentially running at two-thirds of their available capacity, taking up a multitude of rack space. These workarounds create space issues, power and cooling issues, increase storage management complexity -  all while not really eliminating the performance problem.

Mixing Solid State and HDD

An emerging workaround for manufacturers of HDD based storage systems is the mixture of solid state and spinning disk drives. After all, manufacturers were able to mix drive technologies when SATA based disk drives appeared on the market. Forgotten, of course, are the problems when these drives first appeared, such as ensuring proper timing between the faster fibre drive and slower SATA drive. Still a problem today is how to move data between these tiers of storage. 

The problem with mixing solid state disk into storage systems is that, unlike SATA and Fibre HDDs, these devices are vastly different. Unfortunately, legacy storage system manufacturers try to simply plug SSDs into their current shelves, although they weren’t designed to support the high performance capabilities of solid state technology. What’s needed is for the legacy manufacturers to either sell those shelves half populated or sell them full and make the user live with sub-optimal SSD performance. Neither is good for the customer.

The mixture of SSD into legacy systems is also not cost or space-efficient. From a physical perspective, many more SSDs could be installed per shelf since there are no vibration issues and limited heat concerns. Greater density of SSD would lower costs and reduce floor space, neither of which is delivered when mixing SSD in legacy systems.

Finally, there is the challenge of figuring out which data should be on the SSD tier and when. Some manufacturers have implemented complex data movement engines to help with this but those have issues as well, like performance impact during data migration and data QoS definitions (and continuous high power consumption during data movement and uncertain destiny for data “in flight” during hardware failures or connectivity interrupts between shelves).

HDD Performance Workarounds

Sustainable Storage - The Next Generation Of Storage

Mechanical storage served its purpose but its inability to improve performance to meet the growing needs of the data center is resulting in costly workarounds. These schemes increase drive counts and/or server counts while decreasing efficiency result in higher costs to manage, power and physically house storage systems while also increasing the risk of drive failures. What’s needed is more than a band-aid; the data center needs the next generation in storage, based solely on flash technology. This storage, which Nimbus Data Systems calls "Sustainable Storage", is designed to deliver the performance that the data center needs at the right price point without an endless appetite for power and data center floor space.

The challenge flash memory has faced in enterprise acceptance has been with concerns over cost, reliability and integration. The sustainable storage method addresses these concerns by eliminating the constraints of having to co-exist with legacy storage and delivering a network-ready, pure flash based storage system with complete data services, including multi-protocol unified SAN and NAS capabilities, integrated RAID protection, snapshots, replication, and deduplication. The result is a faster, simpler, greener storage system ready for today's storage challenges, and tomorrow’s as well.

Advantages Of Flash Storage

In the rest of our series on Sustainable Storage, we will look at the prerequisites for delivering this type of solution as well as the best applications to take advantage of this new generation of storage.

George Crump, Senior Analyst