Thursday, November 18, 2010

In this article, “traditional” storage arrays refer to the single or redundant storage controller (processing) systems which are connected to shelves of disk drives. The processing power of the entire system is fixed by the number and configuration of the controller(s) it has and can’t be scaled as storage capacity can. A clustered “scale-up” system refers to a modular, or node-based architecture with separate storage and processing modules, each of which can be added to the cluster independently. A clustered “scale-out” system would be a node-based architecture in which processing capacity and storage capacity are combined in every module. This means that scaling capacity also scales processing to some extent, although there may be a choice of the configuration of processing power and storage density available in each module.

Compared with traditional ‘monolithic’ storage arrays, scale-out storage has a key benefit, it allows the controller architecture to grow in conjunction with the storage capacity limiting a performance drop-off as that capacity is increased. Some solutions also offer the flexibility to choose the configuration of the nodes that are added - allowing either a capacity or processing emphasis. This topology should allow storage systems to grow almost without limit and not suffer performance penalties as they do so.

The downside with the scale-out strategy can be cost. Pay as you grow is an appropriate moniker, as these systems get larger, their total cost of ownership (TCO) can be much greater than the fundamental cost of disk drive capacity and the processing power required to support it. There are other factors involved with this architecture that can effectively add to the overall cost per GB.

Efficiency

The physically connected cluster of independent storage nodes includes a number of cost factors that aren’t always readily apparent. Each node includes power supplies, storage controllers, NICs, cabling and the collective ‘sheet metal’ - chassis, connectors, fans, front panel, etc. Each node also consumes power, more watts per GB than a system with fewer chassis would, and may involve more licensing costs. It becomes a case of efficiency, or the lack of it. While expanding storage requires additional drive capacity, adding more physical nodes simply to support those drives can result in a system with an ‘over capacity’ in the other components, especially processing power.

Performance

Storage performance is measured by throughput, latency and IOPS. The logic of adding processing power with capacity is sound, but the impact of managing the distribution of that processing power in a scale-out system can create ‘metadata overhead’. This distributed metadata architecture generates overhead as more nodes are involved in the inter-node communication that accompanies each storage read and write request. In high-node clusters, this can mean more latency, which can affect IOPS and overall throughput. This is especially a concern in high file-count, small file-size environments.

Flexibility

Building a large storage infrastructure out of fixed-configuration nodes may not offer the flexibility to tailor that system to the specific data types stored or workloads processed. For example, a large reference data archive of image files would have a different storage profile than say, data supporting a large virtual server infrastructure or a group of online databases. One big ‘global’ file system can’t be tuned for varying types of performance profiles and consequently, may not have the flexibility to support these different workloads effectively.

Port Support

Running a cluster of storage nodes involves paying for connections in the form of switch ports and cabling. This ‘port support’ includes the acquisition costs, but also the implementation and support efforts required to manage a potentially complex mesh of storage and networking hardware.

Scale-up storage, like the Titan System from BlueArc, may offer a viable alternative to the ‘the pay as you grow’ experience of scale-out storage. These systems decouple the physical storage from the storage controllers producing a more effective combination of comprehensive scalability with flexible configuration.

The specialized controller architecture of modern scale-up systems includes separate resources for the network interface, RAID functions and file system processing, assuring that adequate CPU power is available for each of these data functions. Storage capacity can be increased by adding storage modules which do not automatically include processing power. Similarly, a number of processing modules can be added independently from storage, enabling the scale-up system to grow to the size needed in the configuration to best meet the requirements of current workload and data types. This flexibility to modify the ‘shape’ of a storage system, or its performance profile for a given storage capacity level, is a strong advantage to the clustered, scale-up architecture. It enables capacity to be increased, which brings down the cost per GB, while maintaining acceptable performance.

The distributed node clusters of scale-out systems can improve throughput but not reduce latency. Scale-up systems contain more centralized processing architecture and fewer processing nodes, which means less ‘metadata overhead’ is generated with each storage request, and the system spends less time locating a desired data block. The result is potentially better performance latency. The fewer processing units required to meet the performance demands, the lower the latency is for each storage request and the more efficient the overall use of the resources will be.

As an example, a data set of very large numbers of small files would probably be better supported with a small block-size file system, as opposed to an archive which contains very large satellite image files. Scale-up systems with a ‘global namespace’ can provide the flexibility to support a number of different file systems, each configured for the workloads they’ll see. This flexibility, along with improved overall performance, can make scale-up storage a better fit for an environment in which throughput isn’t the only requirement and where the performance demands of the various workloads require flexible volume tuning.

Save as you Grow

The scale-up architecture often results in a system with many fewer physical boxes than does the scale-out architecture. This means less is spent on ‘sheet metal’ - chassis, power, connectivity and other support components - per GB of total storage, as well as rack space. Scale-up also allows for the addition of capacity alone, without adding any processing power.

In real-world implementations, more storage space is often needed to accommodate older data or increased growth, but performance of the existing processing modules is adequate to support more capacity within the cluster. Overall, this cost efficiency of scale-up storage systems can mean an increasingly lower cost per GB as they scale, to save as they grow. Adding capacity with scale-out nodes that include unneeded processing power does nothing but increase costs.

Since the advent of resource consolidation and sharing in IT infrastructures, there has been a need to scale storage. Implicit in this is the requirement that processing power keep pace with raw capacity, or else performance could suffer. Scale-out storage is a modular architecture which provides a method for scaling processing in conjunction with storage capacity, but can reduce efficiency if adding modules for capacity increases processing power when it’s not needed. Clustered, scale-up storage allows processing to be expanded independently from storage capacity, resulting in systems with fewer modules, less underused processing power and lower overall costs.

Eric Slack, Senior Analyst