Monday, August 15, 2011

With parallel data access, pNFS can maintain and scale performance as capacity expands since adding data movers (or data servers) also increases data access points. Its distributed architecture allows the file system and storage processing functions to be scaled independently from the data handling function, giving the system additional flexibility. The fact that pNFS has been developed as an open standard will enable the pNFS client to be packaged with major operating systems.

pNFS architecture

This article will focus on a common implementation of the pNFS standard, one which uses separate Metadata Servers (MDS) and Data Servers (DS). This architecture enables clients to communicate directly with physical storage modules and move data in parallel, enabling massive scaling and support for parallel node architectures. A separate MDS also allows metadata processing to be configured to meet the needs of different workloads and different file types, apart from storage capacity.

How it works

A third component, the pNFS Client, contacts the MDS to open a file using the pNFS protocol. The MDS transfers the “layout”, device-dependent information about how a file is physically mapped to storage, plus permissions and other file attributes. This would include how a file is distributed throughout the data store, how it’s striped across multiple nodes and spindles, etc. With this information the Client can access a file directly, independent from the MDS, using a storage access protocol, which defines the layout described earlier. Both the pNFS and access protocols are defined by the NFSv4.1 standard.

This direct data access allows multiple data streams between pNFS Clients and the file, block or object storage that the data’s physically written to. When changes occur to data on the Data Servers, they’re synchronized with the MDS via a control protocol, which is not specified in the pNFS standard. This independence allows storage manufacturers the flexibility to create different value added features within their implementations of pNFS.

Why a parallel file system?

The short answer is “performance”. With the rise in popularity of file system storage environments, especially in business critical applications like transactional databases, decision support systems and the internet, storage throughput is becoming a bigger issue. NAS systems are being used in environments once dominated by high-performance block-based SANs. The growth of clustered compute architectures, in high performance computing as an example, is also driving the need for more file-based storage performance. These clustered architectures leverage highly distributed, parallel processing, which means parallel storage and a parallel file system are needed to exploit it.

The increased use of server virtualization in larger, more critical applications is also driving the needs for more storage performance. Parallel architectures are faster than serial architectures, as they can take advantage of the aggregate bandwidth of all the Data Servers involved in a file operation. Plus, the use of parallel hardware infrastructures like scale out NAS systems and distributed node computing are a natural fit for a parallel file system.

pNFS on NAS

Traditional NAS isn’t a particularly good fit for pNFS, as NAS is a ‘siloed’ architecture which funnels all metadata operations and data access through a central controller. Instead, pNFS needs to be implemented on a storage system that can support parallel data access, with the ability scale metadata processing separately from storage capacity. However, pNFS can act as a ‘unified’ protocol, supporting a traditional NAS system as one type of Data Server. Today’s scale out NAS isn’t really architected for separate metadata handling either. While it does support one level of parallelism on the storage side with its multiple node architecture, each client still communicates with the storage systems through a single controller node and this can become a bottleneck for scalability beyond the capabilities of that node.

How to design storage for pNFS

What kinds of design characteristics should a storage system have to best support pNFS? Like any high performance storage system, it should be as efficient as possible with regards to handling data and processing metadata. Storage system manufacturers, like BlueArc, which focused designs on accelerating traditional NAS performance are in an ideal position to exploit the advantages that a parallel file system brings.

Efficiency

When considering data handling, metadata has a disproportionate impact on system efficiency compared with the amount of storage it consumes. That said, pNFS metadata handling does require high IOPS performance, rather than the throughput oriented performance needed for the data servers. Therefore, systems that optimize metadata handling, like locking metadata into the highest performing storage area, can greatly improve performance, especially IOPS. Efficiency can be further improved by moving the most critical, in-data-path operations out of file system software that runs general purpose CPUs. Putting these functions into programmable silicon hardware, like FPGAs, eliminates a potential bottleneck and reduces overhead associated with traditional storage systems which use the system CPU for file system tasks.

Enterprise features

A pNFS storage system should also be architected to provide the reliability and feature set expected from an enterprise NAS platform. Automated tiering of storage types, spinning disk and SSDs, can support dynamic read caching, which keeps files on the fastest storage as they become ‘hotter’. Services like snapshots, replication and deduplication have also become expected features. In addition to pNFS, they should also support other file systems like legacy standard NFS and CIFS client access within a global namespace.

In order to exploit the advantages of pNFS the storage systems must have the ability to spread data files across multiple file systems to increase performance and load balance the system. The distributed but direct I/O between clients and Data Servers makes this possible. In addition to performance and enterprise storage features, a major benefit of a parallel file system like pNFS is flexibility.

Flexibility

Storage systems such as BlueArc’s use purpose-built metadata servers that are designed with efficiency in mind, as well as the concurrent data access required by pNFS. They can be run as a single server (one node acting as both Metadata and Data Server) and then be expanded to include separate Data Servers, as storage requirement grow. The MDS can also be implemented in an HA cluster and can support multiple heterogeneous data servers as a way to consolidate existing storage.

Scale-out storage systems allow performance to be increased as storage grows. But these node architectures still couple metadata and ‘payload’ data processing with  capacity, restricting their ability to optimize storage performance for the workloads involved. The architecture of a ‘Scale-right’ system can fully exploit the separation of metadata processing from data movement, allowing users to configure not just the size, but the ‘shape’ of a storage resource. Some applications require more IOPS while others need more throughput. With the right storage system running pNFS, they can increase performance aspects, like IOPS and throughput in the right proportion, while increasing storage capacity to best match workloads.

Eric Slack, Senior Analyst