Wednesday, March 16, 2011

The first step is to consider what the appeal is for solid state storage in the first place. In most cases it’s to solve a performance problem for a particular set of applications. Most of the time the work-around for these problems is to use maximum amounts of RAM so that local high speed memory can be used for caching. Another way is to add multiple servers and divide up the application (often referred to as application “sharding”) so that many simultaneous requests can be sent to different storage systems with shared server access. A third method is to configure the storage systems with high speed drives where only the outer edges of the platters are formatted to make sure the fastest cylinders are used exclusively. All of these performance workarounds lead to an increased risk of data loss or resource inefficiencies and consume significant portions of the IT budget. Shared PCIe SSD promises to reduces these risks and help contain the costs.

Share PCIe SSD as an Alternative to RAM

As mentioned above one of the first steps to improving application performance is to increase the amount of RAM installed in each server. This allows frequently accessed information to be stored in local, high speed RAM devices, which comprise most computer memory. In essence the application is turning RAM into a storage area. While adding RAM does increase application performance in most cases, this approach has many limitations. First, only so much RAM can be added to a server. Most servers support a maximum of a few hundred GBs, hardly enough to store an entire database and in most cases, not even enough to store a transaction log. If the entire database cannot be stored in RAM then the chances of a cache miss, and potentially even worse performance, exists. Second, RAM is volatile and was never designed to enable memory contents to survive a system re-boot or power failure. Either one of these occurrences could cause data loss if write caching is activated. And, as is more often the case, database administrators will turn on read caching to protect data. While it does reduce the risk of loss much of the performance gain promised by caching is not realized. Third, memory is obviously not storage and not part of the storage stack, so there is no way to make snapshot copies of RAM or to leverage advanced data protection techniques. Finally and potentially most important, memory cannot be reallocated to another system. Shared access to a storage pool is required for server redundancy and for better allocation of resources. For example, if a server needs a high amount of memory during a peak time, it would be ideal if the administrator could ‘borrow’ memory not in use by other systems. Then, after the peak demand has passed, it would also be ideal if that memory could be available for reallocation to another server. It’s unlikely that an administrator would have the time or desire to open server A up and physically move some of its RAM to server B.

The lack of shared access is also what limits the usefulness of dedicated PCIe SSD in large environments. A shared PCIe SSD appliance, like those offered by Aprius, can eliminate that weakness by providing high performance, low latency access to a shared pool of SSD. It does this by transferring PCIe commands on a raw Ethernet frame instead of using iSCSI or Fibre Channel. Shared PCIe removes all of the above limitations and becomes an ideal area for storing cached data. Because of sharing, that cache could be ‘moved’ between servers as needed and could be made large enough to store more of the database’s active data set, reducing the chance of a cache miss.

No Cache

An alternative to using Shared SSD for cache is to use Shared SSD for the entire database. A limitation of stand alone PCIe SSD is the amount of capacity that can be allocated per server. The maximum is about 1-2TBs per card and typically 2-4 cards per server. While those capacities might be acceptable for cache they may not be enough to store the entire database. Shared PCIe SSD appliances commonly have multiple SSD cards that can be combined to provide enough capacity to store entire databases. There are several advantages to this approach. First, there is no risk of latency being introduced because of a cache miss. Second, as stated above, most caches are read-only, meaning they don’t accelerate performance until they are ‘warmed up’ with previously accessed data. With no cache, all read and writes are made directly to and from high speed flash memory. Third, no cache allows for a much better sharing model since the entire database is in one area. This means that the database instance can be moved between servers more easily without having to wait for a cache de-stage.

High Speed Shared Access

One of the most obvious use cases for this type of storage is in the shared database environment like Oracle’s Realtime Application Cluster (RAC). In databases like these multiple physical servers perform operations on the data at the same time. This parallel process allows these databases to service thousands of users with a simple scaling model. Each node or server in the cluster scales performance in a linear fashion. However, it requires that this database storage be accessible by these multiple systems and also requires the storage be high performance, because while the CPU horsepower can scale, most often the storage can’t. In many cases these database administrators are turning to solid state storage. But the requirement for sharing immediately eliminates stand alone PCIe, because while its performance is acceptable, it can’t be shared. The remaining option is to implement niche solid state appliances. While these are shareable, they tend to use highly customized components, something which drives the cost up significantly. Shared PCIe like that offered by Aprius can use off-the-shelf PCIe SSD to maintain competitive pricing while providing a similar high performance shared storage environment, ideal for Oracle RAC and other clustered applications.

Architecture of The Appliances

The shared PCIe systems allow servers to attach over a 10GbE connection. The Aprius PCIe over Ethernet approach for example, supports four 10GbE ports for 40Gb of potential bandwidth to the SSDs. This represents performance similar to what could be achieved if the card was installed in the host itself. These PCIe over Ethernet adaptor cards connect to a switch on the shared SSD appliance, inside which the PCIe SSD cards are inserted. Today’s cards can support two devices or two connections per PCIe card. In the future the card will support being sub-divided even further, and clustered for shared-data applications.

Shared PCIe SSD technology, via solutions like Aprius, allow for sharing SSD without the latency of a network, like fibre or iSCSI. SSD is latentcy-free and the goal of SSD implementations is to preserve that benefit. Shared PCIe accomplishes this and it does so without using highly customized and expensive components. Many environments may find that the combination strikes the right balance of performance, sharing and price.

George Crump, Senior Analyst