Lack of standardization in SSD specs makes a definitive comparison challenging and a final decision difficult to make on the basis of manufacturers’ data alone. Published specs are useful as a first pass, to identify the devices which are NOT worthy of further examination. But users are advised to run their own tests on the most promising candidates, using real world conditions and infrastructure, to arrive at a final decision. As a first step, published data from SSD device manufacturers can provide quantitative information about device performance and expected behavior. But these data can also provide qualitative information about the manufacturer’s understanding of SSDs and their commitment to the testing process, which can impact the reliability of their published specs.

SSD Performance Specifications

The lack of standards and the lack of consistency in how SSDs are configured has lead to confusion on how the performance data are reported to users. Many SSD vendors will report their performance numbers when the drive is empty. This is known as the Fresh Out of Box (FOB) state and in the case of SSD can be quite misleading because of the way data is written to flash drives. Unlike hard disk drives (HDD) which make changes by overwriting data at the bit level, a flash device writes data a block at a time and more importantly, must clear space ahead of each block write, a process commonly referred to as “garbage collection”.

Think of writing data to a flash drive like storing data in 1,000 cups of water. Each cup represents a memory cell on the Flash NAND and the water represents the data that’s being stored. To store new data, water needs to be poured into a cup, a simple process if cups are empty. If they’re not, the cups must be examined to see if the data can be erased, in preparation for a new write and if it is the cup, must be emptied first before the new water added. This takes time and can make the write process for a new flash device much faster than one which has been ‘filled up’ the first time.

Testing a drive fresh out of the box is not indicative at all of what long term performance will be. Steady State is the term used to describe a drive that has had all of its cells written to and each new write is accommodated by running the garbage collection routines described above. This is the state that vendors should be reporting their performance from, one in which all performance variability has died away from the SSD. Micron is an example of a company that does this.

Steady State

As an example of this Steady State concept, see the Write Saturation curves below. This chart, which comes from a test run by Calypso Systems, graphs performance (in IOPS) as a function of time during a test of continuous write activity. The blue curve, representing the Micron 100GB P300, shows the various levels of performance as the device is written to. The first plateau is the FOB region and the flat line at ~500min is the Steady State region. The slope of the region inbetween will vary between devices, as can be seen here, but the point is that accurate long term performance won’t be observed until the device has reached Steady State. The red and green curves represent the Intel 64GB X25E and Samsung 100GB SS80S, respectively.

George Crump, Senior Analyst

Micron Technology is a client of Storage Switzerland

To help with write performance many, if not all, SSD manufacturers will add RAM to their SSD to cache writes, while the garbage collection process detailed earlier is performed. Doing so reduces the performance impact of clearing a memory cell (emptying the cups) but may put data at risk from a drive failure that occurs before the write completes, causing data corruption or even data loss. As a result storage managers may choose to turn write caching off. If the drive is going to be used in a cache-off mode, make sure it is tested in that mode. If published device specs are reported with only the write cache enabled, it’s imperative that they represent a device that was in a true Steady State condition.

SSD Endurance Specifications

SSDs have a finite ‘lifespan’, a product parameter that HDDs don’t have, so performance data alone isn’t enough to make a meaningful comparison. Basically, NAND flash cells have a finite number of times they can be written to - called Program/Erase cycles (P/E cycles). A lifespan figure is reported as Total Bytes Written (TBW) and is essentially a product of its maximum rated P/E cycles and the device’s usable capacity in bytes. There’s a factor called “write amplification” which can impact TBW but beyond the scope of this article. When comparing published specs for SSDs, endurance in TBW should be included in the analysis. If these data aren’t readily available, ask the vendor.

Qualitative analysis

If the maximum rated P/E cycle number seems out of line with other devices, get more information from the supplier. If a device manufacturer doesn’t have complete, understandable specs on performance, exclude them. If a manufacturer isn’t clear (or sure) about how their performance numbers were derived or aren’t forthcoming with information, their test results may be suspect as well.

Suppliers like Micron, which are vertically integrated, meaning they produce NAND substrate and design flash controllers, will probably have a better understanding of the testing process and be in a position to provide more accurate specs and more complete testing data. SSD vendors that buy these components from other suppliers (many vendors do this) often rely on those OEMs for testing data as well, which may diminish their ability to provide complete and accurate supporting information.

The quality of the device itself is affected by the quality of the flash substrate and the capabilities of the controller. This is an area where a tight, vertical integration between the flash NAND manufacturing process and the design of the flash controller can make a big difference. The controller needs to make sure that writes are evenly distributed across the entire flash module, called “wear leveling”. It also needs to handle garbage collection to make sure that cells on the NAND containing old data are cleaned out ahead of new writes, to deliver consistent performance. Every SSD is a combination of NAND flash (substrate) and controller technologies, a combination which can deliver strikingly different performance between manufacturers.

Reporting SSD performance and reliability statistics is in its infancy but the published data from manufacturers still has value for the comparison process, as long as users are clear about what condition is actually being reported on. First, devices must be in a true Steady State condition to give accurate performance data. Second, endurance specs, expressed as maximum Total Bytes Written, should be comparable for products being considered. Finally, the SSD vendor should be able to provide timely, accurate information and have a clear understanding about the origins of these data. With this data users can narrow down the number of products to evaluate further. That hands-on evaluation process is the key to selecting the right SSD drive for the application, and something that will be covered in our next entry “How To Test SSD”.