Tuesday, November 23, 2010

Existing WAN optimization solutions don’t typically provide enough throughput per chassis to be cost-effective for this traffic between data centers. However, a new company called Infineta has developed a product which is designed to improve network performance for organizations replicating between large data centers. According to Infineta, this device can provide 5x better price/performance, 5-10x better throughput and 20x lower port-to-port latency than traditional WAN optimization technologies.

Improving throughput comes down to reducing the amount of data transferred over the WAN, which for optimized networks means deduplication. According to Infineta, the dedupe methods used by WAN optimization solutions are more suited for the data-at-rest profile of a storage environment than the data-in-flight profile of a network environment. Infineta has developed their own deduplication technology, called the Velocity Dedupe Engine (VDE), which uses parallel processes at several levels to produce the data reduction required for high speed WANs.

Current Dedupe is serial process

Current dedupe methods are sequential in nature. The process of defining the breakpoints in each data block and the hash calculation and comparison steps used to identify unique data segments is done for one data block at a time. Consequently, dedupe algorithms typically strive to create the largest blocks possible in order to improve efficiency. In a networking implementation, this means packets are accumulated to make up larger blocks before the dedupe process is run.

Infineta’s technology has nothing to do with blocks. Rather, they perform dedupe on a packet by packet basis. Infineta’s VDE was invented from the ground-up as a highly parallelizable process to handle multi-Gb WAN traffic and it is implemented in custom programmable logic (FPGA) to exploit this parallelism, as opposed to software running on traditional computer CPUs. According to Infineta’s Co-Founder and CTO KVS “Ram” Ramarao, “Other dedupe-capable products operate on fairly large size (e.g., 1KB or greater) ‘chunks’. If anything changes within that chunk, it is treated as a new chunk and cannot be deduped. In contrast, VDE performs string matches at granularity as small as 8 bytes and identifies even extremely small changes.”

Distributed system architecture

With the FPGA architecture, Infineta can create hundreds of active hardware-based mini dedupe engines or ‘nodes’ running in parallel. Within each node, blocks are processed in real time with multiple blocks actually moving through the dedupe steps simultaneously, not waiting until the preceding block has finished.

The VDE technology was developed by Infineta in software first and then implemented in hardware to leverage its potential performance in high-throughput, east-west WAN applications and produce a cost-effective optimization solution. This parallel, hardware-based architecture looks to break the serialization bottleneck that traditional dedupe processes have in networking implementations and deliver a sustained performance.

Storage Switzerland’s Take

Although Storage Switzerland hasn’t tested this solution in our lab, their deduplication technology, and certainly their results are compelling. According to Infineta, VDE produced 80-90% data reduction at line rates up to 10Gbps, without impacting throughput. Clearly data center to data center communication is on the rise and that data traffic is different from the traditional remote office to data center traffic. Infineta seems well positioned to help customers solve challenges that these data centers can create.

Eric Slack, Senior Analyst