Overcoming CPU Chokepoints For NVMe

By Hubert Yoshida posted 08-12-2019 00:00


Marc Staimer of Dragon Slayer Consulting published a recent article on the CPU chokepoint in servers and controllers for NVMe storage. This supports blogs that I have recently posted on CPU architectural limitations and the need for accelerated compute and other computer architectures.

Marc observes that as more NVMe flash SSDs are required, then the supporting hardware gets increasingly complicated. “It usually means more CPUs, either internal or external ones. The storage can be DAS or shared across NVMe-oF. Either way, more CPUs, drives, drive drawers, switches, adapters, transceivers and cables will be required. The general industry consensus is that scaling capacity and performance using NVMe drives and NVMe-oF just requires more hardware. Storage Class Memory technologies will only exacerbate the CPU chokepoint problem, because their increased performance puts even more load pressure on the CPU.”

“But here's the rub. These systems offer quite noticeable diminishing marginal returns. The hardware grows much faster than the performance gains. This occurs no matter how many CPUs or NVMe flash SSDs are added. Eventually, more hardware means a negative return on overall performance.

The root cause of this NVMe performance challenge isn't hardware. It's storage software that wasn't designed for CPU efficiency. Why bother with efficiency when CPU performance was doubling every 18 to 24 months? Features, such as deduplication, compression, snapshots, clones, replication, tiering and error detection and correction, were continually added to storage software. And many of these features were CPU intensive. When storage software is consuming CPU resources, they aren't available for storage I/O to the high-performance drives.

While Hitachi Vantara, has not yet delivered NVMe or NVMe-oF in the enterprise storage VSP platform, we have been making changes to the VSP Storage controller in preparation for the introduction of NVMe and NVMe oF when the standards become finalized. I blogged about this a year ago. Essentially, we have rewritten the SVOS (Storage Virtualization Operating System) VSP controller software for NVMe and released it as SVOS RF where the RF stands for Resilient Flash. This systems software was re architected and designed to optimize and scale NVMe performance. The other thing we did in the hardware was to accelerate compute through the offload of some tasks to FPGAs. We also optimized the data path with improved cache algorithms.

The result of these changes helped to accelerate our performance even without NMVe or NVMe oF. Our Flash performance with SAS (Serial Attached SCSI) are comparable to some of the startups who are delivering NVMe storage systems.

Last year’s August 6, 2018, Gartner Critical Capabilities for Solid State Arrays report provides some insight on what our capabilities will be. In terms of performance rating, the VSP F series came in third in front of several vendors that had NVMe. This evaluation did not include the latest SVOS RF and VSP F900/F700/F370/F350 enhancements which were announced in May 2018 because they did not make Gartner’s cutoff date for the 2018 evaluation. These new enhancements featured an improved overall flash design, with 3x more IOPS, lower latency and 2.5x more capacity than previous VSP all flash systems.

The only two vendors ahead of the F series in performance at that time, were the Kaminario K2 and the Pure Storage Flash Blade, none of which have the high reliability, scalability and enterprise data services of the VSP.  In fact, the VSP F series placed the highest in RAS (reliability, availability, serviceability) of all 18 products that were evaluated. The Kaminario K2 has a proprietary NVMe-oF host connection which they call NVMeF. One can assume that the performance of the Hitachi Vantara All Flash Arrays, even with SCSI/SAS would be higher if the new models of the VSP and SVOS RF had been included in the evaluation. Here are the Product scores for the High-Performance Use Case for the top three places on a scale from 1 to 5 with 5 being the highest.

Kaminario K2                                       4.13

Pure Storage FlashBlade                    4.08

Hitachi VSP F Series                            4.03

Pure Storage M and X Series             4.03

While the standards for NVMe oF are still being worked on and still to be proven, the NVMe standards are pretty close to being finalized, so you can expect to see NVMe coming soon from Hitachi in the near future, and you should expect to see It blow away the competition, since we have already done the ground work to address the choke points that Marc Staimer identifies.


1 comment



04-25-2022 08:25

Nice Read