Enterprise Storage Guide, Guides

Don’t Let Raw Storage Metrics Solely Drive Your Buying Decision

Let’s pretend that you’re buying a car.  As you review the specs, you might find that the Tesla Model S can go from zero to sixty in just 4.2 seconds and that the car has a top speed of 130 miles per hour.

“Wow!”, you think to yourself.  That’s really fast.  But, rarely do you then make the leap to buying the care based on just those metrics.  In fact, it’s more than likely that you consider a number of factors before you take the plunge to buying a new car.  Such considerations might include:

  • How you intend to drive the car.  Are you planning to floor it every time you get into the car to get to 60 miles per hour as quickly as possible?
  • Do you really need to go 120 miles per hour?
  • Is an all electric car really practical with a 230 mile range based on your needs?  This is kind of akin to buying an all-flash array in some ways.  Do you need that kind of performance for every single workload in your organization?

Sidebar: Who is Coho Data?

Coho is a software company that packages and sells on top of hardware.  The company has created a unique hybrid storage array that combines flash and spinning disk.  They think it’s crazy to put all data on flash due to cost. Coho Data’s offering includes a 10 Gb Arista switch at the top of the storage rack – this is not an ordinary switch, though.  It’s a software-defined switch that is an active part of their storage stack.

As of this writing, Coho Data has just gone GA with their first product.  In 2U of rack space, Coho Data provides:

  • 20 TB of usable capacity
  • 3.2 TB PCIe flash capacity
  • 40 Gb access – 4 x 10 GbE Ethernet uplinks to the aforementioned SDN switch
  • 180K IOPS (80% read/20% write, 4K random)

Coho Data believes that as much active data as possible should live in flash.  To that end, the company takes a “flash first” approach.  Cold data lives on spinning disk.  They also believe that, if you’re not pushing flash storage to its limits, you’re paying too much $/IOPS.

Don’t Chase Low Latency At All Costs

I included the sidebar about Coho Data because I’m at Coho HQ learning about their product and some of their philosophy behind storage.  I really like what I’m hearing, hence the title of this article.  Coho learned quickly that, sometimes, the metrics that matter the most – i.e. storage latency – shouldn’t necessarily try to be minimized.  Instead, these metrics should be optimized and balanced against all of the other aspects of the system.

For example, during development, Coho Data learned that they could set the storage queue depth to 1.  If you’re unfamiliar with storage queue depth, it’s basically a metric that defines how many parallel operations the storage will allow.  A queue depth of one means that the storage will always service just a single request at a time.
The result: An impressive latency figure of just 133 microseconds.


The downside: It resulted in a measly 6,708 IOPS from the system. That’s not so good. Through testing, Coho Data ultimate discovered that the sweet spot with regard to queue depth is 32.  At this queue depth, latency jumped to 235 microseconds, but the IOPS potential skyrocketed to 134,538!

“But, “ you may ask, “That doubled the latency!”

To which I say, “Who cares?”  Keep that latency figure in context.  235 microseconds is still an insanely awesome and enviable latency figure, particularly for storage administrators used to dealing with latency issues in than span the milliseconds range.

The key takeaway here is that you must focus on the metrics that matter and keep them in context during the buying decision.  For 99% of today’s workloads, a 235 microsecond response time is absolutely acceptable, particularly when compared against a 22x jump in IOPS capacity.

The Network Shouldn’t Be A Barrier

Today, we’re seeing a trend that involves storage vendors moving storage into host systems and reverting back to a direct attached storage approach to providing capacity.  Coho Data considered this as well and discovered that their solution imposes around a 40 microsecond networking penalty. In other words, it’s almost not even worth considering, let alone worrying about as a part of a new implementation project.

So, Coho Data made the decision to move ahead with an array-based product based on the fact that latency wasn’t really an issue in their design.  This is data-driven decision making if I’ve ever seen it!

The Lessons

Vendors are really good at making their own statistics.  Employ your critical thinking skills when it’s time to buy storage and then look at the right metrics.  If you don’t need a Tesla when the Hyundai will deliverer everything you need, why spend the extra money.

Make sure your metrics fit your use case. Personally, I like the balance that Coho Data brings to the table.  There is good thinking behind their decision.  Make sure you take a similar approach as you explore various vendors.