Top 10 Things You Need to Know About NVMe Devices

Non-Volatile Memory express (NVMe) devices are now mainstream. They are no longer considered exotic, and, as such, they have a wide variety of use cases.
A year ago, Micron supplied me with a couple of enterprise NVMe devices, and I have been using them in my vSphere servers as cache devices and for primary storage without encountering a single issue: they have been rock solid, and my colleagues have also reported similar results with their Intel and Samsung NVMe devices.
NVMe devices function a little bit differently than the traditional storage devices used in datacenters, and if you’re unfamiliar with NVMe devices or are considering moving that direction, here are the top 10 things you should know about them.

  1. NVMe is an interface. NVMe is an interface to flash devices, but NVMe devices have considerably less latency and are more performant than SATA flash devices. Much of the NVMe devices’ performance comes from a streamlined IO stack and ability to handle IO requests in parallel.
  2. The first write doesn’t count. For a variety of reasons, the first write to an NVMe device will be far faster than all subsequent writes. Before you benchmark a device, it’s important to make sure that you precondition the device by writing to the entire device once so that you can get an accurate representation of the speeds the device is actually capable of achieving.
  3. They come in different form factors. The three major form factors for NVMe devices are 1) M.2, which is used on a motherboard, 2) FHHL and HHHL cards, which plug directly into a PCIe slot, and 3) U.2 NVMe devices that fit in a standard 2.5-inch drive slot. Because 2.5-inch NVMe devices are hot-swappable like SSD or HHD devices, are becoming more popular for datacenter servers and in storage arrays.
  4. Capacity is not an issue. Early NVMe devices had limited size capacity, but 4TB devices are widely available today. Furthermore, all the major brands have announced that even larger capacity devices are in the works.
  5. Your OS supports them. All modern operating systems support NVMe devices, and have standard drivers for them. For the most part, modern NVMe drives offer plug-and-play installation.
  6. Your hypervisor supports them. VMware vSphere, Hyper-V, XenServer and KVM all support NVMe devices as cache devices or for primary storage. As an added bonus, many hypervisors even allow direct access to an NVMe device from a virtual machine.
  7. They aren’t that expensive. The cost of NVMe devices has fallen dramatically over the last few years; on an IOPS-per-dollar basis, NVMe devices are now comparable with SSD devices. In the future, the price of NVMe devices will undoubtedly continue to fall, which will further open them up for more use cases.
  8. You can get them in the cloud. You can get NVMe devices on AWS and Google Cloud Platform instances. In today’s competitive market, I’m certain other cloud providers will soon be offering them as well.
  9. NVMe arrays are an option. All the major storage companies, as well as some innovative startups like Mangstor and Apeiron, are either working on or have already figured out how to aggregate an array full of NVMe devices and offer them for consumption as either traditional storage, or as NVMe over fabrics or Ethernet.
  10. They do have a life expectancy, but don’t worry about it. All NVMe devices have a rated life expectancy expressed in Drive Writes Per Day (DWPD) or Terabytes Written (TBW), both of which you can use to figure out if a drive is adequate for use in your datacenters. When the device does start to encroach upon its maximum endurance, it will prevent errors and warn you that it is reaching its end of life. As an added measure, you can query most devices to see how “worn out” it is, and how much life it has left in it.