Open Source Software
NVM Express technology is designed from the ground up for SSDs to create more efficient, scalable, high performance storage interfaces.
Below is a list of open source software that can be used in parallel with NVMe™ technology to further enhance SSD management and performance.
The NVMe storage command line interface utility (nvme-cli) is the standard open source tool to manage NVMe SSDs in Linux. NVMe-CLI is open source, supported in all major distributions, has a powerful feature set that follows the NVMe spec, supports NVMe SSDs as well as NVMe-oF, and offers optional vendor plugins for supplemental information above and beyond the spec. You can find the blog post that covers common management tasks for NVMe SSDs here.
Zoned Namespace (ZNS) Command Set and Zoned Storage
Zoned Storage is a class of storage devices that enables host and storage devices to cooperate to achieve higher storage capacities, increased throughput and lower latencies. The zoned storage interface is available for NVMe with the NVMe Zoned Namespace (ZNS) Command Set specification. Additional ZNS resources:
- General information: ZonedStorage.io
- Libraries and Tools with Zoned Namespace (ZNS) Command Set Support
- libzbd
libzbdis a library providing functions simplifying access to zoned block device information and the execution of zone management operations. - nvme-cli
nvme-cli allows interaction with devices that supports the Zoned Namespace (ZNS) Command Set. Including reporting zones, performing zone actions, and other ZNS specific commands. - libnvme
libnvme is the C development library for NVMe on Linux. libnvme provides type defintions for NVMe specification structures, enumerations, and bit fields, helper functions to construct, dispatch, and decode commands and payloads, and utilities to connect, scan, and manage nvme devices on a Linux system. - fio
fio is a flexible I/O tester. - blkzone
blkzone is a command line tool used to run zone commands on devices that support the Zoned Block Commands (ZBC), Zoned-device ATA Commands (ZAC), or Zoned Namespace (ZNS) Command Set.
- libzbd
- Applications with Zoned Namespace (ZNS) Command Set Support
- RocksDB (pull request submitted).
- Ceph (preparation patches).
- QEMU ZNS Emulation Support (pull request submitted).
- File-Systems with Zoned Namespace (ZNS) Command Set Support
- General Linux Enablement
- Linux Kernel ZNS Support (General availability in the 5.9 release)
Storage Performance Developer Kit
The Storage Performance Development Kit (SPDK) provides a set of tools and libraries for writing high performance, scalable, user-mode storage applications. The bedrock of SPDK is a user space, polled-mode, asynchronous, lockless NVMe driver. This provides zero-copy, highly parallel access directly to an SSD from a user space application. The driver is written as a C library with a single public header.
SPDK further provides a full block stack as a user space library that performs many of the same operations as a block stack in an operating system. This includes unifying the interface between disparate storage devices, queueing to handle conditions such as out of memory or I/O hangs, and logical volume management.
Finally, SPDK provides NVMe-oF, iSCSI, and vhost servers built on top of these components that are capable of serving disks over the network or to other processes. The standard Linux kernel initiators for NVMe-oF and iSCSI interoperate with these targets, as well as QEMU with vhost. These servers can be up to an order of magnitude more CPU efficient than other implementations. These targets can be used as examples of how to implement a high performance storage target or used as the basis for production deployments.
SPDK has an active development community comprising core maintainers from Intel, Mellanox, and Hitachi, as well as thousands of contributions from across the industry each year. SPDK is also currently deployed in a wide variety of software products and services from SANs to managed distributed cloud databases, powering some of the world’s most heavily used storage stacks.
xNVMe provides the means to program and interact with NVMe devices from user space. The foundation of xNVMe is libxnvme, a user space library for working with NVMe devices. It provides a C API for memory management, such as allocating physical/DMA transferable memory when needed. The libxnvme user space library provides a unifying C API for NVMe tool builders, application developers, and for anyone wanting their host-defined software to run on more than a single platform.
RocksDB is a storage engine with key/value interface, where keys and values are arbitrary byte streams. RocksDB can adapt to a variety of production environments, including pure memory, Flash, hard disks or remote storage. RocksDB is designed for application servers wanting to store up to a few terabytes of data on local or remote storage systems and is specially optimized for storing small to medium size key-values on fast storage. RocksDB Releases are published on the RocksDB GitHub page.