Skip to main content

Accessing NVMe volumes for physical NVMe device targets

For SLES 15 SP1 and RHEL 8.1, I/O is directed to the physical NVMe device targets by the Linux host. A native NVMe multipathing solution manages the physical paths underlying the single apparent physical device displayed by the host.

Note
It is best practice to use the links in /dev/disk/by-id/ rather than /dev/nvme0n1, for example: 
# ls /dev/disk/by-id/ -l lrwxrwxrwx 1 root root 13 Oct 18 15:14 nvme- 
eui.0000320f5cad32cf00a0980000af4112 -> ../../nvme0n1

Physical NVMe devices are I/O targets​

Run I/O to the physical nvme device path. There should only be one of these devices present for each namespace using the following format:
/dev/nvme[subsys#]n[id#]

All paths are virtualized using the native multipathing solution underneath this device.

You can view your paths by running:
# nvme list-subsys

Results
nvme-subsys0 - NQN=nqn.1992-08.com.netapp:5700.600a098000d709d6000000005e27796e
\
 +- nvme0 fc traddr=nn-0x200200a098d709d6:pn-0x204200a098d709d6 host_traddr=\
nn-0x200000109b211680:pn-0x100000109b211680 live
 +- nvme1 fc traddr=nn-0x200200a098d709d6:pn-0x204300a098d709d6 host_traddr=\
nn-0x200000109b21167f:pn-0x100000109b21167f live

If you specify a namespace device when using the nvme list-subsys command, it provides additional information about the paths to that namespace:
# nvme list-subsys /dev/nvme0n1 
nvme-subsys0 - NQN=nqn.1992-08.com.netapp:5700.600a098000d709d6000000005e27796e
\
 +- nvme0 fc traddr=nn-0x200200a098d709d6:pn-0x204200a098d709d6 host_traddr=\
nn-0x200000109b211680:pn-0x100000109b211680 live
 +- nvme1 fc traddr=nn-0x200200a098d709d6:pn-0x204300a098d709d6 host_traddr=\
nn-0x200000109b21167f:pn-0x100000109b21167f live

There are also hooks into the multipath commands to allow you to view your path information for native failover through them as well:
#multipath -ll
Result:
eui.000007e15e903fac00a0980000d663f2 [nvme]:nvme0n1 NVMe,NetApp E-Series,98620002
size=207618048 features='n/a' hwhandler='ANA' wp=rw
|-+- policy='n/a' prio=n/a status=n/a\
| `- 0:10:1    nvme0c10n1    0:0 n/a   n/a   live
`-+- policy='n/a' prio=n/a status=n/a\
  `- 0:32778:1 nvme0c32778n1 0:0 n/a   n/a   live