Set up NVMe over RoCE on the host side

NVMe initiator configuration in an NVMe-RoCE environment includes installing and configuring the rdma-core and nvme-cli packages, configuring initiator IP addresses, and setting up the NVMe-oF layer on the host.

1. Install the rdma and nvme-cli packages:

   # zypper install rdma-core
   # zypper install nvme-cli
   

   RHEL 7

   
   # yum install rdma-core
   # yum install nvme-cli
   

2. Setup IPv4 IP addresses on the ethernet ports used to connect NVMe over RoCE. For each network interface, create a configuration script that contains the different variables for that interface.

   The variables used in this step are based on server hardware and the network environment. The variables include the IPADDR and GATEWAY. These are example instructions for the latest SUSE Linux Enterprise Server 12 service pack:

   Create the example file/etc/sysconfig/network/ifcfg-eth4 as follows:

   BOOTPROTO='static'
      BROADCAST=
      ETHTOOL_OPTIONS=
      IPADDR='192.168.1.87/24'
      GATEWAY='192.168.1.1'
      MTU=
      NAME='MT27800 Family [ConnectX-5]'
      NETWORK=
      REMOTE_IPADDR=
      STARTMODE='auto'
   

   Create the second example file/etc/sysconfig/network/ifcfg-eth5 as follows:

   BOOTPROTO='static'
      BROADCAST=
      ETHTOOL_OPTIONS=
      IPADDR='192.168.2.87/24'
      GATEWAY='192.168.2.1'
      MTU=
      NAME='MT27800 Family [ConnectX-5]'
      NETWORK=
      REMOTE_IPADDR=
      STARTMODE='auto'
   

3. Enable the network interfaces:

    
   # ifup eth4
   # ifup eth5
    
   

4. Set up the NVMe-oF layer on the host.
   1. Create the following file under /etc/modules-load.d/ to load the nvme-rdma kernel module and make sure the kernel module will always be on, even after a reboot:

      # cat /etc/modules-load.d/nvme-rdma.conf 
        nvme-rdma