PCIe slots and PCIe adapters

This topic provides installation rules for PCIe adapters.

Slot configurations

The following lists the PCIe slot configurations for each server view.

*E: empty

Server view	PCIe slots
	Slots 1–3 on riser 1: x16/x8/x8 (Gen 4/5) x16/x16/E (Gen 4/5) E/x16 (Gen 4/5)/x16 (Gen 4) Slots 4–6 on riser 2: x16/x8/x8 (Gen 4/5) x16/x16/E (Gen 4/5) E/x16 (Gen 4/5)/x16 (Gen 4) Slots 7–8 on riser 3: x16/x16 (Gen 4/5) x8/x8 (Gen 4/5)
	Slots 1–3 on riser 1: x16/x8/x8 (Gen 4/5) x16/x16/E (Gen 4/5) E/x16 (Gen 4/5)/x16 (Gen 4) Slots 4–6 on riser 2: x16/x8/x8 (Gen 4/5) x16/x16/E (Gen 4/5) E/x16 (Gen 4/5)/x16 (Gen 4) Slots 7–8 on riser 3: x8 (Gen 5) Slot 9 on riser 4: x8 (Gen 4) Slot 10 on riser 4: x8 (Gen 4/5)
	Slots 1–3 on riser 1: x16/x8/x8 (Gen 4/5) x16/x16/E (Gen 4/5) E/x16 (Gen 4/5)/x16 (Gen 4) Slots 4–6 on riser 2: x16/x8/x8 (Gen 4/5) x16/x16/E (Gen 4/5) E/x16 (Gen 4/5)/x16 (Gen 4)
	Slots 1–3 on riser 1: x16/x8/x8 (Gen 4/5) x16/x16/E (Gen 4/5) E/x16 (Gen 4/5)/x16 (Gen 4) Slot 6 on riser 2: x16 (Gen 4)
	Slot 3 on riser 1: x16 (Gen 4) Slot 6 on riser 2: x16 (Gen 4)
	Slot 11 on riser 5: x16 (Gen 4) Slot 12 on riser 5: x16 (Gen 4)

Note

• The server supports Gen 4 and Gen 5 PCIe riser cards, but not both in the same system.

• Gen 4 riser cards support Gen 4 and Gen 5 PCIe adapters (except Gen 5 retimer cards), but not both in the same system.

• Gen 5 riser cards support Gen 4 and Gen 5 PCIe adapters, but not both in the same system.

• The priority of x8 slots is higher than x16 slots for x8 adapters.

• 7mm drive cage installation rules:

  • For server models with 8 PCIe slots or 10 PCIe slots, a 2FH+7mm SSD drive cage can be installed on slot 3 or slot 6, but not both at the same time.

  • For server models with an 8 x 2.5-inch/2 x 3.5-inch rear drive cage, one of the 7mm drive cages can be installed:

    • 2FH+7mm SSD drive cage: slot 3

    • 7mm SSD drive cage: slot 6

  • For server models with a 4 x 3.5-inch rear drive cage or a GPU installed, a low-profile 7mm drive cage can be installed only on slot 6.

  • For server models with a 4 x 2.5-inch rear drive cage, no 7mm drive cage is supported.

• Serial port module installation rules:

  • For server models with 8 PCIe slots, 10 PCIe slots, or a 4 x 2.5-inch rear drive cage:

    • If both riser 1 and riser 2 use the x16/x16/E riser card and a 7mm drive cage is installed on slot 6, a serial port module can be installed on slot 3. If no 7mm drive cage is installed, a serial port module can be installed on slot 6.

    • If riser 1 uses the x16/x16/E riser card but riser 2 is not installed or not the x16/x16/E riser card, a serial port module can be installed on slot 3 when no 7mm drive cage is installed.

    • If riser 1 does not use the x16/x16/E riser card but riser 2 uses the x16/x16/E riser card, a serial port module can be installed on slot 6 when no 7mm drive cage is installed.

    • If both riser 1 and riser 2 do not use the x16/x16/E riser card, no serial port module is supported.

  • For server models with an 8 x 2.5-inch/2 x 3.5-inch rear drive cage:

    • If riser 1 uses the x16/x16/E riser card, a serial port module can be installed on slot 3 and a 7mm SSD cage can be installed on slot 6.

    • If riser 1 does not use the x16/x16/E riser card, a 7mm drive cage and a serial port module cannot be installed at the same time. If no 7mm drive cage is installed, a serial port module can be installed on slot 6.

  • For server models with a 4 x 3.5-inch rear drive cage, a 7mm drive cage and a serial port module cannot be installed at the same time. If no 7mm drive cage is installed, a serial port module can be installed on slot 6.

  • For server models with a double-wide GPU, the serial port module can be installed only on slot 6 when no 7mm drive cage is installed.

Supported PCIe adapters and slot priorities

The following table lists the recommended slot installation priority for common PCIe adapters.

PCIe adapter	Maximum supported	Suggested slot priority
GPU adapternote
FHFL/HHHL double-wide GPU	3	1 CPU: 2 2 CPUs: 5, 2, 7
HHHL single-wide GPU	8	1 CPU: 1, 2, 3 2 CPUs: 4, 5, 1, 2, 7, 8, 6, 3
PCIe Retimer card
ThinkSystem x16 Gen 4.0 Re-timer adapter (for 4x NVMe)	3	1 CPU: 1, 2, 3 2 CPUs: 1, 2, 4, 5, 3
Internal custom form factor (CFF) RAID/HBA/Expander
5350-8i, 9350-8i, 9350-16i	1	Not installed in PCIe slots. The CFF RAID/HBA/Expander adapter is supported only in the 2.5-inch drive bay chassis, which is installed between the front backplane and the system board assembly.
440-16i, 940-16i
ThinkSystem 48 port 12Gb Internal Expander
Internal standard form factor (SFF) RAID/HBA adapternote
4350-8i, 5350-8i, 440-8i, 540-8i, 940-8i	3	1 CPU: 2, 3, 1 2 CPUs: Without double-wide GPUs: 2, 3, 5, 6, 1, 4 With double-wide GPUs: 2, 3, 6, 5, 1, 4
4350-16i, 440-16i, 540-16i, 940-16i	2
940-32i	1
9350-8i	3	1 CPU: 3, 2, 1 2 CPUs: Without double-wide GPUs: 3, 2, 5, 6, 1, 4 With double-wide GPUs: 3, 2, 6, 5, 1, 4
9350-16i	2
External RAID/HBA adapter
440-8e, 440-16e	8	1 CPU: 2, 3, 1 2 CPUs: When 2FH riser 3 is installed: 5, 2, 6, 3, 7, 8, 4, 1 When rear 4LP riser is installed: 5, 2, 6, 3, 8, 10, 4, 1
940-8e	4
FC HBA adapter
All supported FC HBA adapters	10	1 CPU: 2, 3, 1 2 CPUs: 5, 2, 6, 3, 7, 8, 9, 10, 4, 1
NIC adapter
ThinkSystem Intel E810-DA4 10/25GbE SFP28 4-port PCIe Ethernet Adapter ThinkSystem Broadcom 57504 10/25GbE SFP28 4-port PCIe Ethernet Adapter ThinkSystem Broadcom 57508 100GbE QSFP56 2-port PCIe 4 Ethernet Adapter V2 ThinkSystem Mellanox ConnectX-6 Dx 100GbE QSFP56 2-port PCIe Ethernet Adapter	6	1 CPU: 1, 2 2 CPUs: 4, 1, 5, 2, 7, 8
ThinkSystem Broadcom 57416 10GBASE-T 2-port PCIe Ethernet Adapter ThinkSystem Broadcom 57454 10GBASE-T 4-port PCIe Ethernet Adapter ThinkSystem AMD X3522 10/25GbE DSFP28 2-port PCIe Ethernet Adapter	8	1 CPU: 2, 3, 1 2 CPUs: 5, 2, 6, 3, 7, 8, 4, 1
All other supported NIC adapters	10	1 CPU: 2, 3, 1 2 CPUs: 5, 2, 6, 3, 7, 8, 9, 10, 4, 1
InfiniBand adapter
Mellanox ConnectX-6 HDR100 IB/100GbE VPI 1-port x16 PCIe 3.0 HCA w/ Tall Bracket Mellanox ConnectX-6 HDR100 IB/100GbE VPI 2-port x16 PCIe 3.0 HCA w/ Tall Bracket Mellanox ConnectX-6 HDR IB/200GbE Single Port x16 PCIe Adapter w/ Tall Bracket ThinkSystem Nvidia ConnectX-7 NDR200/HDR QSFP112 2-port PCIe Gen5 x16 InfiniBand Adapter ThinkSystem NVIDIA ConnectX-7 NDR400 OSFP 1-port PCIe Gen5 Adapter	6	1 CPU: 1, 2 2 CPUs: 4, 1, 5, 2, 7, 8

Note

1. Rules for GPU adapters:

   • All installed GPU adapters must be identical. For thermal requirement on GPUs, see GPU configurations.

   • The air baffles for GPU adapters vary according to the GPU type. For detailed information, see GPU replacement.

   • If GPU adapters are installed, no middle drive cage, rear drive cage, or PCIe SSD cards are supported.

   • GPU adapters are not supported on the rear 4LP riser.

   • If a double-wide GPU adapter is installed in slot 5, 2, or 7, the adjacent slot 4, 1, or 8 respectively is not available.

2. The installation priority of internal storage controllers is as follows: CFF RAID/HBA > 940 32i > Gen 4 HBA > Gen 4 RAID > Gen 3 HBA > Gen 3 RAID > RAID for tri-mode.

3. Rules for the internal SFF RAID/HBA adapters:

   • RAID 940 series or 9350 series adapters require a RAID flash power module.

   • Mixing of RAID/HBA 4350/5350/9350 adapters (Gen 3) and RAID/HBA 440/540/940 adapters (Gen 4) in the same system is not allowed.

   • RAID/HBA adapters belonged to the same generation (Gen 3 or Gen 4) are allowed to be mixed in the same system.

   • The RAID/HBA 4350/5350/9350 adapters cannot be mixed with Intel E810 Ethernet adapters in the same system.

   • The front riser 5 card supports external RAID/HBA adapters, FC HBA adapters, NIC adapters, and ConnectX-6 IB adapters.

   • The RAID 940-8i or RAID 940-16i adapter supports Tri-mode. When Tri-mode is enabled, the server supports SAS, SATA and U.3 NVMe drives at the same time. NVMe drives are connected via a PCIe x1 link to the controller.

     Note

     To support Tri-mode with U.3 NVMe drives, U.3 x1 mode must be enabled for the selected drive slots on the backplane through the XCC Web GUI. Otherwise, the U.3 NVMe drives cannot be detected. For more information, see U.3 NVMe drive can be detected in NVMe connection, but cannot be detected in Tri-mode.

   • ThinkSystem AMD X3522 10/25GbE DSFP28 2-port PCIe Ethernet Adapter is not supported on 4LP Riser 3/4 and slot 2 and slot 3 of x16/x8/x8 Gen4/Gen5 Riser 1/2.

   • When only one RAID/HBA adapter is installed and the adapter is SFF 9350-8i or 9350-16i, it is recommended to install the adapter in slot 3 for better performance.