Skip to main content

PCIe slots and PCIe adapters

This topic provides installation rules for PCIe adapters.

Slot configurations without DWCM​

The following table lists the PCIe slot configurations for each server view without a Direct Water Cooling Module (DWCM).

*E: empty

Server viewPCIe slots


Slots 1–3 on riser 1:

  • x16/x8/x8 (Gen 4/5)

  • x16/x16/E (Gen 4/5)

  • E/x16 (Gen 4/5)/x16 (Gen 4)

Slots 4–6 on riser 2:

  • x16/x8/x8 (Gen 4/5)

  • x16/x16/E (Gen 4/5)

  • E/x16 (Gen 4/5)/x16 (Gen 4)

Slots 7–8 on riser 3:

  • x16/x16 (Gen 4/5)

  • x8/x8 (Gen 4/5)



Slots 1–3 on riser 1:

  • x16/x8/x8 (Gen 4/5)

  • x16/x16/E (Gen 4/5)

  • E/x16 (Gen 4/5)/x16 (Gen 4)

Slots 4–6 on riser 2:

  • x16/x8/x8 (Gen 4/5)

  • x16/x16/E (Gen 4/5)

  • E/x16 (Gen 4/5)/x16 (Gen 4)

Slots 7–8 on riser 3:

  • x8 (Gen 5)

Slot 9 on riser 4:

  • x8 (Gen 4)

Slot 10 on riser 4:

  • x8 (Gen 4/5)



Slots 1–3 on riser 1:

  • x16/x8/x8 (Gen 4/5)

  • x16/x16/E (Gen 4/5)

  • E/x16 (Gen 4/5)/x16 (Gen 4)

Slots 4–6 on riser 2:

  • x16/x8/x8 (Gen 4/5)

  • x16/x16/E (Gen 4/5)

  • E/x16 (Gen 4/5)/x16 (Gen 4)





Slots 1–3 on riser 1:

  • x16/x8/x8 (Gen 4/5)

  • x16/x16/E (Gen 4/5)

  • E/x16 (Gen 4/5)/x16 (Gen 4)

Slot 6 on riser 2:

  • x16 (Gen 4)



Slot 3 on riser 1:

  • x16 (Gen 4)

Slot 6 on riser 2:

  • x16 (Gen 4)



Slot 11 on riser 5:

  • x16 (Gen 4/5)

Slot 12 on riser 5:

  • x16 (Gen 4/5)

Slot configurations with DWCM​

The following table lists the PCIe slot configurations for each server view with a DWCM.

*E: empty

Server viewPCIe slots


Slots 1–3 on riser 1:

  • x16/x8/x8 (Gen 4/5)

  • x16/x16/E (Gen 4/5)

  • E/x16 (Gen 4/5)/x16 (Gen 4)

Slots 4–5 on riser 2:

  • x16/x16/E (Gen 4/5)

Slots 7–8 on riser 3:

  • x16/x16 (Gen 4/5)

  • x8/x8 (Gen 4/5)



Slots 1–3 on riser 1:

  • x16/x8/x8 (Gen 4/5)

  • x16/x16/E (Gen 4/5)

  • E/x16 (Gen 4/5)/x16 (Gen 4)

Slots 4–5 on riser 2:

  • x16/x16/E (Gen 4/5)

Slots 7–8 on riser 3:

  • x8 (Gen 5)

Slot 9 on riser 4:

  • x8 (Gen 4)

Slot 10 on riser 4:

  • x8 (Gen 4/5)



Slots 1–3 on riser 1:

  • x16/x8/x8 (Gen 4/5)

  • x16/x16/E (Gen 4/5)

  • E/x16 (Gen 4/5)/x16 (Gen 4)

Slots 4–5 on riser 2:

  • x16/x16/E (Gen 4/5)





Slots 1–3 on riser 1:

  • x16/x8/x8 (Gen 4/5)

  • x16/x16/E (Gen 4/5)

  • E/x16 (Gen 4/5)/x16 (Gen 4)



Slot 3 on riser 1:

  • x16 (Gen 4)

Note

  • Gen 4 riser cards support only Gen 4 PCIe adapters.

  • Gen 5 riser cards support Gen 4 and Gen 5 PCIe adapters.

  • The priority of x8 slots is higher than x16 slots for x8 adapters.

Supported PCIe adapters and slot priorities​

The following table lists the recommended slot installation priority for common PCIe adapters.

PCIe adapterMaximum supportedSuggested slot priority
GPU adapter
FHFL/HHHL DW3

  • 1 CPU: 2

  • 2 CPUs: 5, 2, 7

HHHL SW: L47

  • 1 CPU: 1, 2, 3

  • 2 CPUs: 4, 5, 1, 2, 6, 3, 7

HHHL SW: A28

  • 1 CPU: 1, 2, 3

  • 2 CPUs: 2, 3, 5, 6, 7, 8, 1, 4

DPU adapter
ThinkSystem NVIDIA BlueField-3 VPI QSFP112 2P 200G PCIe Gen5 x162

  • 1 CPU: 1, 2

  • 2 CPUs: 4, 5, 1, 2

PCIe Retimer card

ThinkSystem 4-Port PCIe Gen4 NVMe Retimer Adapter

ThinkSystem PCIe Gen5 NVMe Retimer Adapter

3

  • 1 CPU: 1, 2, 3

  • 2 CPUs: 1, 2, 4, 5, 3

Internal custom form factor (CFF) RAID/HBA/Expander

5350-8i, 9350-8i, 9350-16i

1

Not installed in PCIe slots.

The CFF RAID/HBA/Expander adapter is supported only in the 2.5-inch drive bay chassis, which is installed between the front backplane and the system board assembly.

440-16i, 940-16i

ThinkSystem 48 port 12Gb Internal Expander

Internal standard form factor (SFF) RAID/HBA adapter

4350-8i, 5350-8i, 440-8i, 540-8i, 940-8i

3

  • 1 CPU: 2, 3, 1

  • 2 CPUs:

    • Without double-wide GPUs: 2, 3, 5, 6, 1, 4

    • With double-wide GPUs: 2, 3, 6, 5, 1, 4

4350-16i, 440-16i, 540-16i, 940-16i

2

940-32i, 9450-16i, 5450-16i, 4450-16i, 9450-8i

1
9350-8i3
  • 1 CPU:

    • With other SFF RAID adapters: 2, 3, 1

    • Without other SFF RAID adapters: 3, 2, 1

  • 2 CPUs:

    • When double-wide GPUs are installed:

      • With other SFF RAID adapters: 2, 3, 5, 6, 1, 4

      • Without other SFF RAID adapters: 3, 2, 5, 6, 1, 4

    • When no double-wide GPUs are installed:

      • With other SFF RAID adapters: 2, 3, 6, 5, 1, 4

      • Without other SFF RAID adapters: 3, 2, 6, 5, 1, 4

9350-16i2
External RAID/HBA adapter

440-8e, 440-16e

8

  • 1 CPU: 2, 3, 1

  • 2 CPUs:

    • When 2FH riser 3 is installed: 5, 2, 6, 3, 7, 8, 4, 1

    • When rear 4LP riser is installed: 5, 2, 6, 3, 8, 10, 4, 1

940-8e

4

450W-16e

5

  • 1 CPU: 1, 2

  • 2 CPUs: 4, 1, 5, 2 7

FC HBA adapter
All supported FC HBA adapters10

  • 1 CPU: 2, 3, 1

  • 2 CPUs: 5, 2, 6, 3, 7, 8, 9, 10, 4, 1

NIC adapter

ThinkSystem Intel E810-DA4 10/25GbE SFP28 4-port PCIe Ethernet Adapter

ThinkSystem Broadcom 57504 10/25GbE SFP28 4-port PCIe Ethernet Adapter

ThinkSystem Broadcom 57508 100GbE QSFP56 2-port PCIe 4 Ethernet Adapter V2

ThinkSystem Mellanox ConnectX-6 Dx 100GbE QSFP56 2-port PCIe Ethernet Adapter

6

  • 1 CPU: 1, 2

  • 2 CPUs: 4, 1, 5, 2, 7, 8

ThinkSystem Broadcom 57416 10GBASE-T 2-port PCIe Ethernet Adapter

ThinkSystem Broadcom 57454 10GBASE-T 4-port PCIe Ethernet Adapter

ThinkSystem AMD X3522 10/25GbE DSFP28 2-port PCIe Ethernet Adapter

8

  • 1 CPU: 2, 3, 1

  • 2 CPUs: 5, 2, 6, 3, 7, 8, 4, 1

All other supported NIC adapters10

  • 1 CPU: 2, 3, 1

  • 2 CPUs: 5, 2, 6, 3, 7, 8, 9, 10, 4, 1

InfiniBand adapter

All supported IB adapters

6

  • 1 CPU: 1, 2

  • 2 CPUs: 4, 1, 5, 2, 7, 8

Note

  1. Rules for GPU adapters:

    • All installed GPU adapters must be identical. For thermal requirement on GPUs, see Thermal rules.

    • The air baffles for GPU adapters vary according to the GPU type. For detailed information, see GPU replacement.

    • If GPU adapters are installed, no middle drive cage, rear drive cage, or PCIe SSD cards are supported.

    • GPU adapters are not supported on the rear 4LP riser.

    • If a double-wide GPU adapter is installed in slot 5, 2, or 7, the adjacent slot 4, 1, or 8 respectively is not available.

    • The GPU adapter L4 is supported only on Gen5 riser with re-timer slot.

    • Servers equipped with 400 W GPU adapters do not support ThinkSystem NVIDIA BlueField-3 VPI QSFP112 2P 200G PCIe Gen5 x16.

  2. The installation priority of internal storage controllers is as follows: CFF RAID/HBA > 940 32i > Gen 4 HBA > Gen 4 RAID > Gen 3 HBA > Gen 3 RAID > RAID for tri-mode.

  3. Windows OSs do not support 440 8i/16i HBAs in configurations including the 12 x 3.5-inch SAS/SATA expander backplane and a middle or rear backplane.

  4. Rules for the internal SFF RAID/HBA adapters:

    • RAID 940/9350/9450 series adapters require a RAID flash power module.

    • Mixing of RAID/HBA 4350/5350/9350 adapters (Gen 3) and RAID/HBA 440/540/940 adapters (Gen 4) in the same system is not allowed.

    • RAID/HBA adapters belonged to the same generation (Gen 3 or Gen 4) are allowed to be mixed in the same system.

    • The front riser 5 card supports external RAID/HBA adapters, FC HBA adapters, NIC adapters, and ConnectX-6 IB adapters.

    • The RAID 940-8i or RAID 940-16i adapter supports Tri-mode. When Tri-mode is enabled, the server supports SAS, SATA and U.3 NVMe drives at the same time. NVMe drives are connected via a PCIe x1 link to the controller.

      Note
      To support Tri-mode with U.3 NVMe drives, U.3 x1 mode must be enabled for the selected drive slots on the backplane through the XCC Web GUI. Otherwise, the U.3 NVMe drives cannot be detected. For more information, see U.3 NVMe drive can be detected in NVMe connection, but cannot be detected in Tri-mode.

    • ThinkSystem AMD X3522 10/25GbE DSFP28 2-port PCIe Ethernet Adapter is not supported on 4LP Riser 3/4 and slot 2 and slot 3 of x16/x8/x8 Gen4/Gen5 Riser 1/2.

    • When only one RAID/HBA adapter is installed and the adapter is SFF 9350-8i or 9350-16i, it is recommended to install the adapter in slot 3 for better performance.

  5. The x16 Gen5 retimer card is only supported on ThinkSystem SR650 V3 x16/x16/E PCIe Gen5 Riser1&2.

  6. Gen 5 riser cards do not support the following adapters on servers equipped with 9005 series processors:

    • ThinkSystem Broadcom 57414 10/25GbE SFP28 2-port PCIe Ethernet Adapter V2

    • ThinkSystem Broadcom 57416 10GBASE-T 2-port PCIe Ethernet Adapter

    • ThinkSystem QLogic QLE2772 32Gb 2-Port PCIe Fibre Channel Adapter

  7. ThinkSystem Broadcom 57412 10GBASE-T 4-port PCIe Ethernet Adapter is supported only on Gen 4 riser cards.

  8. ThinkSystem Broadcom 57412 10GBASE-T 4-port OCP Ethernet Adapter is not supported when the OCP slot is located at the front of the server.