Skip to main content

Install the GB200-NVL4 board

Use this information to install the GB200-NVL4 board.

About this task

Required tools
  • Screwdriver for PH 2, T10, and T15 screws

  • GB200 NVL4 NVLink interposer board installation jig

  • GB200 NVL4 NVLink interposer board removal fixture

  • GB200 NVL4 TIM Jig

  • T15 screwdriver

  • T10 bit together with torque wrench

  • GPU cold plate handle assembly

Important
Gap pad, putty pad, and PCM replacement guidelines
  • For gap pad, putty pad, and PCM locations and instruction, see Gap pad, putty pad, and PCM identification and location.

  • Before replacing the gap pad, putty pad, or PCM, gently clean the surface with an alcohol cleaning pad.

  • Hold the gap pad, putty pad, or PCM carefully to avoid deformation. Make sure no screw hole or opening is blocked by the gap pad, putty pad, or PCM material.

Attention
  • Read Installation Guidelines and Safety inspection checklist to ensure that you work safely.

  • Turn off the corresponding DWC tray that you are going to perform the task on.

  • Disconnect all external cables from the enclosure.

  • Use extra force to disconnect QSFP cables if they are connected to the solution.

  • To avoid damaging the water loop, always use the water loop carrier when removing, installing or folding the water loop.

Firmware and driver download: You might need to update the firmware or driver after replacing a component.

Procedure

  1. Using an alcohol cleaning pad, gently clean the PCM and putty pads on the GB200-NVL4 board that was not being replaced.
    Figure 1. PCM and putty pads on the GB200-NVL4 board

    1 Only for Dual CX8 Mezzanine Module configuration
  2. Remove the protective covers from the GB200-NVL4 board.
    Figure 2. Removing cover from the GB200-NVL4 board

  3. Install the handles on the GB200-NVL4 board.
    1. Loosen the two handles that are stored in handle slots in corner of the tray.
    2. Lift up the two handles.
      Figure 3. Lifting the two handles from the tray

    3. Install the two handles to the handle sockets on the GB200-NVL4 board, as shown in the illustration below.
    4. Fasten the handles to secure them to the board.
      Figure 4. Installing the handles to the GB200-NVL4 board

  4. Hold the two handles and move the GB200-NVL4 board over the tray. Align the GB200-NVL4 board with the guide flanges on the tray; then, install it to the tray. See the illustrations below for the positions of right-side and left-side GB200-NVL4 boards.
    Figure 5. Installing the right-side GB200-NVL4 board

    Figure 6. Installing the left-side GB200-NVL4 board to the tray

  5. Remove the two handles.
    1. Loosen the two handles.
    2. Lift up the handles from the GB200-NVL4 board.
      Figure 7. Removing handles from GB200-NVL4 board

  6. Return the handles to handle slots in the tray.
    1. Install the handles to the handle slots in the corner of the tray.
    2. Fasten the handles to secure them to the tray.
    Figure 8. Storing GB200-NVL4 handles to the tray

  7. Install the NVLink interposer board. See Install the NVLink interposer board.
    Note
    Complete Step 4 to Step 14 in Install the NVLink interposer board.
  8. Clean the PCM and thermal grease from the GPU cold plate using an alcohol cleaning pad. Then, apply new thermal grease to the GPU cold plate.
    1 PCM.
    2 SC777 V4 GB200 Grease SE 7921. Reapply new grease after cleaning off old grease with an alcohol cleaning pad. The SC777 V4 GB200 Grease SE 7921 must be used in this area.
    Figure 9. PCM and thermal grease on the GPU cold plate

    Figure 10. Applying grease to the GPU cold plate
    Note
    Apply a line of 0.2 ml of grease on the cold plate. Make sure to evenly distribute the grease, and the line of grease does not reach the edges of the cold plate. The SC777 V4 GB200 Grease SE 7921 must be used in this area.


  9. Install three PCMs on the GB200-NVL4 boards.
    Note
    Install PCMs to the right-side and left-side GB200-NVL4 boards.
    1. Align the PCM stencil to the guide pins on the GB200-NVL4 board. Then, install the stencil to the board.
    2. Following the stencil cutouts, install three PCM to the GB200-NVL4 board.
    3. Remove the stencil.
    Figure 11. Installing PCM to both GB200-NVL4 boards

  10. Installing putty pads with the plastic jig.
    Note
    Install the putty pads to the right-side and left-side GB200-NVL4 boards.
    1. Keep the pads facing down, and align the plastic jig to the two guide pins on the GB200-NVL4 board. Then, gently place it on the board.
      Figure 12. Placing the plastic jig on the GB200-NVL4 board

    2. Gently pat each pad to help it stick to the board.
    3. Remove the plastic jig. Inspect the plastic jig to ensure that no putty pads remain on it. If any pads remain on the plastic jig, reapply the jig to stick the putty pads to GB200-NVL4 board.
    Figure 13. Sticking the putty pads to the GB200-NVL4 board

  11. (ONLY for Dual CX8 mezzanine module configuration) Install putty pads and PCM to the CX8 mezzanine module.
    Note
    Install the putty pads and PCM to the right-side and left-side CX8 mezzanine modules.
    1. Install two putty pads to the CX8 mezzanine module.
      Figure 14. Installing CX8 mezzanine module putty pads

    2. Install two PCM to the CX8 mezzanine module.
      Figure 15. Installing CX8 mezzanine module PCM

  12. Install the GPU cold plates to the GB200-NVL4 boards.
    1. Loosen two (2x) thumbscrews to release the GPU cold plates from the cross bar.
    2. Align the GPU cold plates to the guide pins; then, flip the GPU cold plate to install it to the GB200-NVL4 board.
    Figure 16. Installing the GPU cold plates to the GB200-NVL4 board

    Figure 17. GPU cold plates guide pins

  13. Place the water loop hoses between the screws as shown in the illustration below.
    Figure 18. Water loop hoses placement

  14. Install the T15 extension bit to the torque screwdriver; then, set the torque screwdriver to the proper torque, and fasten twenty-one (21x) T15 thumbscrews to engage the GPU cold plate to the GB200-NVL4 board. Make sure to follow the sequence shown in the illustration below.
    Note
    • Screw 16, 17, 18, 19, 20, 21 are ONLY for Dual CX8 mezzanine module configuration. Fasten/loosen fifteen (15x) screws for MCIO breakout board.

    • For reference, the torque required for the screws to be fully tightened/removed is 1.0 +/- 4% in-lbs, 0.11 +/- 4% m.

    • Installing the T15 extension bit to the torque screwdriver before installing the screws.
      Figure 19. T15 extension bit

    Figure 20. Engage GPU cold plate to the GB200-NVL4 board

  15. Loosen eight (8x) thumbscrews to release the GPU cold plate handle. Make sure to follow the sequence shown in the illustration below.
    Figure 21. Loosening eight GPU cold plate handle thumb screws

  16. Releasing the retaining thumb screw.
    1. Loosen the retaining thumb screw.
    2. Pull the retaining thumbscrew out of the slot.
    Figure 22. Releasing the retaining thumb screw

  17. Remove the GPU cold plate handles.
    Figure 23. Removing the GPU cold plate handles

  18. With a torque screwdriver set to the proper torque, fasten twenty-one (21x) thumb screws to install the water loop to the GB200-NVL4 board. Make sure to follow the sequence shown in the illustration below.
    Note
    • Screw 16, 17, 18, 19, 20, 21 are ONLY for Dual CX8 mezzanine module configuration. Fasten/loosen fifteen (15x) screws for MCIO breakout board.

    • For reference, the torque required for the screws to be fully tightened/removed is 5.2 +/- 4% in-lbs, 0.58 +/- 4% N-m.

    Figure 24. Installing water loop to the GB200-NVL4 board

  19. Install putty pads to the conduction plate as shown in the illustration below. Make sure to use the correct putty pads and conduction plate according to system configuration. Before installing the putty pads, use an alcohol cleaning pad to gently clean off old putty pads from the bottom side of the conduction plate and from GPU cold plate.
    Figure 25. Conduction plate for GPU cold plate
    • 1 Conduction plate for MCIO breakout board configuration
    • 2 Conduction plate for Dual CX8 mezzanine module configuration


  20. Install the conduction plate to the GPU cold plate.
    1. Align the conduction plates to the screw holes on the GPU cold plates. Then, place the conduction plates to the GPU cold plates.
      Figure 26. Aligning conduction plate with screw holes on GPU cold plate (MCIO breakout board configuration)

      Figure 27. Aligning conduction plate with screw holes on GPU cold plate (Dual CX8 mezzanine module configuration)

    2. Follow the sequence shown in the illustrations below to fasten three Torx T10 screws to secure the conduction plate to the GPU cold plate.
      Figure 28. Aligning cold plate with screw hole on water loop (MCIO breakout board configuration)

      Figure 29. Aligning cold plate with screw hole on water loop (Dual CX8 mezzanine module configuration)

  21. Take the covers removed in Step 4, and install them to the defect GB200-NVL4 board.
    Figure 30. Installing protective covers to defect GB200-NVL4 board

After you finish
  1. Install the CX8 mezzanine module or the MCIO breakout board. See Install a CX8 mezzanine module or Install the MCIO breakout board.

  2. Install the E3.S drive cables. See Internal cable routing.

  3. Install the PCIe riser cables. See Internal cable routing.

  4. Follow the sequence below to connect the GB200-NVL4 board cables:

    1. GB200-NVL4 boards cross-link HMC connection cable routing

    2. GB200-NVL4 boards cross-link MCIO connection cable routing

    3. GB200-NVL4 to host board connection cable routing

    4. 12V power connector cable routing

    5. Sideband cable routing

  5. (Left-side GB200-NVL4 board only) Install the HMC module. See Install the HMC module.

  6. Install the NVLink bridge. See Install the NVLink interposer board.

  7. Install the rear wall. See Install the rear wall.

    Complete cable routing for bus bar connector module sideband cable and System management sideband cable.

  8. Install the cross bar. See Install the cross bar.

  9. Install the tray cover. See Install the tray cover.

  10. Install the tray into the enclosure. See Install a tray in the enclosure.

  11. Connect all required external cables to the solution.
    Note
    Use extra force to connect QSFP cables to the solution.
  12. Check the power LED on each node to make sure it changes from fast blink to slow blink to indicate all nodes are ready to be powered on.

  13. Update the vital product data (VPD). See Update the Vital Product Data (VPD).

    Machine type number and serial number can be found on the ID label, see Identify the solution and access the Lenovo XClarity Controller.