Skip to main content

Install a CX8 mezzanine module

Use this information to install a CX8 mezzanine module.

About this task

Important
Gap pad, putty pad, and PCM replacement guidelines
  • For gap pad, putty pad, and PCM locations and instruction, see Gap pad, putty pad, and PCM identification and location.

  • Before replacing the gap pad, putty pad, or PCM, gently clean the surface with an alcohol cleaning pad.

  • Hold the gap pad, putty pad, or PCM carefully to avoid deformation. Make sure no screw hole or opening is blocked by the gap pad, putty pad, or PCM material.

Attention
  • Read Installation Guidelines and Safety inspection checklist to ensure that you work safely.

  • Turn off the corresponding DWC tray that you are going to perform the task on.

  • Disconnect all external cables from the enclosure.

  • Use extra force to disconnect QSFP cables if they are connected to the solution.

  • To avoid damaging the water loop, always use the water loop carrier when removing, installing or folding the water loop.

Firmware and driver download: You might need to update the firmware or driver after replacing a component.

Procedure

Note
  • Depending on the model, your solution might look slightly different from the illustration.

  • CX8 mezzanine module and MCIO breakout board are mutually exclusive.

  1. Install the CX8 mezzanine module to the GB200-NVL4 board.
    Figure 1. Installing CX8 mezzanine module

  2. Follow the sequence shown in the illustration below. With a torque screwdriver set to the proper torque, fasten four T15 thumb screws to install the CX8 mezzanine module to the GB200-NVL4 board. The screw installation sequence is > > >
    Note
    Slightly press the CX8 mezzanine module when fastening the thumb screws.


    Figure 2. Fastening CX8 mezzanine module thumb screws

  3. Align the P12V middle bus bars to slots on the P12V front bus bar. Then, slide the P12V middle bus bar into the slots to install it to the tray.
    Attention
    • When installing both the right-side bus bar and the left-side bus bar, make sure to follow the bus bar installation sequence: Left-side bus bar > Right-side bus bar

    • Pay extra attention and care when moving the P12V middle bus bar. Make sure the bus bar does not hit the connectors on the GB200-NVL4 board.
      Figure 3. Prevent P12V middle bus bar hitting the connectors on GB200-NVL4 board

    Figure 4. Installing left-side P12V middle bus bar

    Figure 5. Installing right-side P12V middle bus bar

  4. Install and fasten screws to secure the bus bar according to the table below.
    • 1a Install two (2x) PH1 screws, on right-side bus bar

    • 2a Install one (1x) PH1 screws, on left-side bus bar

      • Install the screws with a torque screwdriver set to 5.0 +/- 0.5 in-lbs, or 0.56 +/- 0.056 N-m.

    • 1b Fasten four (4x) T15 thumb screws, on right-side bus bar

    • 2b Fasten four (4x) T15 thumb screws, on left-side bus bar

      First, install the screws with a torque screwdriver set to 1.5 in-lbs, or 0.169 N-m. Then, set the torque screwdriver to 17.5 in-lbs, or 1.977 N-m, and tighten the screws again.

    Figure 6. Installing screws on the P12V middle bus bars

    Figure 7. Installing screws on the right-side bus bar

    Figure 8. Installing screws on the left-side bus bar

  5. With a torque screwdriver set to the proper torque, install the eight (8x) T30 screws to the P12V front bus bars.
    Note

    For reference, the torque required for the screws to be fully tightened/removed is 15.0 +/- 0.5 in-lbs, or 1.69 +/- 0.056 N-m.

    Figure 9. Installing P12V front bus bar screws

  6. Using an alcohol cleaning pad, gently clean the PCM and putty pads on the GB200-NVL4 board.
    Figure 10. PCM and putty pads on the GB200-NVL4 board

    1 Only for Dual CX8 Mezzanine Module configuration
  7. Clean the PCM and thermal grease from the GPU cold plate using an alcohol cleaning pad. Then, apply new thermal grease to the GPU cold plate.
    1 PCM.
    2 SC777 V4 GB200 Grease SE 7921. Reapply new grease after cleaning off old grease with an alcohol cleaning pad. The SC777 V4 GB200 Grease SE 7921 must be used in this area.
    Figure 11. PCM and thermal grease on the GPU cold plate

    Figure 12. Applying grease to the GPU cold plate
    Note
    Apply a line of 0.2 ml of grease on the cold plate. Make sure to evenly distribute the grease, and the line of grease does not reach the edges of the cold plate. The SC777 V4 GB200 Grease SE 7921 must be used in this area.


  8. Install three PCMs on the GB200-NVL4 boards.
    Note
    Install PCMs to the right-side and left-side GB200-NVL4 boards.
    1. Align the PCM stencil to the guide pins on the GB200-NVL4 board. Then, install the stencil to the board.
    2. Following the stencil cutouts, install three PCM to the GB200-NVL4 board.
    3. Remove the stencil.
    Figure 13. Installing PCM to both GB200-NVL4 boards

  9. Installing putty pads with the plastic jig.
    Note
    Install the putty pads to the right-side and left-side GB200-NVL4 boards.
    1. Keep the pads facing down, and align the plastic jig to the two guide pins on the GB200-NVL4 board. Then, gently place it on the board.
      Figure 14. Placing the plastic jig on the GB200-NVL4 board

    2. Gently pat each pad to help it stick to the board.
    3. Remove the plastic jig. Inspect the plastic jig to ensure that no putty pads remain on it. If any pads remain on the plastic jig, reapply the jig to stick the putty pads to GB200-NVL4 board.
    Figure 15. Sticking the putty pads to the GB200-NVL4 board

  10. (ONLY for Dual CX8 mezzanine module configuration) Install putty pads and PCM to the CX8 mezzanine module.
    Note
    Install the putty pads and PCM to the right-side and left-side CX8 mezzanine modules.
    1. Install two putty pads to the CX8 mezzanine module.
      Figure 16. Installing CX8 mezzanine module putty pads

    2. Install two PCM to the CX8 mezzanine module.
      Figure 17. Installing CX8 mezzanine module PCM

  11. Install the GPU cold plates to the GB200-NVL4 boards.
    1. Loosen two (2x) thumbscrews to release the GPU cold plates from the cross bar.
    2. Align the GPU cold plates to the guide pins; then, flip the GPU cold plate to install it to the GB200-NVL4 board.
    Figure 18. Installing the GPU cold plates to the GB200-NVL4 board

    Figure 19. GPU cold plates guide pins

  12. Place the water loop hoses between the screws as shown in the illustration below.
    Figure 20. Water loop hoses placement

  13. Install the T15 extension bit to the torque screwdriver; then, set the torque screwdriver to the proper torque, and fasten twenty-one (21x) T15 thumbscrews to engage the GPU cold plate to the GB200-NVL4 board. Make sure to follow the sequence shown in the illustration below.
    Note
    • Screw 16, 17, 18, 19, 20, 21 are ONLY for Dual CX8 mezzanine module configuration. Fasten/loosen fifteen (15x) screws for MCIO breakout board.

    • For reference, the torque required for the screws to be fully tightened/removed is 1.0 +/- 4% in-lbs, 0.11 +/- 4% m.

    • Installing the T15 extension bit to the torque screwdriver before installing the screws.
      Figure 21. T15 extension bit

    Figure 22. Engage GPU cold plate to the GB200-NVL4 board

  14. Loosen eight (8x) thumbscrews to release the GPU cold plate handle. Make sure to follow the sequence shown in the illustration below.
    Figure 23. Loosening eight GPU cold plate handle thumb screws

  15. Releasing the retaining thumb screw.
    1. Loosen the retaining thumb screw.
    2. Pull the retaining thumbscrew out of the slot.
    Figure 24. Releasing the retaining thumb screw

  16. Remove the GPU cold plate handles.
    Figure 25. Removing the GPU cold plate handles

  17. With a torque screwdriver set to the proper torque, fasten twenty-one (21x) thumb screws to install the water loop to the GB200-NVL4 board. Make sure to follow the sequence shown in the illustration below.
    Note
    • Screw 16, 17, 18, 19, 20, 21 are ONLY for Dual CX8 mezzanine module configuration. Fasten/loosen fifteen (15x) screws for MCIO breakout board.

    • For reference, the torque required for the screws to be fully tightened/removed is 5.2 +/- 4% in-lbs, 0.58 +/- 4% N-m.

    Figure 26. Installing water loop to the GB200-NVL4 board

  18. Connect the OSFP sideband cable to the CX8 mezzanine module.
    Figure 27. Connecting OSFP sideband cable

  19. Remove the cable holder from the OSFP sideband cable.
    Figure 28. Removing cable holder from the OSFP sideband cable

  20. If necessary, remove the cable connector covers from all the OSFP adapter cables.
    Figure 29. Removing covers from the OSFP adapter cables

  21. Connect the OSFP adapter cables to the CX8 mezzanine module.
    Figure 30. Connecting OSFP cable to the module

  22. Install the cable retainer.
    1. Install the cable retainer to the CX8 mezzanine module.
    2. With a torque screwdriver set to the proper torque, install two screws to secure the cable retainer.
    Note

    For reference, the torque required for the screws to be fully tightened/removed is 5.2 +/- 0.5 in-lbs, or 0.58 +/- 0.056 N-m.

    Figure 31. Installing cable retainer

  23. Return the OSFP adapter cable removal tool.
    1. Place the tool in the in-tray tool box.
    2. Close the cover of the tool box.
    Figure 32. Restoring OSFP adapter cable removal tool

  24. Return the OSFP adapter cable connector covers.
    1. Place the covers in the in-tray tool box.
    2. Close the cover of the tool box.
    Figure 33. Restoring OSFP adapter cable connector cover

  25. Return the OSFP sideband cable holder.
    1. Place the holder in the in-tray tool box.
    2. Close the cover of the tool box.
    Figure 34. Restoring OSFP sideband cable holder

After you finish
  1. Connect cables to the GB200-NVL4 board. See Internal cable routing.

  2. Install the cross bar. See Install the cross bar.

  3. Install the tray cover. See Install the tray cover.

  4. Install the tray into the enclosure. See Install a tray in the enclosure.

  5. Connect all required external cables to the solution.
    Note
    Use extra force to connect QSFP cables to the solution.
  6. Check the power LED on each node to make sure it changes from fast blink to slow blink to indicate all nodes are ready to be powered on.