Install the rear H100/H200 GPU cold plate module

Follow instructions in this section to install the rear H100/H200 GPU cold plate module. The procedure must be executed by a trained technician.

About this task

Attention

• Read Installation Guidelines and Safety inspection checklist to ensure that you work safely.
• Touch the static-protective package that contains the component to any unpainted metal surface on the server; then, remove it from the package and place it on a static-protective surface.
• A torque screwdriver is available for request if you do not have one at hand.

Note

Make sure you have the required tools listed below available to properly replace the component:

• Torx T10 head screwdriver
• Torx T15 head screwdriver
• Phillips #1 head screwdriver
• Phillips #2 head screwdriver
• Flat head screwdriver
• Alcohol cleaning pad
• H100/H200 PCM Kit
• SR780a V3 water loop putty pad kit
• SR780a V3 water loop service kit
• H100/H200 GPU service fixture kit

Important

Putty pad/phase change material (PCM) replacement guidelines

• Before replacing the putty pad/PCM, gently clean the hardware surface with an alcohol cleaning pad.
• Hold the putty pad/PCM carefully to avoid deformation. Make sure no screw hole or opening is blocked by the putty pad/PCM.
• Do not use expired putty pad/PCM. Check the expiry date on putty pad/PCM package. If the putty pads/PCM are expired, acquire new ones to properly replace them.

The following illustration shows the GPU numbering and corresponding slot numbering in XCC.

Figure 1. GPU numbering

The following illustration shows the components for rear H100/H200 GPU cold plate module.

Figure 2. Rear H100/H200 GPU cold plate module components identification

Table 1. Rear H100/H200 GPU cold plate module components

1 Manifold	2 Hose tie
3 Leakage sensor module	4 Shipping bracket
5 GPU cold plate	6 GPU slot number label
7 GPU cold plate screw torque label

Procedure

1. Make sure the GPU complex is installed in the chassis.
2. Replace the Phase Change Material on the rear H100/H200 GPU cold plate module.
   1. Remove the liner from one side of the pad.
   2. Align the PCM with the marking on the bottom of the cold plate, and place it onto the cold plate; then, apply finger pressure across the entire surface area of the PCM to remove any trapped air and allow 1-2 minutes dwell time until it is firmly attached. Carefully remove the remaining top liner.
   3. Repeat to replace the PCM on the four cold plates.
      Attention

      • PCM cannot be reused. PCM must be replaced with new ones every time the water loop is removed.

      • After PCM is replaced, there is an expected short duration of throttling before the GPU returns to normal operation. This is due to the PCM requiring a break-in period after being replaced.

      • After replacing the GPU PCM and putty pads, servicers must perform the H100/H200 GPU PCM TIM melting procedure to monitor the GPU until the PCM Thermal Interface Material (TIM) has melted. See PCM TIM Melt Procedure for Neptune Water Cooled Servers with Nvidia H100 H200 GPUs (trained technicians only).

      Figure 3. PCM application
      
      
3. Replace the putty pads (x5) on the GPU.
   1. Remove the liner from one side of the pad.
   2. Make sure to align the putty pads to the GPU VR (1) and the markings on GPU; then, place the pads onto the GPU and apply light finger pressure across the entire surface area of the pads to ensure adhesion. Carefully remove the remaining top liner.
   3. Repeat to replace all putty pads on the four GPUs.
      Attention

      • Putty pad cannot be reused. Putty pad must be replaced with new ones every time the water loop is removed.
      • After replacing the GPU PCM and putty pads, servicers must perform the H100/H200 GPU PCM TIM melting procedure to monitor the GPU until the PCM Thermal Interface Material (TIM) has melted. See PCM TIM Melt Procedure for Neptune Water Cooled Servers with Nvidia H100 H200 GPUs (trained technicians only).

      Figure 4. GPU putty pads replacement
      
      

      1 GPU VR (Cover the GPU VR with putty pad)

4. Install the rear H100/H200 GPU cold plate module.
   1. Hold the rear H100/H200 GPU cold plate module by the shipping brackets; then, align the guide slots on the manifold with the guide pins marked with A and gently place the cold plate module onto the four rear GPUs.
   2. Ensure the guide slots on the manifold are securely engaged with the guide pins marked with A on the chassis.
      Figure 5. Installing the rear H100/H200 GPU cold plate module
      
      
5. Loosen the six captive screws that secure the shipping brackets to the rear H100/H200 GPU cold plate module; then, remove the shipping brackets from the rear H100/H200 GPU cold plate module.
   Figure 6. Removing the shipping brackets
   
   
6. Adjust the cold plate until the two guide pins are seated in the guide holes on the GPU. Repeat to adjust the four cold plates.
   Figure 7. Adjusting the GPU cold plates
   
   
7. Use the H100/H200 GPU service fixture kit to install the cold plate. The following illustration shows the components for the H100/H200 GPU service fixture.
   Figure 8. H100/H200 GPU service fixture components identification
   
   

   Table 2. H100/H200 GPU service fixture

   1 H100/H200 GPU service fixture	2 Handle
   3 Clamp head	4 Spacer
   5 Table for rear GPU cold plate	6 Table for front GPU cold plate

8. Install the H100/H200 GPU service fixture to the rear GPU cold plates. Align the guide slots marked 'Rear' on the fixture with the guide pins on the chassis, then carefully install the fixture into the chassis. Fasten six M3 screws (C3/P2/P3) (PH2, 6 x M3, 0.7 newton-meters, 6 inch-pounds) to secure the fixture to the chassis.
   Figure 9. Installing fixture to the rear GPU cold plates

   
   
   
9. Align the table with the four screws on the cold plate, then carefully position it onto the cold plate with the “FRONT” sign facing the front of the chassis as shown. Before placing the table, ensure that all sensor cables and hoses are moved out of the way.
   Important

   • Ensure that no sensor cables or hoses are pinched in between the table and the cold plate.

   • Ensure that the “FRONT” sign is facing the front side of the chassis.

   Figure 10. Placing the table
   
   
10. Adjust the clamp head.
    1. Loosen the hex nut on the top of the lever.
    2. Loosen the hex nut at the bottom of the lever.
       Figure 11. Loosening the screws on the clamp head
       
       
    3. Push the handle forward to a slightly straightened position. Adjust the clamp head so that it rests at the center of the table as illustrated.
    4. Tighten the hex nut at the bottom of the lever.
    5. Attach the hex nut socket bit to the screwdriver. Hold the clamp head in position with one hand; then, tighten the hex nut on the top of the lever using the screwdriver.
       Figure 12. Adjusting the clamp head
       
       
11. Place the spacer onto the clamp head and hold the spacer while pulling the handle into a fully straightened position.
    Figure 13. Pulling the handle
    
    
12. Attach the T10 extended bit to the torque screwdriver. Follow the screw sequence specified on the fixture and cold plate label, and repeat to fully tighten the sixteen Torx T10 screws with the screwdriver set to the proper torque.
    1. Set the torque screwdriver to 0.4±0.05 newton-meter, 3.5±0.5 pound-inch.
    2. Fasten the screws 720 degrees following the screw installation sequence: → → →
       Note

       Make sure to follow screw installation sequence to prevent GPU cold plate tilting.
    3. Repeat until all the screws on the GPU cold plate are fully tightened.
    Figure 14. Repeat to fully tighten all the screws
    
    

    Figure 15. GPU cold plate screw tightening sequence
    
    

    Figure 16. Installing the GPU cold plates
    
    
13. After fastening the screws with the fixture, remove the table and re-torque the screws. Follow the screw installation sequence → → → to re-torque the screws on the cold plate with a Torx T10 screwdriver set to 0.4±0.05 newton-meter, 3.5±0.5 pound-inch.
    Figure 17. Re-torque the cold plate screws
    
    
14. Adjust the position of the handle and clamp head assembly by loosening and tightening the four hex socket screws (4 x M6, 0.7 newton-meters, 6 inch-pounds) with a 5 mm hex bit screwdriver. Repeat from step 10 to step 14 until all four cold plates are installed.
    Figure 18. Adjusting handle
    
    
15. After adjusting the handle and clamp head assembly, reattach the four hex socket screws (4 x M6, 0.7 newton-meters, 6 inch-pounds) back to the fixture.
    Figure 19. Reattaching screws
    
    
16. Remove the H100/H200 GPU service fixture kit from the chassis after all the cold plates are installed.
17. The following illustration shows the hose holder location.
    Figure 20. Hose holder location
    
    
18. Place the hoses on the hose guides and the hose holders.
    1. Place the rear H100/H200 GPU cold plate module hoses and cables on the hose guides, and secure them with hose ties. See Fan control board cable routing and Leakage sensor module cable routing.
       Figure 21. Securing the hoses and cables with hose ties
       
       
    2. Place the left side rear H100/H200 GPU cold plate module hose on (1) hose holder C, and the right side rear H100/H200 GPU cold plate module hose on (2) hose holder B. Ensure the guiding labels on the hoses match with the markings on the hose holders.
       Figure 22. Placing the hoses on hose holders
       
       

       1 Hose holder C (left side)

       2 Hose holder B (right side)

       Important

       • Check the guiding labels on the hoses and hose holders before installation.

19. Reposition the rear H100/H200 GPU cold plate module manifold as illustrated.
    1. Disengage the manifold from the guide pins marked with A; then, move the manifold to the guide pins marked with B.
    2. Ensure the guide slots on the manifold bracket are securely engaged with the guide pins marked with B.
       Figure 23. Repositioning the rear H100/H200 GPU cold plate module manifold
       
       
20. Fasten the four M3 screws (W7-W8) (PH2, 4 x M3, 0.5 newton-meters, 4.3 inch-pounds) to secure the rear H100/H200 GPU cold plate module manifold to the chassis.
    Figure 24. Installing the rear H100/H200 GPU cold plate module manifold
    
    
21. If you are installing the rear H100/H200 GPU cold plate module after installing a new GPU complex, ensure that the NVSwitch cold plate module and the front H100/H200 GPU cold plate module are installed before installing the rear fan cage support bracket.
22. Install the rear fan cage support bracket.
    1. Align the rear fan cage support bracket with the corresponding screw holes; then, install the rear fan cage support bracket on top of hose holder B/C as illustrated.
    2. Fasten the four M3 screws (PH2, 4 x M3, 0.5 newton-meters, 4.3 inch-pounds) to secure the rear fan cage support bracket to the fan cage.
    3. Fasten the eight M3 screws (PH2, 8 x M3, 0.5 newton-meters, 4.3 inch-pounds) to secure the rear fan cage support bracket to the chassis.
       Figure 25. Installing the rear fan cage support bracket