Install a rear H100/H200 GPU

Follow instructions in this section to install a rear H100/H200 GPU. The procedure must be executed by a trained technician.

About this task

Attention

• Read Installation Guidelines and Safety inspection checklist to ensure that you work safely.
• Touch the static-protective package that contains the component to any unpainted metal surface on the server; then, remove it from the package and place it on a static-protective surface.
• A torque screwdriver is available for request if you do not have one at hand.

Note

Make sure you have the required tools listed below available to properly replace the component:

• Torx T10 head screwdriver
• Torx T15 head screwdriver
• Phillips #1 head screwdriver
• Phillips #2 head screwdriver
• Flat head screwdriver
• Alcohol cleaning pad
• H100/H200 PCM Kit
• SR780a V3 water loop putty pad kit
• SR780a V3 water loop service kit
• H100/H200 GPU service fixture kit

Important

Putty pad/phase change material (PCM) replacement guidelines

• Before replacing the putty pad/PCM, gently clean the hardware surface with an alcohol cleaning pad.
• Hold the putty pad/PCM carefully to avoid deformation. Make sure no screw hole or opening is blocked by the putty pad/PCM.
• Do not use expired putty pad/PCM. Check the expiry date on putty pad/PCM package. If the putty pads/PCM are expired, acquire new ones to properly replace them.

The following illustration shows the GPU numbering and corresponding slot numbering in XCC.

Figure 1. GPU numbering

Physical GPU socket	Slot numbering in XCC	Logical number in nvidia-smi
GPU 1	Slot 21	1
GPU 2	Slot 24	2
GPU 3	Slot 22	0
GPU 4	Slot 23	3
GPU 5	Slot 17	5
GPU 6	Slot 20	6
GPU 7	Slot 18	4
GPU 8	Slot 19	7

Procedure

1. (Optional) For new GPU, remove the connector covers at the bottom.
   Figure 2. Removing connector covers
   
   
2. Gently place the GPU down on the GPU baseboard.
3. Follow the sequence shown in the illustration below to fasten the four Torx T15 screws to secure the GPU to the GPU baseboard.
   Note

   First set the torque screwdriver to 0.1-0.12 newton-meters, 0.9-1.1 inch-pounds to fasten the screws for a few rounds. Then set the torque screwdriver to 0.58-0.62 newton-meters, 5-5.5 inch-pounds to fully fasten the screws.
   Figure 3. Installing the GPU
   
   
4. Replace the Phase Change Material (PCM) on the rear GPU cold plate.
   1. Remove the liner from one side of the pad.
   2. Align the PCM with the marking on the bottom of the cold plate, and place it onto the cold plate; then, apply finger pressure across the entire surface area of the PCM to remove any trapped air and allow 1-2 minutes dwell time until it is firmly attached. Carefully remove the remaining top liner.
      Attention

      • PCM cannot be reused. PCM must be replaced with new ones every time the water loop is removed.

      • After PCM is replaced, there is an expected short duration of throttling before the GPU returns to normal operation. This is due to the PCM requiring a break-in period after being replaced.

      • After replacing the GPU PCM and putty pads, servicers must perform the H100/H200 GPU PCM TIM melting procedure to monitor the GPU until the PCM Thermal Interface Material (TIM) has melted. See PCM TIM Melt Procedure for Neptune Water Cooled Servers with Nvidia H100 H200 GPUs (trained technicians only).

      Figure 4. PCM application

      
      
      
5. Replace the putty pads (x5) on the GPU.
   1. Remove the liner from one side of the pad.
   2. Make sure to align the putty pads to the GPU VR (1) and the markings on GPU; then, place the pads onto the GPU and apply light finger pressure across the entire surface area of the pads to ensure adhesion. Carefully remove the remaining top liner.
      Attention

      • Putty pad cannot be reused. Putty pad must be replaced with new ones every time the water loop is removed.
      • After replacing the GPU PCM and putty pads, servicers must perform the H100/H200 GPU PCM TIM melting procedure to monitor the GPU until the PCM Thermal Interface Material (TIM) has melted. See PCM TIM Melt Procedure for Neptune Water Cooled Servers with Nvidia H100 H200 GPUs (trained technicians only).

      Figure 5. GPU putty pads replacement
      
      

      1 GPU VR (Cover the GPU VR with putty pad)

6. Remove the service bracket and GPU cold plate assembly from the manifold.
   1. Loosen the captive screw that secure the service bracket to the manifold.
   2. Lift the service bracket and GPU cold plate assembly away from the manifold to remove it.
      Figure 6. Removing the service bracket and the GPU cold plate assembly
      
      
7. Place the cold place onto the GPU.
   1. Flip over the service bracket and GPU cold plate assembly; then, gently place the GPU cold plate onto the GPU.
   2. Adjust the cold plate until the two guide pins are seated in the guide holes on the GPU.
      Figure 7. Adjusting the cold plate
      
      
8. Loosen the captive screw to remove the service bracket from the cold plate.
   1. Loosen the captive screw that secure the service bracket to the GPU cold plate.
   2. Lift the service bracket away from the GPU cold plate to remove it.
      Figure 8. Removing the service bracket
      
      
9. Use the H100/H200 GPU service fixture kit to install the cold plate. The following illustration shows the components for the H100/H200 GPU service fixture.
   Figure 9. H100/H200 GPU service fixture components identification
   
   

   Table 1. H100/H200 GPU service fixture

   1 H100/H200 GPU service fixture	2 Handle
   3 Clamp head	4 Spacer
   5 Table for rear GPU cold plate	6 Table for front GPU cold plate

10. Install the H100/H200 GPU service fixture to the rear GPU cold plates. Align the guide slots marked 'Rear' on the fixture with the guide pins on the chassis, then carefully install the fixture into the chassis. Fasten six M3 screws (C3/P2/P3) (PH2, 6 x M3, 0.7 newton-meters, 6 inch-pounds) to secure the fixture to the chassis.
    Figure 10. Installing fixture to the rear GPU cold plates

    
    
    
11. Adjust the position of the handle and clamp head assembly by loosening and tightening the four hex socket screws (4 x M6, 0.7 newton-meters, 6 inch-pounds) with a 5 mm hex bit screwdriver.
    Figure 11. Adjusting handle
    
    
12. After adjusting the handle and clamp head assembly, reattach the four hex socket screws (4 x M6, 0.7 newton-meters, 6 inch-pounds) back to the fixture.
    Figure 12. Reattaching screws
    
    
13. Align the table with the four screws on the cold plate, then carefully position it onto the cold plate with the “FRONT” sign facing the front of the chassis as shown. Before placing the table, ensure that all sensor cables and hoses are moved out of the way.
    Important

    • Ensure that no sensor cables or hoses are pinched in between the table and the cold plate.

    • Ensure that the “FRONT” sign is facing the front side of the chassis.

    Figure 13. Placing the table
    
    
14. Adjust the clamp head.
    1. Loosen the hex nut on the top of the lever.
    2. Loosen the hex nut at the bottom of the lever.
       Figure 14. Loosening the screws on the clamp head
       
       
    3. Push the handle forward to a slightly straightened position. Adjust the clamp head so that it rests at the center of the table as illustrated.
    4. Tighten the hex nut at the bottom of the lever.
    5. Attach the hex nut socket bit to the screwdriver. Hold the clamp head in position with one hand; then, tighten the hex nut on the top of the lever using the screwdriver.
       Figure 15. Adjusting the clamp head
       
       
15. Place the spacer onto the clamp head and hold the spacer while pulling the handle into a fully straightened position.
    Figure 16. Pulling the handle
    
    
16. Attach the T10 extended bit to the torque screwdriver. Follow the screw sequence specified on the fixture and cold plate label, and repeat to fully tighten the sixteen Torx T10 screws with the screwdriver set to the proper torque.
    1. Set the torque screwdriver to 0.4±0.05 newton-meter, 3.5±0.5 pound-inch.
    2. Fasten the screws 720 degrees following the screw installation sequence: → → →
       Note

       Make sure to follow screw installation sequence to prevent GPU cold plate tilting.
    3. Repeat until all the screws on the GPU cold plate are fully tightened.
    Figure 17. Repeat to fully tighten all the screws
    
    

    Figure 18. GPU cold plate screw tightening sequence
    
    

    Figure 19. Installing the GPU cold plates
    
    
17. After fastening the screws with the fixture, remove the table and re-torque the screws. Follow the screw installation sequence → → → to re-torque the screws on the cold plate with a Torx T10 screwdriver set to 0.4±0.05 newton-meter, 3.5±0.5 pound-inch.
    Note

    Remove the table before re-torque.

    Figure 20. Re-torque the cold plate screws
    
    
18. Remove the H100/H200 GPU service fixture kit from the chassis after the cold plate is installed.
19. Reinstall the leakage sensor module cable to the GPU cold plate.
    1. Remove the leakage sensor module cable from the adjacent cable clips.
    2. Route the leakage sensor module cable back onto the GPU cold plate; then, reinstall it in the cable clips on the cold plate.
       Figure 21. Installing the leakage sensor module cable