Install the water loop

Use this information to install the water loop in the SD665-N V3 tray.

Procedure

1. If you are installing a new water loop to the tray, before starting to install the water loop, complete replacing the VR conduction plate first, see Install the VR conduction plate for water loop replacement. If you are reinstalling the water loop after replacing the system board, processor, or power distribution board, skip this step.
2. When removing the water loop from the package box, make sure to hold the touch points marked in grey in the illustration below.
   Attention

   Holding the water loop anywhere other than the touch points may cause damage to it.

   Figure 1. Touching points when removing water loop from package box

   Top image	Isometric view of water loop
   Bottom image	Top view of the water loop

   
   
   
3. If needed, remove plastic grease covers from the underside of processor cold plates.
   Figure 2. Plastic grease covers removal
   
   
4. Fully loosen four captive thumbscrews on the water loop carrier.
   Figure 3. Loosening water loop carrier captive thumbscrews
   
   

Compute node water loop gap pad/putty pad replacement. Make sure to follow Gap pad/putty pad replacement guidelines.

5. Check the both gap pads and putty pads on the bottom side and top side of the compute node water loop.
   1. Replace the putty pads with new ones.
   2. Replace the gap pads with new ones if any of them is damaged or detached.
   Figure 4. Water loop bottom side putty pads (compute node side)

   Attention

   Putty pad cannot be reused. Whenever the water loop is removed, putty pads must be replaced with new ones before reinstalling the water loop.

   
   
   

   Figure 5. Water loop bottom side gap pads (compute node side)
   
   

   Figure 6. Water loop top side gap pads (compute node side)
   
   

GPU node water loop gap pad/putty pad replacement. Make sure to follow Gap pad/putty pad replacement guidelines.

6. Replace the Phase Change Material (PCM) and putty pads on the GPU node water loop with new ones.
   1. Install the PCM jig on the GPU cold plate.
   2. Attach the PCM to the square opening of the jig.
   3. Repeat to replace the PCM of all four GPU cold plates.
      Attention

      • PCM cannot be reused. PCM must be replaced with new ones every time the water loop is removed.

      • After PCM is replaced, there is an expected short duration of throttling before the GPU returns to normal operation. This is due to the PCM requiring a break-in period after being replaced.

      Figure 7. Water loop GPU cold plate PCM replacement (GPU node)
      
      
   4. Replace the putty pads on the GPU node water loop.
      Attention

      Putty pad cannot be reused. Whenever the water loop is removed, putty pads must be replaced with new ones before reinstalling the water loop.

      Figure 8. Water loop putty pads replacement (GPU node)
      
      
   5. Replace the putty pads (x5) on the GPU. Make sure to align the putty pads to the GPU VR (1) and the markings on GPU. Repeat to replace all putty pads on the four GPUs.

      1 GPU VR (Cover the GPU VR with putty pad)

      Attention

      Putty pad cannot be reused. Whenever the water loop is removed, putty pads must be replaced with new ones before reinstalling the water loop.

      Figure 9. GPU putty pads replacement
      
      
7. Install the compute node side of the water loop to the tray.
   1. Carefully hold the water loop with both hands and flip it.
   2. Carefully position the water loop onto the three guide pins, one near the front of the node and two near the rear of the node; then, gently lower down the water loop and ensure it is seated firmly on the system board.
   Attention

   Keep the water loop at an upright position to prevent damaging the system board.

   Figure 10. Water loop carrier installation
   
   
8. Unfold and install the other side of the water loop as shown.
   Figure 11. Unfolding water loop
   
   

Install water loop to the compute node

9. Install processor cold plate screws (x12 Torx T20 screws for the CPU node). Follow the screw sequence specified below and tighten the screws with a torque screwdriver. Fully tighten each screw; then, proceed to the next screw.
   Note

   For reference, the torque required for the screws to be fully tightened/removed is 1.12-1.46 newton-meters, 10-13 inch-pounds

   Figure 12. Processor cold plate installation

   Fully tighten each screw the order below:

   Processor	Screw sequence
   A	2 > 1 > 4 > 6 > 5 > 3
   B	2 > 1 > 5 > 3 > 4 > 6

   
   
   
   
   
10. Loosen water loop carrier screws (12x Phillips #2 screws).
    Note

    Select screw holes marked as R on the rear of the shipping bracket.

    Figure 13. Loosening water loop carrier screws
    
    
11. Remove the water loop carrier from the compute node.
    Figure 14. Water loop carrier removal (Compute node)
    
    
12. Install water loop screws and quick connect screws (14x Torx T10 screws per node) with a torque screwdriver set to the proper torque.
    Note

    • For reference, the torque required for the screws to be fully tightened/removed is 5.0+/- 0.5 lbf-in, 0.55+/- 0.05 N-M.

    • Install the 1 VR cold plate screws (x2) with washers. Use the washers previously removed from the water loop.

    Figure 15. VR cold plate screws with washer
    
    

    Figure 16. Water loop screws and quick connect screws installation

    
    
    
13. Install the five Torx T10 screws to secure the quick connect.
    Note

    For reference, the torque required for the screws to be fully tightened/removed is 5.0+/- 0.5 lbf-in, 0.55+/- 0.05 N-M.

    Figure 17. Quick connect screw installation (Compute node)
    
    
14. Replace the putty pads on the top side and the bottom side of the OSFP module conduction plate.
    Attention

    Putty pad cannot be reused. Whenever a component is removed, putty pads must be replaced with new ones before reinstalling the component.

    Figure 18. OSFP module conduction plate putty pads replacement
    
    

    1 Conduction plate top putty pad 2 Conduction plate bottom putty pad

Make sure to follow Gap pad/putty pad replacement guidelines.

15. Install the OSFP module conduction plate onto the water loop.

    Screw Type	Screwdriver Type
    1 M3x5 screw (x3)	Phillips #1 head screwdriver
    2 M3 screw (x2)	T10 screwdriver

    Figure 19. Installing the OSFP module conduction plate
    
    
16. Install the two Hex screws to the OSFP module with a 4.5 mm hex head screwdriver.
    Figure 20. OSFP module conduction plate Hex screws installation
    
    

Install water loop to the GPU node

17. Loosen water loop carrier screws (x20 Phillips #2 screws).
    Figure 21. Water loop screws and quick connect screws installation (GPU node)
    
    
18. Remove the water loop carrier from the GPU node.
    Figure 22. Water loop carrier removal (GPU node)
    
    
19. Install GPU cold plate screws (x16 T10 screws).
    1. Install GPU cold plate screws following the GPU cold plates installation sequence: GPU 2 > GPU 4 > GPU 1 > GPU 3
       Figure 23. GPU numbering
       
       
    2. Set the torque screwdriver to 1.5 +/- 0.5 lb-In (0.1 +/- 0.06 N-m); then, fasten the GPU cold plate screws.
    3. Set the torque screwdriver to 3.5 +/- 0.5 lb-In (0.4 +/- 0.06 N-m); then, fasten the GPU cold plate screws until all screws are fully tightened.
       Note

       Make sure to follow the screw sequence on the GPU cold plate label: → → →

       Figure 24. GPU cold plate screw installation
       
       
20. Follow the screw installation sequence specified on the network board label, and install network cold plate screws (x8 Torx T10 screws) with a torque screwdriver set to the proper torque.
    Note

    For reference, the torque required for the screws to be fully tightened/removed is 5.0+/- 0.5 lbf-in, 0.55+/- 0.05 N-M.

    Figure 25. Network card screw installation
    
    
21. Install the quick connect screws (x4 Torx T10) with a torque screwdriver set to the proper torque.
    Note

    For reference, the torque required for the screws to be fully tightened/removed is 5.0+/- 0.5 lbf-in, 0.55+/- 0.05 N-M.

    Figure 26. Quick connect screw installation (GPU node)
    
    
22. Install water loop screws and quick connect screws (x13 Torx T10 screws) with a torque screwdriver set to the proper torque.
    Note

    For reference, the torque required for the screws to be fully tightened/removed is 5.0+/- 0.5 lbf-in, 0.55+/- 0.05 N-M.

    Figure 27. Water loop Torx T10 screws installation (GPU node)
    
    
23. Install the Hex screw (x1) and the PH1 screws (x3).
    Note

    For reference, the torque required for the screws to be fully tightened/removed is 5.0+/- 0.5 lbf-in, 0.55+/- 0.05 N-M.

    Figure 28. Water loop Hex and PH1 screws installation (GPU node)
    
    
24. Install the cable tie to the GPU board.
    Figure 29. Installing the cable tie
    
    
25. Connect the carrier board power cable.
    Figure 30. Connecting carrier board power cable
    
    

    Cable	From (carrier board)	To (GPU node power distribution board)
    1 Carrier board power cable	Power and side band connector	Power connector