Install the rear B200 GPU cold plate module

Follow instructions in this section to install the rear B200 GPU cold plate module. The procedure must be executed by a trained technician.

About this task

Attention

• Read Installation Guidelines and Safety inspection checklist to ensure that you work safely.
• Touch the static-protective package that contains the component to any unpainted metal surface on the server; then, remove it from the package and place it on a static-protective surface.
• A torque screwdriver is available for request if you do not have one at hand.

Note

Make sure you have the required tools listed below available to properly replace the component:

• Torx T15 head screwdriver
• Torx T15 200mm extension bit
• Phillips #1 head screwdriver
• Phillips #2 head screwdriver
• Alcohol cleaning pad
• B200 PCM
• B200 SXM6 PAD-1
• B200 SXM6 PAD-2
• B200 GPU F&R Shipping bkt Kit

B200 (GPU & Retimer NVSwitch) (service & shipping bkt) Kit are reusable and mandatory when servicing GPUs and GPU cold plate modules. It is recommended to keep them at the facility where the server operates for future replacement needs.

Important

Putty pad/phase change material (PCM) replacement guidelines

• Before replacing the putty pad/PCM, gently clean the hardware surface with an alcohol cleaning pad.
• Hold the putty pad/PCM carefully to avoid deformation. Make sure no screw hole or opening is blocked by the putty pad/PCM.
• Do not use expired putty pad/PCM. Check the expiry date on putty pad/PCM package. If the putty pads/PCM are expired, acquire new ones to properly replace them.

The following illustration shows the B200 GPU numbering and corresponding slot numbering in XCC.

Figure 1. B200 GPU numbering

Physical GPU socket	Slot numbering in XCC	Logical number in nvidia-smi
GPU 1	Slot 21	4
GPU 2	Slot 24	7
GPU 3	Slot 22	5
GPU 4	Slot 23	6
GPU 5	Slot 17	0
GPU 6	Slot 20	3
GPU 7	Slot 18	1
GPU 8	Slot 19	2

The following illustration shows the components for rear B200 GPU cold plate module.

Figure 2. Rear B200 GPU cold plate module components identification

Table 1. Rear B200 GPU cold plate module components

1 Manifold	2 Hose tie
3 Leakage sensor module	4 Shipping bracket
5 GPU cold plate	6 GPU slot number label
7 GPU cold plate screw torque label

Procedure

1. Make sure the GPU complex is installed in the chassis.
2. Replace the Phase Change Material (PCM) on the front GPU cold plate module.
   1. Ensure the shipping bracket is attached to the GPU cold plate module. Flip over the module and place it on a surface with the cold plate facing upward.
   2. Apply the PCM jig to the GPU cold plate.
   3. Remove the liner from one side of the pad. Align the PCM with the jig and place it onto the cold plate. Remove the jig; then, apply finger pressure across the entire surface area of the PCM to remove any trapped air and allow 1-2 minutes dwell time until it is firmly attached. Carefully remove the remaining top liner.
   4. Repeat to replace the PCM on the four cold plates.
      Attention

      • PCM cannot be reused. PCM must be replaced with new ones every time the water loop is removed.

      Figure 3. PCM application
      
      
3. Replace the putty pads (x10) on the GPU.
   1. Follow the B200 GPU application instructions to apply the putty pads.
      Note

      • Apply the putty pads from B200 SXM6 PAD-1 to the six locations marked with number 1
      • Apply the putty pads from B200 SXM6 PAD-1 to the two locations marked with number 2
      • Apply the putty pads from B200 SXM6 PAD-2 to the two locations on the GPU VR marked with number 3 and 4 (gray color)

      Figure 4. GPU putty pads instructions
      
      
   2. Remove the liner from one side of the pad.
   3. Make sure to align the two gray colored putty pads to the GPU VR (1) and the markings; then, place the pads to cover the GPU VR as illustrated and apply light finger pressure across the entire surface area of the pads to ensure adhesion. Carefully remove the remaining top liner.
   4. Align the putty pads to the markings on the GPU; then, place the pads onto the GPU and apply light finger pressure across the entire surface area of the pads to ensure adhesion. Carefully remove the remaining top liner.
   5. Repeat to replace all putty pads on the four GPUs.
      Attention

      • Putty pad cannot be reused. Putty pad must be replaced with new ones every time the water loop is removed.

      Figure 5. GPU putty pads replacement
      
      

      1 GPU VR (Cover the GPU VR with putty pad)

4. Install the rear B200 GPU cold plate module.
   1. Hold the rear B200 GPU cold plate module by the shipping brackets; then, align the guide slots on the manifold with the guide pins and gently place the cold plate module onto the four rear GPUs.
   2. Ensure the guide slots on the manifold are securely engaged with the guide pins on the chassis.
      Figure 6. Installing the rear B200 GPU cold plate module
      
      
5. Remove the shipping brackets.
   1. Loosen the twenty captive screws that secure the two shipping brackets to the front B200 GPU cold plate module.
   2. Lift the shipping brackets out of the chassis.
      Figure 7. Removing the shipping brackets
      
      
6. Adjust the cold plate until the two guide pins are seated in the guide holes on the GPU. Repeat to adjust the four cold plates.
   Figure 8. Adjusting the GPU cold plates
   
   
7. Fasten the screws by 360 degrees following the screw installation sequence: → → → , and repeat to fully tighten the sixteen Torx T15 screws with the screwdriver set to the proper torque.
   Note

   • (Except for the brand-new cold plate module) Ensure the TIM breaker screw is loosened to its initial position before tightening the cold plate screws.

   • Loosen the TIM breaker screw to return it to its initial position.
     
     
   • Close the lid. If the lid cannot be closed, the TIM breaker screw needs to be further loosened.
     
     
   1. First set the torque screwdriver to 1.0±0.1 inch-pounds, 0.112±0.0112 newton-meters to fasten the screws for a few rounds. Then set the torque screwdriver to 5.3±0.212 inch-pounds, 0.6±0.024 newton-meters to fully fasten the screws.
      Note

      • Make sure to follow screw sequence to prevent cold plate tilting.
   2. Repeat until all the screws on the four GPU cold plates are fully tightened.
   Figure 9. GPU cold plate screw tightening sequence
   
   
8. The following illustration shows the hose holder location.
   Figure 10. Hose holder location
   
   
9. Place the hoses on the hose guides and the hose holders.
   1. Place the rear B200 GPU cold plate module hoses and cables on the hose guides, and secure them with hose ties. See Fan control board cable routing and Leakage sensor module cable routing.
      Note

      When securing cables on the hose holder, ensure not to route the cables on top of the hoses.

      Important

      • Check the guiding labels on the hoses and hose holders before installation.

      • Ensure to keep the front GPU cold plate module hose to the right of the rear GPU cold plate module hose.

        
        
        

      • Ensure not to cover the joints with the hoses.

        
        
        

      Figure 11. Securing the hoses and cables with hose ties
      
      
   2. Place the left side rear B200 GPU cold plate module hose on (1) hose holder B, and the right side rear B200 GPU cold plate module hose on (2) hose holder B. Ensure the guiding labels on the hoses match with the markings on the hose holders.
      Figure 12. Placing the hoses on hose holders
      
      

      1 Hose holder B (left side)

      2 Hose holder B (right side)

10. Reposition the rear B200 GPU cold plate module manifold as illustrated.
    1. Disengage the manifold from the guide pins marked with A; then, move the manifold to the guide pins marked with B.
    2. Ensure the guide slots on the manifold bracket are securely engaged with the guide pins marked with B.
       Figure 13. Repositioning the rear B200 GPU cold plate module manifold
       
       
11. Fasten the four M3 screws (W7-W8) (PH2, 4 x M3, 0.5 newton-meters, 4.3 inch-pounds) to secure the rear B200 GPU cold plate module manifold to the chassis.
    Figure 14. Installing the rear B200 GPU cold plate module manifold
    
    
12. If you are installing the rear B200 GPU cold plate module after installing a new GPU complex, ensure that the NVSwitch and retimer cold plate module and the front B200 GPU cold plate module are installed before installing the rear fan cage support bracket.
13. Install the rear fan cage support bracket.
    1. Align the rear fan cage support bracket with the corresponding screw holes; then, install the rear fan cage support bracket on top of hose holder B/C as illustrated.
    2. Fasten the four M3 screws (PH2, 4 x M3, 0.5 newton-meters, 4.3 inch-pounds) to secure the rear fan cage support bracket to the fan cage.
    3. Fasten the eight M3 screws (PH2, 8 x M3, 0.5 newton-meters, 4.3 inch-pounds) to secure the rear fan cage support bracket to the chassis.
       Figure 15. Installing the rear fan cage support bracket