Skip to main content

Water leakage and leakage sensor problems

Use this information to resolve issues related to water leakages and leakage sensor.

The N1380 enclosure and the SC750 V4 traySC777 V4 tray design is robust, and is unlikely to leak. If the water is observed outside of the enclosure, make sure the both enclosure and enclosure Power Conversion Stations (PCS) have been disconnected. If no water is observed outside of the enclosure, but there is a suspicion of a water leakage in the enclosure or in one of the eight trays, complete the following steps to determine the source of the leak.
Note

  • A small leakage may not reach either of the leak sensors to trigger a warning. Visual confirmation of a small leakage may be required.

  • The procedures above only cover the Lenovo DWC solution (from the Eaton Ball valves up through the manifold and into the enclosures and trays). If your datacenter Cooling Distribution Unit experiences repeated or frequent low water alerts then you should inspect the data center plumbing between the Cooling Distribution Unit and the Lenovo DWC solution (rack).

This section consists of the following information:

Suspicious leakage symptoms​

The following situations might occur due to leakage problems:

  • Processor over temperature error indicated by the System Error "!" LED being solid ON at the front of the node

  • The entire enclosure shut down unexpectedly (including all nodes, PCS, and SMM3)

  • Enclosure SMM3 management may report the events related to water leakages and leakage sensor, see below for more information.

Possible causes of leakage​

The following are the possible causes of leakage:

  • Leakage at quick connects during installation or removal procedures

  • Leakage in the water loop tubing

Identify the type of leakage with SMM3 messages​

Follow the procedures below to identify the type of leakage with SMM3 messages.

  1. Check the SMM3 messages.

    • When leakage happens, a power-on node will shut down in five seconds, and a power-off node will shut down in one second. In common practices, there is a centralized server management tool that collects the SMM3 messages. Check the management tool for SMM3 messages.

  2. Determine which components to inspect according to the SMM3 messages.

    1. Enclosure leakage messages

      1. Check user action in Messages

      2. Follow detailed procedures in Enclosure leakage problem troubleshooting

        Enclosure leakage SMM3 messages IDSeverityDescriptionInformation
        1A07010027WarningChassis LeakSnsr Other FRU, transition to Non-Critical from OK was asserted.Leakage sensor is abnormal.
        1A07020027CriticalChassis LeakSnsr: Other FRU, transition to Critical from less severe was asserted.Leakage was detected.
        1A07030027CriticalChassis LeakSnsr: Other FRU, transition to Non-recoverable from less severe.Leakage sensor is absent.

    2. Tray leakage messages

      1. Check user action in Messages

      2. Follow detailed procedures in Tray leakage problem troubleshooting

        Tray leakage SMM3 messages IDSeverityDescriptionInformation
        1A07010028 / 1A07010029 / 1A0701002A / 1A0701002B / 1A0701002C / 1A0701002D / 1A0701002E / 1A0701002FWarningTray 1 / Tray 2 / Tray 3 / Tray 4 / Tray 5 / Tray 6 / Tray 7 / Tray 8 : Tray # LeakSnsr : Other FRU, transition to Non-Critical from OK was asserted.Leakage sensor is abnormal.
        1A07020028 / 1A07020029 / 1A0702002A / 1A0702002B / 1A0702002C / 1A0702002D / 1A0702002E / 1A0702002FCriticalTray 1 / Tray 2 / Tray 3 / Tray 4 / Tray 5 / Tray 6 / Tray 7 / Tray 8 : Tray # LeakSnsr : Other FRU, transition to Critical from less severe was asserted.Leakage was detected.
        1A07030028 / 1A07030029 / 1A0703002A / 1A0703002B / 1A0703002C / 1A0703002D / 1A0703002E / 1A0703002FCriticalTray 1 / Tray 2 / Tray 3 / Tray 4 / Tray 5 / Tray 6 / Tray 7 / Tray 8 : Tray # LeakSnsr: Other FRU, transition to Non-recoverable from less severe was asserted.Leakage sensor is absent.

    3. Power Conversion Station (PCS) leakage messages

      • Check user action in Messages

      • Follow detailed procedures in Power Conversion Station (PCS) leakage problem troubleshooting

        PCS leakage SMM3 messages IDSeverityAffected PCSDescriptionInformation
        086F01006CCriticalPCS 1PCS # Leakage : Power Supply, Failure Detected was asserted.Leakage was detected.
        086F01006DCriticalPCS 2
        086F01006ECriticalPCS 3
        086F01006FCriticalPCS 4

Identify the type of leakage with Power Conversion Station (PCS) LEDs behavior​

Follow the procedures below to identify the type of leakage with PCS LEDs.
Table 1. Power Conversion Station (PCS) LEDs behavior for leakage identification
PCS LEDs behaviorIndication

  • Input (AC) power LED: OFF

  • Output (DC) power LED: OFF

  • Error LED: ON

PCS leakage. Leakage occurred in the PCS with the Error LED ON.

To inspect leakage, see

Power Conversion Station (PCS) leakage problem troubleshooting.

  • Input (AC) power LED: ON

  • Output (DC) power LED: OFF

  • Error LED: OFF

Enclosure leakage or tray leakage.

To inspect leakage, see Enclosure leakage problem troubleshooting and Tray leakage problem troubleshooting.

Identify the type of leakage with XCC warning messages​

XCC web interface also shows warning messages related to water leakage or leakage sensor problems. To view XCC warning messages, go to XCC web interface > Events.

XCC web GUI warning messagesIndicationActions

The cooling liquid has leaked to critical state with sensor Chassis Drip

Enclosure leakage.

To inspect water leakage or check leakage sensor, see Enclosure leakage problem troubleshooting.

The cooling liquid has leaked to non-recoverable state with sensor Chassis Drip

Enclosure leakage sensor may be absent or abnormal. Check if the enclosure leakage sensor is installed properly.

The cooling liquid has leaked to critical state with sensor Tray Drip

Tray leakage.

To inspect water leakage or check leakage sensor, see Tray leakage problem troubleshooting.

The cooling liquid has leaked to non-recoverable state with sensor Tray Drip

Tray leakage sensor may be absent or abnormal. Check if the enclosure leakage sensor is installed properly.

Enclosure leakage problem troubleshooting​

Complete the following steps in order until you are able to isolate the cause of the potential leak.

Make preparation for this task:

  1. Make sure to disconnect power cords from all PCS. See Disconnecting power cords from all Power Conversion Stations (PCS).

  2. Make sure you have the following items at hand:

    1. A new enclosure leakage sensor (if leakage was detected)

    2. Stubby screwdriver

    3. An absorbent cloth

  3. Access the enclosure mid-plate. Inspect the manifolds for moisture. See Remove the mid-plate assembly.

    Note
    It is important to visually inspect the bottom of the enclosure with a flashlight prior to re-installing the components into the enclosure.
If enclosure leakage sensor is abnormal or absent:

  1. Check if the enclosure leakage sensor is correctly installed. To reinstall the enclosure leakage sensor, see Leakage sensor replacement.

  2. Use the absorbent cloth to dry the wet components or inside of the enclosure whenever necessary.

  3. Reinstall all components back to the enclosure. Then, connect the power cords to all the enclosure PCS, and power on the enclosure. See:

  4. If you are unable to identify the problem in the steps above, then you may need to contact Product Engineer for the further assistance.

If enclosure leakage is detected:

  1. Remove the trays from the enclosure. Inspect the tray-rear-end quick connects for moisture. See Remove a DWC tray from the enclosure.

  2. If the manifolds or quick connects have leakage problem, discard it. Then, install a new one. See:

  3. Replace the leakage sensor with a new one. See Leakage sensor replacement.

  4. Reinstall all components back to the enclosure. Then, connect the power cords to all the enclosure PCS, and power on the enclosure. See:

  5. If you are unable to identify the problem in the steps above, then you may need to contact Product Engineer for the further assistance.

Tray leakage problem troubleshooting​

Complete the following steps in order until you are able to isolate the cause of the potential leak:

Make preparation for this task:

  1. Make sure to disconnect power cords from all PCS. See Disconnecting power cords from all Power Conversion Stations (PCS).

  2. Make sure you have the following items at hand:

    1. A new tray leakage sensor (if leakage was detected)

    2. An absorbent cloth

  3. SMM3 message has reported a tray with leakage problem. Remove the reported tray from the enclosure. Also, remove the trays on the right and on the left of the reported tray from the enclosure. Remove the tray top cover and inspect the water loop for moisture. See Remove the tray cover.

  4. Use the absorbent cloth to dry the wet components or inside of the tray and enclosure whenever necessary.

    Note
    It is important to visually inspect the bottom of the enclosure with a flashlight prior to re-installing the components into the tray and enclosure.
If tray leakage sensor is abnormal or absent:

  1. Check if the tray leakage sensor is correctly installed. To reinstall the tray leakage sensor, see Leakage sensor module replacement.

  2. Reinstall all components back to the enclosure. Then, connect the power cords to all the enclosure PCS, and power on the enclosure. See SC750 V4 hardware repalcement procedures.

  3. If you are unable to identify the problem in the steps above, then you may need to contact Product Engineer for the further assistance.

If tray leakage is detected:

  1. If the water loop has leakage problem, discard it. Then, install a new one. See Water loop replacement.

  2. Replace the tray leakage sensor with a new one. See Leakage sensor module replacement.

  3. Reinstall all components back to the enclosure. Then, connect the power cords to all the enclosure PCS, and power on the enclosure. See SC750 V4 hardware repalcement procedures對的.

  4. If you are unable to identify the problem in the steps above, then you may need to contact Product Engineer for the further assistance.

Power Conversion Station (PCS) leakage problem troubleshooting​

Complete the following steps in order until you are able to isolate the cause of the potential leak:

  1. Make sure to disconnect power cords from all PCS. See Disconnecting power cords from all Power Conversion Stations (PCS).

  2. Make sure you have the following items at hand:

    1. A new Power Conversion Station (PCS)

    2. An absorbent cloth

  3. SMM3 message has reported a PCS with leakage problem. The PCS Leds also indicates which PCS has leakage problem (see Identify the type of leakage with Power Conversion Station (PCS) LEDs behavior). Replace the reported PCS with a new one. See Power Conversion Station (PCS) replacement.

  4. Use the absorbent cloth to dry the wet components or inside of the tray whenever necessary.

    Note
    It is important to visually inspect the bottom of the enclosure with a flashlight prior to re-installing the components into the enclosure.

  5. Connect the power cords to all the enclosure PCS, and power on the enclosure. See N1380 hardware repalcement procedures.

  6. If you are unable to identify the problem in the steps above, then you may need to contact Product Engineer for the further assistance.

Disconnecting power cords from all Power Conversion Stations (PCS)​

Before removing components to inspect leakage, it is important to disconnect power cords from all enclosure Power Conversion Stations. Disconnecting power cords for at least two minutes is crucial to avoid PCS latching.

Follow the steps below to disconnect power cords from PCS.

  1. Rotate the power socket latches outwards.

  2. Disconnect the power cable from the PCS.

    Figure 1. Disconnecting the power cord from PCS