Skip to main content

Replacing a Cisco FC switch

You must use Cisco-specific steps to replace a failed Cisco FC switch.

Before you begin

You need the admin password and access to an FTP or SCP server.

About this task

This procedure is nondisruptive and takes approximately two hours to complete.

In the examples in this procedure, FC_switch_A_1 is the healthy switch and FC_switch_B_1 is the impaired switch. The switch port usage in the examples is shown in the following table:

RolePorts
FC-VI connections1, 4
HBA connections2, 3, 5, 6
FC-to-SAS bridge connections7, 8
ISL connections36, 40

The examples show two FC-to-SAS bridges. If you have more, you must disable and subsequently enable the additional ports.

Your switch port usage should follow the recommended assignments.

Port assignments for FC switches

  1. Disable the ISL ports on the healthy switch to fence off the impaired switch.

    These steps are performed on the healthy switch.
    1. Enter configuration mode: conf t
    2. Disable the ISL ports on the healthy switch with the interface and shut commands.

      Example

      FC_switch_A_1# conf t
      FC_switch_A_1(config)# interface fc1/36
      FC_switch_A_1(config)# shut
      FC_switch_A_1(config)# interface fc1/40
      FC_switch_A_1(config)# shut

    3. Exit configuration mode and copy the configuration to the startup configuration.

      Example

      FC_switch_A_1(config)# end
      FC_switch_A_1# copy running-config startup-config
      FC_switch_A_1#
  2. Fence off the FC-VI and HBA ports on the impaired switch (if it is still running).

    These steps are performed on the impaired switch.
    1. Enter configuration mode: conf t
    2. If the impaired switch is still operational, disable the FC-VI and HBA ports on the impaired switch with the interface and shut commands.

      Example

      FC_switch_B_1(config)# interface fc1/1
      FC_switch_B_1(config)# shut
      FC_switch_B_1(config)# interface fc1/4
      FC_switch_B_1(config)# shut
      FC_switch_B_1(config)# interface fc1/2-3
      FC_switch_B_1(config)# shut
      FC_switch_B_1(config)# interface fc1/5-6
      FC_switch_B_1(config)# shut
    3. Exit configuration mode and copy the configuration to the startup configuration.

      Example

      FC_switch_B_1(config)# end
      FC_switch_B_1# copy running-config startup-config
      FC_switch_B_1#
  3. If the impaired switch is still operational, determine the WWN for the switch: show wwn switch

    Example

    FC_switch_B_1# show wwn switch
    Switch WWN is 20:00:54:7f:ee:e3:86:50
    FC_switch_B_1#
  4. Boot and preconfigure the replacement switch, prior to physically installing it.

    At this point the replacement switch is not cabled to the MetroCluster configuration. The ISL ports on the partner switch are disabled (in shut mode) and offline.
    1. Power on the replacement switch and let it boot up.
    2. Check the firmware version on the replacement switch to confirm that it matches the version of the other FC switches: show version
    3. Configure the replacement switch as described in the MetroCluster Installation and Configuration Guide, skipping the Configuring zoning on a Cisco FC switch section.

      Fabric-attached MetroCluster installation and configuration

      You will configure zoning later in this procedure.

    4. Disable the FC-VI, HBA, and storage ports on the replacement switch.

      Example

      FC_switch_B_1# conf t
      FC_switch_B_1(config)# interface fc1/1
      FC_switch_B_1(config)# shut
      FC_switch_B_1(config)# interface fc1/4
      FC_switch_B_1(config)# shut
      FC_switch_B_1(config)# interface fc1/2-3
      FC_switch_B_1(config)# shut
      FC_switch_B_1(config)# interface fc1/5-6
      FC_switch_B_1(config)# shut
      FC_switch_B_1(config)# interface fc1/7-8
      FC_switch_B_1(config)# shut
      FC_switch_B_1# copy running-config startup-config
      FC_switch_B_1#

  5. Physically replace the impaired switch:
    1. Power off the impaired switch.
    2. Power off the replacement switch.
    3. Uncable and remove the impaired switch, carefully noting which cables connected to which ports.
    4. Install the replacement switch in the rack.
    5. Cable the replacement switch exactly as the impaired switch was cabled.
    6. Power on the replacement switch.
  6. Enable the ISL ports on the replacement switch.

    Example

    FC_switch_B_1# conf t
    FC_switch_B_1(config)# interface fc1/36
    FC_switch_B_1(config)# no shut
    FC_switch_B_1(config)# end
    FC_switch_B_1# copy running-config startup-config
    FC_switch_B_1(config)# interface fc1/40
    FC_switch_B_1(config)# no shut
    FC_switch_B_1(config)# end
    FC_switch_B_1#

  7. Verify that the ISL ports on the replacement switch are up: show interface brief
  8. Adjust the zoning on the replacement switch to match the MetroCluster configuration:
    1. Distribute the zoning information from the healthy fabric.

      Example

      In this example, FC_switch_B_1 has been replaced and the zoning information is retrieved from FC_switch_A_1:
      FC_switch_A_1(config-zone)# zoneset distribute full vsan 10
      FC_switch_A_1(config-zone)# zoneset distribute full vsan 20
      FC_switch_A_1(config-zone)# end
    2. On the replacement switch, verify that the zoning information was properly retrieved from the healthy switch: show zone

      Example

      FC_switch_B_1# show zone
      zone name FC-VI_Zone_1_10 vsan 10
        interface fc1/1 swwn 20:00:54:7f:ee:e3:86:50
        interface fc1/4 swwn 20:00:54:7f:ee:e3:86:50
        interface fc1/1 swwn 20:00:54:7f:ee:b8:24:c0
        interface fc1/4 swwn 20:00:54:7f:ee:b8:24:c0

      zone name STOR_Zone_1_20_25A vsan 20
        interface fc1/2 swwn 20:00:54:7f:ee:e3:86:50
        interface fc1/3 swwn 20:00:54:7f:ee:e3:86:50
        interface fc1/5 swwn 20:00:54:7f:ee:e3:86:50
        interface fc1/6 swwn 20:00:54:7f:ee:e3:86:50
        interface fc1/2 swwn 20:00:54:7f:ee:b8:24:c0
        interface fc1/3 swwn 20:00:54:7f:ee:b8:24:c0
        interface fc1/5 swwn 20:00:54:7f:ee:b8:24:c0
        interface fc1/6 swwn 20:00:54:7f:ee:b8:24:c0

      zone name STOR_Zone_1_20_25B vsan 20
        interface fc1/2 swwn 20:00:54:7f:ee:e3:86:50
        interface fc1/3 swwn 20:00:54:7f:ee:e3:86:50
        interface fc1/5 swwn 20:00:54:7f:ee:e3:86:50
        interface fc1/6 swwn 20:00:54:7f:ee:e3:86:50
        interface fc1/2 swwn 20:00:54:7f:ee:b8:24:c0
        interface fc1/3 swwn 20:00:54:7f:ee:b8:24:c0
        interface fc1/5 swwn 20:00:54:7f:ee:b8:24:c0
        interface fc1/6 swwn 20:00:54:7f:ee:b8:24:c0
      FC_switch_B_1#
    3. Find the WWNs of the switches.

      Example

      In this example, the two switch WWNs are as follows:

      • FC_switch_A_1: 20:00:54:7f:ee:b8:24:c0

      • FC_switch_B_1: 20:00:54:7f:ee:c6:80:78

      FC_switch_B_1# show wwn switch
      Switch WWN is 20:00:54:7f:ee:c6:80:78
      FC_switch_B_1#
              
      FC_switch_A_1# show wwn switch
      Switch WWN is 20:00:54:7f:ee:b8:24:c0
      FC_switch_A_1#
    4. Remove zone members that do not belong to the switch WWNs of the two switches.

      Example

      In this example, no member interface in the output shows that the following members are not associated with the switch WWN of either of the switches in the fabric and must be removed:

      • zone name FC-VI_Zone_1_10 vsan 10

        • interface fc1/1 swwn 20:00:54:7f:ee:e3:86:50

        • interface fc1/2 swwn 20:00:54:7f:ee:e3:86:50

      • zone name STOR_Zone_1_20_25A vsan 20

        • interface fc1/5 swwn 20:00:54:7f:ee:e3:86:50

        • interface fc1/8 swwn 20:00:54:7f:ee:e3:86:50

        • interface fc1/9 swwn 20:00:54:7f:ee:e3:86:50

        • interface fc1/10 swwn 20:00:54:7f:ee:e3:86:50

        • interface fc1/11 swwn 20:00:54:7f:ee:e3:86:50

      • zone name STOR_Zone_1_20_25B vsan 20

        • interface fc1/8 swwn 20:00:54:7f:ee:e3:86:50

        • interface fc1/9 swwn 20:00:54:7f:ee:e3:86:50

        • interface fc1/10 swwn 20:00:54:7f:ee:e3:86:50

        • interface fc1/11 swwn 20:00:54:7f:ee:e3:86:50

      The following example shows the removal of these interfaces:
       FC_switch_B_1# conf t
       FC_switch_B_1(config)# zone name FC-VI_Zone_1_10 vsan 10
       FC_switch_B_1(config-zone)# no member interface fc1/1 swwn 20:00:54:7f:ee:e3:86:50
       FC_switch_B_1(config-zone)# no member interface fc1/2 swwn 20:00:54:7f:ee:e3:86:50
       FC_switch_B_1(config-zone)# zone name STOR_Zone_1_20_25A vsan 20
       FC_switch_B_1(config-zone)# no member interface fc1/5 swwn 20:00:54:7f:ee:e3:86:50
       FC_switch_B_1(config-zone)# no member interface fc1/8 swwn 20:00:54:7f:ee:e3:86:50
       FC_switch_B_1(config-zone)# no member interface fc1/9 swwn 20:00:54:7f:ee:e3:86:50
       FC_switch_B_1(config-zone)# no member interface fc1/10 swwn 20:00:54:7f:ee:e3:86:50
       FC_switch_B_1(config-zone)# no member interface fc1/11 swwn 20:00:54:7f:ee:e3:86:50
       FC_switch_B_1(config-zone)# zone name STOR_Zone_1_20_25B vsan 20
       FC_switch_B_1(config-zone)# no member interface fc1/8 swwn 20:00:54:7f:ee:e3:86:50
       FC_switch_B_1(config-zone)# no member interface fc1/9 swwn 20:00:54:7f:ee:e3:86:50
       FC_switch_B_1(config-zone)# no member interface fc1/10 swwn 20:00:54:7f:ee:e3:86:50
       FC_switch_B_1(config-zone)# no member interface fc1/11 swwn 20:00:54:7f:ee:e3:86:50
       FC_switch_B_1(config-zone)# save running-config startup-config
       FC_switch_B_1(config-zone)# zoneset distribute full 10
       FC_switch_B_1(config-zone)# zoneset distribute full 20
       FC_switch_B_1(config-zone)# end
       FC_switch_B_1# copy running-config startup-config

    5. Add the ports of the replacement switch to the zones.

      Example

      All the cabling on the replacement switch must be the same as on the impaired switch:
       FC_switch_B_1# conf t
       FC_switch_B_1(config)# zone name FC-VI_Zone_1_10 vsan 10
       FC_switch_B_1(config-zone)# member interface fc1/1 swwn 20:00:54:7f:ee:c6:80:78
       FC_switch_B_1(config-zone)# member interface fc1/2 swwn 20:00:54:7f:ee:c6:80:78
       FC_switch_B_1(config-zone)# zone name STOR_Zone_1_20_25A vsan 20
       FC_switch_B_1(config-zone)# member interface fc1/5 swwn 20:00:54:7f:ee:c6:80:78
       FC_switch_B_1(config-zone)# member interface fc1/8 swwn 20:00:54:7f:ee:c6:80:78
       FC_switch_B_1(config-zone)# member interface fc1/9 swwn 20:00:54:7f:ee:c6:80:78
       FC_switch_B_1(config-zone)# member interface fc1/10 swwn 20:00:54:7f:ee:c6:80:78
       FC_switch_B_1(config-zone)# member interface fc1/11 swwn 20:00:54:7f:ee:c6:80:78
       FC_switch_B_1(config-zone)# zone name STOR_Zone_1_20_25B vsan 20
       FC_switch_B_1(config-zone)# member interface fc1/8 swwn 20:00:54:7f:ee:c6:80:78
       FC_switch_B_1(config-zone)# member interface fc1/9 swwn 20:00:54:7f:ee:c6:80:78
       FC_switch_B_1(config-zone)# member interface fc1/10 swwn 20:00:54:7f:ee:c6:80:78
       FC_switch_B_1(config-zone)# member interface fc1/11 swwn 20:00:54:7f:ee:c6:80:78
       FC_switch_B_1(config-zone)# save running-config startup-config
       FC_switch_B_1(config-zone)# zoneset distribute full 10
       FC_switch_B_1(config-zone)# zoneset distribute full 20
       FC_switch_B_1(config-zone)# end
       FC_switch_B_1# copy running-config startup-config
    6. Verify that the zoning is properly configured: show zone

      Example

      The following example output shows the three zones:
       FC_switch_B_1# show zone
         zone name FC-VI_Zone_1_10 vsan 10
           interface fc1/1 swwn 20:00:54:7f:ee:c6:80:78
           interface fc1/2 swwn 20:00:54:7f:ee:c6:80:78
           interface fc1/1 swwn 20:00:54:7f:ee:b8:24:c0
           interface fc1/2 swwn 20:00:54:7f:ee:b8:24:c0
       
         zone name STOR_Zone_1_20_25A vsan 20
           interface fc1/5 swwn 20:00:54:7f:ee:c6:80:78
           interface fc1/8 swwn 20:00:54:7f:ee:c6:80:78
           interface fc1/9 swwn 20:00:54:7f:ee:c6:80:78
           interface fc1/10 swwn 20:00:54:7f:ee:c6:80:78
           interface fc1/11 swwn 20:00:54:7f:ee:c6:80:78
           interface fc1/8 swwn 20:00:54:7f:ee:b8:24:c0
           interface fc1/9 swwn 20:00:54:7f:ee:b8:24:c0
           interface fc1/10 swwn 20:00:54:7f:ee:b8:24:c0
           interface fc1/11 swwn 20:00:54:7f:ee:b8:24:c0

         zone name STOR_Zone_1_20_25B vsan 20
           interface fc1/8 swwn 20:00:54:7f:ee:c6:80:78
           interface fc1/9 swwn 20:00:54:7f:ee:c6:80:78
           interface fc1/10 swwn 20:00:54:7f:ee:c6:80:78
           interface fc1/11 swwn 20:00:54:7f:ee:c6:80:78
           interface fc1/5 swwn 20:00:54:7f:ee:b8:24:c0
           interface fc1/8 swwn 20:00:54:7f:ee:b8:24:c0
           interface fc1/9 swwn 20:00:54:7f:ee:b8:24:c0
           interface fc1/10 swwn 20:00:54:7f:ee:b8:24:c0
           interface fc1/11 swwn 20:00:54:7f:ee:b8:24:c0
       FC_switch_B_1#
    7. Enable the connectivity to storage and the controllers.

      Example

      The following example shows the port usage:
      FC_switch_A_1# conf t
      FC_switch_A_1(config)# interface fc1/1
      FC_switch_A_1(config)# no shut
      FC_switch_A_1(config)# interface fc1/4
      FC_switch_A_1(config)# shut
      FC_switch_A_1(config)# interface fc1/2-3
      FC_switch_A_1(config)# shut
      FC_switch_A_1(config)# interface fc1/5-6
      FC_switch_A_1(config)# shut
      FC_switch_A_1(config)# interface fc1/7-8
      FC_switch_A_1(config)# shut
      FC_switch_A_1# copy running-config startup-config
      FC_switch_A_1#

  9. Verify the operation of the MetroCluster configuration in ONTAP:
    1. Check whether the system is multipathed: node run -node node-name sysconfig -a
    2. Check for any health alerts on both clusters: system health alert show
    3. Confirm the MetroCluster configuration and that the operational mode is normal: metrocluster show
    4. Perform a MetroCluster check: metrocluster check run
    5. Display the results of the MetroCluster check:metrocluster check show
    6. Check for any health alerts on the switches (if present): storage switch show