Skip to main content

Required settings on intermediate switches

When sharing ISL traffic in a shared network, the configuration of the intermediate switches provided by the customer must ensure that the MetroCluster traffic (RDMA and storage) meets the required service levels across the entire path between the MetroCluster sites.

The following examples are for Cisco Nexus 3000 switches and IP Broadcom switches. Depending on your switch vendor and models, you must ensure that your intermediate switches have an equivalent configuration.

Cisco Nexus switches​

The following diagram gives an overview of the required settings for a shared network when the external switches are Cisco switches.



In this example, the following policies and maps are created for MetroCluster traffic:

  • A MetroClusterIP_Ingress policy is applied to ports on the intermediate switch that connect to the MetroCluster IP switches.

    The MetroClusterIP_Ingress policy maps the incoming tagged traffic to the appropriate queue on the intermediate switch. Tagging happens on the node-port, not on the ISL. Non-MetroCluster traffic that is using the same ports on the ISL remains in the default queue.

  • A MetroClusterIP_Egress policy is applied to ports on the intermediate switch that connect to ISLs between intermediate switches

You must configure the intermediate switches with matching QoS access-maps, class-maps, and policy-maps along the path between the MetroCluster IP switches. The intermediate switches map RDMA traffic to COS5 and storage traffic to COS4.

The following example shows the configuration for a customer-provided Cisco Nexus 3000 switch. If you have Cisco switches, you can use the example to configure the switch along the path without much difficulty. If you do not have Cisco switches, you must determine and apply the equivalent configuration to your intermediate switches.

The following example shows the class map definitions:

Note
This example is for configurations using Cisco MetroCluster IP switches. You can follow this example regardless of the switch types of the switches carrying MetroCluster traffic that do not connect to a MetroCluster IP switch.
class-map type qos match-all rdma
          match cos 5
        class-map type qos match-all storage
          match cos 4

The following example shows the policy map definitions:

policy-map type qos MetroClusterIP_Ingress
          class rdma
            set dscp 40
            set cos 5
            set qos-group 5
          class storage
            set dscp 32
            set cos 4
            set qos-group 4
        policy-map type queuing MetroClusterIP_Egress
          class type queuing c-out-8q-q7
            priority level 1
          class type queuing c-out-8q-q6
            priority level 2
          class type queuing c-out-8q-q5
            priority level 3
            random-detect threshold burst-optimized ecn
          class type queuing c-out-8q-q4
            priority level 4
            random-detect threshold burst-optimized ecn
          class type queuing c-out-8q-q3
            priority level 5
          class type queuing c-out-8q-q2
            priority level 6
          class type queuing c-out-8q-q1
            priority level 7
          class type queuing c-out-8q-q-default
            bandwidth remaining percent 100
            random-detect threshold burst-optimized ecn

MetroCluster IP Broadcom switches​

The following diagram gives an overview of the required settings for a shared network when the external switches are IP Broadcom switches.



Configurations using MetroCluster IP Broadcom switches require additional configuration:

  • For exterior switches you must configure the access and class maps to classify the traffic on ingress to the customer network.

Note
This is not required on configurations using MetroCluster IP switches.
The following example shows how to configure the access and class maps on the first and last customer switches connecting the ISLs between the MetroCluster IP Broadcom switches.
ip access-list storage
  10 permit tcp any eq 65200 any
  20 permit tcp any any eq 65200
ip access-list rdma
  10 permit tcp any eq 10006 any
  20 permit tcp any any eq 10006

class-map type qos match-all storage
  match access-group name storage
class-map type qos match-all rdma
  match access-group name rdma
  • You need to assign the ingress policy to the ISL switch port on the first customer switch.

The following example shows the class map definitions:

Note
This example is for configurations using Cisco MetroCluster IP switches. You can follow this example regardless of the switch types of the switches carrying MetroCluster traffic that do not connect to a MetroCluster IP switch.
class-map type qos match-all rdma
   match cos 5
class-map type qos match-all storage
   match cos 4

The following example shows the policy map definitions:

policy-map type qos MetroClusterIP_Ingress
   class rdma
      set dscp 40
      set cos 5
      set qos-group 5
   class storage
      set dscp 32
      set cos 4
      set qos-group 4
policy-map type queuing MetroClusterIP_Egress
   class type queuing c-out-8q-q7
      priority level 1
   class type queuing c-out-8q-q6
      priority level 2
   class type queuing c-out-8q-q5
      priority level 3
      random-detect threshold burst-optimized ecn
   class type queuing c-out-8q-q4
      priority level 4
      random-detect threshold burst-optimized ecn
   class type queuing c-out-8q-q3
      priority level 5
   class type queuing c-out-8q-q2
      priority level 6
   class type queuing c-out-8q-q1
      priority level 7
   class type queuing c-out-8q-q-default
      bandwidth remaining percent 100
      random-detect threshold burst-optimized ecn

Intermediate customer switches​

  • For intermediate customer switches, you must assign the egress policy to the ISL switch ports.
  • For all other interior switches along the path that carry MetroCluster traffic, follow the class map and policy map examples in the section Cisco Nexus 3000 switches.