Month: August 2014

Multihop FCoE to Server in vPC + NPV/NPIV

If you’re like me and had a difficult time at first understanding how to present separate VSAN fabrics to a server in a single vPC, I hope this post helps you out. I’ll be covering this exact scenario, as well as multihop FCoE with Nexus 5K switches in two topologies – Without NPV/NPIV and with NPV/NPIV.

You can run ethernet and FCoE from a server in a vPC connected to two switches. The ethernet portion of the CNA will look like a port-channel, and the fibre channel portion will look like separate connection to separate fabrics. In this topology, we have 1 northbound storage device, 2 north switches, 2 south switches, and an server connected with CNAs. Our end goal is to present the left-side A fabric of our storage device to the server on CNA1 via VSAN 101, and present the right-side B fabric of our storage device to the server on CNA2 via VSAN 102. The server should be in a vPC to both south switches, using VLAN 10 for Data, and separate VSANs for each CNA. All switches should participate in the fabric (no NPV or NPIV).

 Our topology looks like this:




Fibre Channel over IP (FCIP) for CCIE DC

Fibre Channel over IP (FCIP) is a tunneling protocol used to connect FC networks across IP networks, such as a WAN. It uses TCP with the DF bit set. Being that this is IP storage, it is only supported on the MDS platform. The basic configuration is straight forward, but be aware that there are lots of configurable tweaks.  In this blog post I’ll be going through the configuration of several FCIP topologies, feel free to follow along.  At the end I’ll post a quick template.

Reference (This document is quite excellent):

Below is the topology we’re looking at.  We have a server in Data Center 1 that needs to attach to JBOD storage in Data Center 2 over the IP network.


To accomplish this, we’ll create an FCIP tunnel between MDS1 in Data Center 1 and MDS2 in Data Center 2. (more…)

FC Security for CCIE DC – Fabric Binding

Fabric binding ensures that switches configured in the fabric binding database are permitted to connect to the switch. If a switch tries to join the fabric, and that switch is not in the fabric binding database, access is denied.  Fabric binding is configured on a per-VSAN basis.

From Cisco, “This feature helps prevent unauthorized switches from joining the fabric or disrupting current fabric operations. It uses the Exchange Fabric Membership Data (EFMD) protocol to ensure that the list of authorized switches is identical in all switches in the fabric.”

This is very similar to Port Security, except Fabric Binding is for switches only (not devices). Switches bind to the fabric instead of interfaces like in Port Security. Additionally, Fabric Binding is manually configured on each switch, it cannot be distributed through CFS.

More information can be found here:

Steps involved:

1. Enable the fabric binding feature
2. Configure a list of sWWNs and their corresponding domain IDs for devices permitted in the fabric
3. Active the fabric binding database
4. Copy the fabric binding database to the fabric binding config database


FC Security for CCIE DC – FC Port Security

Fibre Channel port security prevents unauthorized Fibre Channel devices and switches from logging into the fabric. This protects the fabric from accidents, malicious intent or attacks such as WWN identity spoofing. It’s configured on a per-VSAN basis.  

Everything covered here can be found in this configuration guide:

You have a few options to choose from when configuring Port Security:

1. Configure with auto-learning and CFS distribution
2. Configure with auto-learning without CFS distribution
3. Configure with manual database

The first method is definitely most practical, as you can configure once, learn the current environment, and use Cisco Fabric Services (CFS) to distribute throughout the fabric. I’ll be following this method in this blog post, feel free to follow along.  Also added a quick template at the bottom.


FC Security for CCIE DC – FC-SP / DHCHAP

Fibre Channel Security Protocol (FC-SP) provides the capabilities for Diffie-Hellman Challenge Handshake Authentication Protocol (DHCHAP) to authenticate switches and/or hosts attempting to enter the fabric. The terms FC-SP and DHCHAP are used interchangeably. Unlike most FC feature, DHCHAP is not configured on a per-VSAN basis.

All things in this post can be found in the configuration guide:

Steps involved to configure FC-SP:

2. (Optional) Configure the hash algorithm and Diffie-Hellamn groups
3. Configure the DHCHAP password for the local switch
4. Configure the DHCHAP password for the remote switches/devices in the fabric
5. Configure and enable DHCHAP on interfaces
_a. Modes
_b. Reauthentication
6. Verify



Fibre Channel (FC) Basics for CCIE DC

When first looking at the blueprint for the CCNA/CCNP/CCIE Data Center track, one of my biggest fears was storage. My entire career thus far has been based on traditional IP data networks, not storage networks. I’m used to things like MAC addresses and IP addresses, not WWPNs and FCIDs. This is a completely foreign technology to most Network Engineers. You have to think back, at some point we were young and hopeful CCNAs-to-be, we knew nothing, but that didn’t stop us! Intimidation is over-rated, so throw fear aside and know that persistence always wins.

So you’ve read all about FC, and now you want to see how to configure it. In this blog post I’ll be going through a basic FC configuration, covering some fundamental Fibre Channel topics along the way, such as:

Zoning (Basic and Enhanced)
FC Aliases
Device Aliases
Domain ID Modification
FSPF (with traffic engineering)
SAN Port-channels


Upgrade SAN-OS to NX-OS

I recently purchased a pair of MDS 9216i switches for my CCIE Data Center studies, as they will suit me for the majority of my storage studies (minus FCoE). The MDS’s shipped with old code, SAN-OS 3.0, and I needed them upgraded to at least NX-OS per the blueprint. For a lab with disruptive capabilities, this is super easy to do.

I quickly found these resources, which walked me through the upgrade:

First, a quick show version to see what I’m currently running:


FabricPath for CCIE DC

We’re all cut from the same cloth, or in other words, fabric. It only makes sense that we connect with each other in the most immediate way, with all lines of communication open and inviting. In this blog post I’ll be looking at FabricPath, it’s purpose and how it pertains to the CCIE Data Center lab exam. I’ll also run through a configuration, observing behaviors along the way. For those just looking for a sample config, a full config is provided at the bottom of this post.

This post assumes you already have a basic understanding of FabricPath.  For those looking for details on FabricPath, here are some great resources that helped me along the way.  

Nexus 7000 FabricPath
Cisco FabricPath Best Practices

Cisco Live:
BRKDCT-3313 – FabricPath Operation and Troubleshooting (2014)
BRKDCT-2081 – Cisco FabricPath Technology and Design (2014)


What is FabricPath and why use it?