FC/FCoE Connectivity options as of 6/27/2011

About a month ago I received an email from someone asking "When will the FCoE Tech Book be updated to properly reflect EMCs support for native FCoE on our arrays?" 

My initial reaction was "OH!, <golly-gee>" and then the inevitable"Of all the <interesting> pieces of information to <absentmindedly> leave out".   

My actual response was "what format would you like to see the options presented in?"

In response, they provided a link to a blog post on virtualeverything and said “it would be really nice if you could do the same for EMC Storage.” 

After looking at the format of the blog post referenced, I completely agreed and decided to copy it.  I also decided to add Brocade FCoE information as well as a list of configuration rules to hopefully avoid the “why don’t you support x with y” types of questions.  That having been said, if you have any questions, please let me know. 

I intend to update the configuration rules and diagrams on an as needed basis.

Configuration rules

The following configuration rules were used to create the topology diagrams.  If there is a discrepancy between the diagrams and the configuration rules, the configuration rules take precedence:

1. The Nexus 2232 requires a connection to a Nexus 5000 in order to function.
2. The Nexus 2232 does not support native Fibre Channel ports.
3. The Nexus 4001i requires a connection to a Nexus 5000 in order to support FCoE.
4. The Nexus 4001i does not support native Fibre Channel ports.
5. The Nexus 7000 does not support native Fibre Channel ports.
6. The Fibre Channel ports on the Nexus 5000 cannot be used for end device connectivity while the switch is in NPV mode.
7. FC-SW interoperability between Brocade and Cisco is not supported.  When attaching a Brocade switch to a Cisco FCF and vice versa, either NPV mode must be used on the Cisco switch or AG mode must be used on the Brocade switch.
8. The Fibre Channel ports on the Brocade 8000 cannot be used for end device connectivity while the switch is in NPV mode.
9. All host connectivity to the UCS 6100 is via FCoE.
10. The Fibre Channel ports on the UCS 6100 cannot be used for end device connectivity while the switch is in NPV mode.
11. The UCS 6100 does not currently provide an FC zoning interface.  As a result, when the 6100 is running in FC-SW mode, it must be connected to another Cisco switch that is also running in switch mode.
12. Cascading switches that are running in NPV / AG mode is not supported.
13. The Nexus 4001i does not support external VN_Port connections
14. EMC does not currently support FCoE on the Nexus 7000 or MDS.  We expect to release support for this in September.
15. Storage ports cannot be connected to the Nexus 5000 while it is running in NPV mode.
16. The Brocade 8000 and DCX do not support VE_Ports
17. Cisco UCS does not currently support VE_Ports.
18. FCoE NPV uplinks between Cisco and Brocade switches are not supported.
19. VN_Port to VN_Port connections (a host directly connected to storage) without the use of an FCF is not supported with FC-BB-5. 

Supported connectivity options:

The supported connectivity options have been broken down into several categories.  Each category focuses on the connectivity options for a particular connectivity component.  The categories are:

1. Configurations specific to Nexus 5000
2. Configurations specific to Nexus 2000
3. Configurations specific to Nexus 4001i
4. Configurations specific to Nexus 6100
5. Configurations specific to MDS
6. Configurations specific to Brocade 8000 / DCX

Configurations specific to Nexus 5000:

Please note:  The first configuration shown is not specific to the 5000.  It has been included to call your attention to the fact that native FC supports a host being directly connected to storage but FCoE does not.

  

Configurations specific to Nexus 2000:

 

Configurations specific to Nexus 4001i:

 

Configurations specific to 6100:

 

Configurations specific to MDS:

 

Configurations specific to Brocade 8000 / DCX:

 

Thanks for reading!

Comments

[Zappom]

Hi Erik,
excellent and very handy Plattform to see very fast what works.
Chapeau...

[Erik Smith]

Thanks!

[Otto le Roux]

Awesome work, thanks mate. This is going to be very useful.

[Jerry Liu]

Hi, Erik,

Does Brocade DCX support AG mode?

Jerry

[Erik Smith]

Hi Jerry, apologies for the delayed response, somehow I missed your comment.

No they don't support AG mode on the DCX. If you try to do this you will get the following error message.

DCSwitch:admin> ag --modeenable
Access Gateway mode is not supported on this platform
DCSwitch:admin>

[Shanon R Lawton]

Erik, is eLab testing the Nexus 7K and MDS FCoE line cards as well?

[Erik Smith]

Hi Shanon, yes we're in the process of testing FCoE on N7K now. We were due to release support for it in the October ESM but we ran into an issue that prevented us from completing testing by the ESM (EMC Support Matrix) cut-off date.

The problem turned out to be relatively minor and I hope to provide a post next week that will describe the issue.

In November we're planning to add an N7k case study to the FCoE tech book and I'll be updating my "supported FCoE Topologies" blog post at the same time.

[dika]

Hi Erik, nice share. very usefull post.

it could be more helpfull if u would to show me the configuration FCoE at nexus 5000 connect to UCS via Nexus 2000. im confused where i should bind my vfc interface, i already bind at nexus 2000 interface eth101/1/1 but my ucs pwwwn doesnt show at flogi database.

im very thankfull if u response my questions.

regards

[Erik]

Hi Dika, apologies for the delayed response, I received a burst of spam comments about the same time as yours and I didn't notice your comment until just now.

We did put together a UCS specific case study, but I'm not sure it included the Nexus 2k. Take a look at the following and if you don't find what you need let me know. http://www.emc.com/collateral/hardware/technical-documentation/h10524-fcoe-case-studies-tb.pdf

[John Law]

Hi, Erik, does Cisco 4001i support direct attached with native FCoE storage? That means Nexus 5000 series switches did not required between 4001i and FCoE storage.

[Erik]

Hi John, unfortunately no. The 4001i must be connected to a Nexus 5k in order for the FCoE virtual links to be formed.

Regards, Erik