FCoE login failure when connecting to Nexus 5548

A couple of months ago while we were installing our Nexus 5548s in the E-Lab, we noticed an FCoE login problem.  Although we configured the 5548s in exactly the same manner described for the Nexus 5020 / 5010 in the FCoE TechBook, the FCoE ENodes would not log in.  At first we thought it was a problem with the ENodes, but an xgig trace showed that the 5548 was not transmitting any DCBX or FIP frames.  As a result, I sent an email off to my friends at Cisco that essentially asked “WTFF?” (Where’s The FIP Frames?).  A short time later, Cisco sent a message similar to  the following:

Hi Erik, 

QoS class-fcoe is not enabled by default on 5548 (to conserve buffers).  You should have seen a warning when you enabled feature fcoe.

Switch(config)# feature fcoe

FC license checked out successfully

fc_plugin extracted successfully

FC plugin loaded successfully

FCoE manager enabled successfully

FC enabled on all modules successfully

Warning: Ensure class-fcoe is included in qos policy-maps of all types

To accomplish this, attach these qos policies under system qos

**********************************************

switch (config-sys-qos)# service-policy type qos input

fcoe-default-in-policy

switch (config-sys-qos)# service-policy type queuing input

fcoe-default-in-policy

switch (config-sys-qos)# service-policy type queuing output

fcoe-default-out-policy

switch (config-sys-qos)# service-policy type network-qos

fcoe-default-nq-policy

**********************************************

Please run the above commands on the switch and let us know if that resolves your issue.

This is also documented in the Cisco QOS document.

I ran the commands they provided and the ENodes immediately logged in.  I’ve since discussed the “Warning” message with Cisco and have encouraged them to consider providing a prompt to the user that would ask if they would like to automatically enable the appropriate service policy.

In any case, since the 5548 was added to the January EMC Support Matrix and the 5548 specific information will not be in the EMC FCoE TechBook until February, I wanted to make sure that these new QOS settings were clearly spelled out somewhere by EMC. 

Thanks for reading!

Comments

[zwliy]

I'm gonna share it in Facebook, this is really awesome

[Ernesto Garcia]

HI Erik,
Thanks for this great post.
We have been trying to connect Brocade 1020CNAs to Nexus 5548UP, but we haven’t been able to get them to work.
We have the QoS configuration in place, so this is not the problem in our case. I was wondering, what driver version are you using in your CNAs and what is the software version in your nexus?
The problem seems to be that during the dcbx negotiation between the CNAs and the nexus, they choose “Protocol: CIN” instead of CEE, therefore the FLOGI is failing.
Any help will be greatly appreciated.

Thanks
Ernesto

[Erik Smith]

Hi Ernesto, this sounds like the other interop problem that I found on the Nexus 5596 and is covered at the following: http://brasstacksblog.typepad.com/brass-tacks/2011/06/interoperability-issue-between-brocade-cna-and-cisco-nexus-5596.html

Although I've never actually observed this problem on the 5548, the issue is timing related and as a result it theoretically could happen on the 5548.

[Austinjeremy]

The above commands are still required as of 5.0(3)N2(2a)

This is exactly what I typed:
system qos
service-policy type qos input fcoe-default-in-policy
service-policy type queuing input fcoe-default-in-policy
service-policy type queuing output fcoe-default-out-policy
service-policy type network-qos fcoe-default-nq-policy

Also, this can be helpful:
http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/troubleshooting/guide/n5K_ts_fcoe.html

[Erik Smith]

Hi thanks for the update and the link. Based on what I've heard recently, I don't believe Cisco is planning on changing this anytime soon.

Regards, Erik

[Dat]

Thanks a lot! It really helps me to resolve the issue.