Partner Mac Address: 00:00:00:00:00:00

While setting up bonding on a Linux host running SLES 11, a colleague and I noticed that the output from cat /proc/net/bonding/bond0 showed a “Partner Mac Address:”  of 00:00:00:00:00:00. (BTW, we’ve also observed this symptom on at least one other distro (Debian)).  We were interested in the Partner MAC Address field because we were trying to troubleshoot a connectivity issue and we thought (I believe mistakenly) that this was an indication of some kind of configuration problem on the switch.  It turns out that this symptom is related to a switch configuration parameter and the short answer is: check to make sure that you have set the port-channel on the switch to “mode active”. 

For a more detailed explanation, please read on. 

Bonding overview

Before I dive into the details, a brief description of bonding would probably be helpful.  Bonding (known as NIC Teaming in windows environments) allows for two separate physical interfaces to be treated as a single logical interface.  There are a number of different ways to use bonding, so for the sake of this example, I am going to assume we want to use Active/Active.  More specifically we are going to use the setting “mode=802.3ad miimon=100”.  For more information about bonding, I highly recommend that you review the bonding section of the linux foundation website .

Topology

The approximate physical topology we had configured in the lab is shown below:

The bonding configuration on the host that can be found in /etc/sysconfig/network/ifcfg-bond0 was as follows:

 

And the current status of the bond as found via cat /proc/net/bonding/bond0 was:

 

When we noticed this, we weren’t sure if a Partner Mac Address of all zeros was a problem or not but we were able to ping other devices that could only be accessed via the bond interface.  The question was, were both interfaces up and part of the bond, or not? 

The first way we thought to verify this was to use tcpdump –i bond0 and compare the traffic on the newly configured bond with a known working bond on another host.  When we did this, we noticed there were lots of LACP frames being exchanged between the host and the switch in the known working configuration and there weren’t any LACP frames being exchanged between the host and the switches we were having problems with. 

Next, we reviewed the configurations from both of the Cisco switches in our topology and verified that they were the same:

 

Next, we reviewed the mac address table on both of the Nexus switches using show mac address-table.

 

Since the same host MAC Address was shown in both tables, we came to the conclusion that the bond was working properly. However, we were still curious about the Partner MAC Address value, so on a hunch we changed the switch interface configuration so that both members would be using mode active

and the Partner MAC Address was now properly populated 

To get to the bottom of where the Partner MAC Address value was coming from, we took an xgig trace of the bond initializing both with and without "mode active".  After reviewing the trace, it appears that server pulls the Partner MAC Address out of the LACP frames from the switch, see below

In the case where the Partner MAC Address is set to zero, the port-channel is set to “mode on” and the switch doesn’t transmit any LACP frames.  As a result the server is unable to populate this field. 

I don’t believe there is a problem either way but at least now you’ll have a basic idea of why the Partner MAC Address is zero if you’re ever wondering.

Thanks for reading!

Comments

[Waldemar Pera]

By default, channel-group mode is On without LACP. If you enable globally LACP and set channel-group to Active, everything is ok. (http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/configuration/guide/cli/EtherChannel.html).

Another workaround, it's off LACP on Linux side.

[Erik Smith]

Hi Waldemar, that link is a great reference and I probably should have included a couple of switch configuration references in my originl post.

However, the point of the blog post was to explain why the Partner MAC Address was zero and (to me at least) the link between LACP and this field was not immediately apparent. Based on the feedback I've already received, it seems like I may be in the minority here!

Thanks for taking the time to comment!

Erik

[Kevin Joubert]

Is the "Partner MAC Address" sort of a virtual hardware address for the connection pair? I have a strange situation where two systems are showing the same Partner MAC Address on their respective bond0 interfaces. I am thinking that these somehow the 4 ports (2 connections in each bond0 on each server) are portchanneled together.. and they shouldn't be I don't think.

[Erik]

Hi Kevin, I believe the Partner MAC Address is the MAC Address of the switch or switch pair that you are connecting to. If I noticed the same Partner MAC Address on two different servers that were connected to the same switch / switch pair, I wouldn't be concerned...