If you’ve ever looked a little more closely at the MAC address used by vmkernel interface 0 (vmk0), you may have noticed that it uses the burned-in address (BIA) of one of your network adapters (usually vmnic0). In fact, this is something that Pantol and I cover in our standard switch chapter of Networking for VMware Administrators (here’s an excerpt).

Recently, a related question popped up in the VMTN forums from a user posting about a duplicate MAC address alarm – you can read about that here. The red herring was finding that vmk0 used the same MAC address as a hardware NIC – but let’s go deeper into why this is both normal and not the cause for duplicate MAC alarms.

By and large, the BIA of your physical NICs are not used by any frames being generated by the ESXi host. Virtual machines have their own virtual NICs that use VMware’s OUI (00:50:56.x.x.x), and so the VM’s own MAC would appear in the source portion of the frame header as it leaves the ESXi host. The same goes for all vmkernel interfaces other than vmk0, which clones the MAC from the first vmnic it finds during installation to avoid using a VMware OUI that might already be in use.

Once vmk0 clones the MAC address, you’re pretty much done with ever seeing a MAC address from your NICs on your upstream switch. A few exceptions: out of band interfaces (iLO, DRAC), IPMI, PXE, and other system connections that are referenced outside of the hypervisor. These tend to have their own MAC addresses in most cases.

MAC Address Tables

Let’s dig deeper. Here’s a look at the four network adapters being used by ESXi host ESX1.

I’ve also verified that vmk0 is using the same MAC address as vmnic0.

I’ll pull up a MAC address table from the switch that ESX1 is using, specifically on VLAN 20 (lab stuff). I’ve highlighted the non-VMware OUI MACs in yellow (ignore the final one on Po8; that’s the VLAN interface on my upstream switch).

Of these six addresses, three of them are being used by vmk0 (one per lab host) and the other three are being used by IPMI.

Did you notice that vmnic0’s MAC address (00:25:90:73:07:ab) appears on the list as being reachable via port gi5? Great – so how do we know if this is being learned via traffic from vmk0 or vmnic0?

Validating The MAC Address Relationship

Answer: Let’s ask LLDP. According to the switch, vmnic0 is plugged into port gi29.

This tells us that the physical NIC is not the source of frames from 00:25:90:73:07:ab, or else the switch would have learned of the path back to gi29 when it saw the address 00:25:90:73:07:ab being sourced on that port.

Let’s double check vmk0 to see if this story matches. I’ll use esxtop to tell me which vmnic is being used by vmk0.

I’ve highlighted the line in blue; vmk0 is currently using vmnic3.

According to LLDP, vmnic3 is plugged into gi5. Port gi5 is also where we saw MAC 00:25:90:73:07:ab. Thus, we’ve proven that vmk0 is the entity using that MAC address, not the physical NIC.

I could also move around the vmk0 interface to different ports to watch the table update, but I think the point is made.

Caveats

A few caveats I had not pondered have come in via Gilles Le Ridou:

@ChrisWahl Duplicate MAC address may occur if reusing the NIC on another server when the first server got a new NIC.

— Gilles Le Ridou (@leridou) December 29, 2014

Definitely be careful when re-using or re-purposing hardware, including “moving the ESXi installation to a new server and then re-use the first server.” Good call! 🙂

CDP, LLDP, and LACP

There’s also been some very educational back-and-forth on the VMware subreddit around using control protocols (CDP/LLDP) and LACP while having a vmk0 using a physical NIC MAC address. These are definitely situations where you could have a conflict, but the assumption is that your control and data protocols are sharing the same layer 2 segment. This is seriously frowned upon.

Let’s do a quick walk through the lab. Below is the switch configuration for Po7, which is an LACP LAG going to an ESXi host using an enhanced LACP configuration. It contains two member ports: gi51 and gi52. The native VLAN is 1 (simply because it’s not specified on the config).

Both NICs show up in the MAC address table on VLAN 1. If the vmk on VLAN 20 happened to be using the same BIA of my NIC, it would not cause a duplicate conflict because they are on different L2 segments.

To demonstrate that this particular switch uses whichever VLAN is native for LLDP, I’ll walk the native VLAN from 1 to 2. The MAC address table has updated the NICs to VLAN 2.

I can also confirm this by checking LLDP on the ESXi side. As a reminder, this particular network is also using enhanced LACP (mode active) with two member ports.

If you’re hitting a conflict on this VLAN with a vmk or VM, I’d advise using a tagged VLAN for data traffic that is not VLAN 1.

Thoughts

Making a long story short, the MAC addresses on your ESXi server’s physical NICs rarely see the light of day. They are almost never learned by the upstream switch. Pretty much zero traffic will use those addresses on an ESXi host. If you’re getting duplicate MAC warnings, it’s likely that someone has cloned your interface address (read Hersey Cartwright’s post on that) or has in some way copied the MAC elsewhere onto a VM or vmkernel interface.

I’ve actually run into instances where an admin has cloned the boot volume of an ESXi host to a number of other ESXi hosts, causing duplicate MAC address warnings to occur. This is because every host had the same MAC address assigned to vmk0, which caused flapping across the various switch ports. This is also referred to as a layer 8 problem. 🙂