Nested ESXi Host Networking

Table of Contents

Overview

Getting this configured correctly has been a recurring challenge for me. If you search through my blog, you’ll find a few earlier attempts where I tried to sort it out, only to fall back on using a standard vSwitch instead of a distributed one. While the standard vSwitch works, I really prefer the flexibility and features of a distributed virtual switch.

This post pulls together the research and steps I’ve gathered so that I won’t have to struggle with it again in the future. Hopefully, it will also help a few of you who may be running into the same issue.

Research

When an ESXi host boots up (either on a fresh install or after a reset), the vmk0 Management Network interface typically assumes the MAC address of the physical host. This can lead to issues when running multiple hosts or enabling features like DRS, since the physical switch may see duplicate or unexpected MAC activity.

We can verify this behavior from the command line.

First, list the physical NICs on the host. These are the ports where the server’s network cables connect. On Cisco UCS hardware, for example, these MAC addresses are assigned via policy.

esxcli network nic list

Take note of the MAC address on vmnic0.

Next, check the MAC address assigned to the VMkernel NIC:

From this output, you’ll notice that vmk0 is using the same MAC address as the physical NIC.

esxcli network ip interface list

This duplication is the core of the problem. The remainder of this post walks through how to delete and recreate vmk0 so that ESXi generates a unique MAC address for the management interface.

Solution

The goal is to recreate the vmk0 interface so that ESXi assigns it a new, system-generated MAC address. This can’t be done through SSH alone — you’ll need access through the out-of-band management interface (iLO, iDRAC, CIMC, etc.) or directly at the console.

Enable ESXi Shell

First, make sure the ESXi Shell is enabled. This can be done from the ESXi Host Client if it’s available, or from the Direct Console User Interface (DCUI) under Troubleshooting Options.

At the console, press Alt + F1 to bring up a login prompt.

Remove the Current `vmk0` Interface

Delete the existing vmk0 NIC:

esxcli network ip interface remove --interface-name=vmk0

Verify it has been removed:

esxcli network ip interface list

Create a New `vmk0` Interface

Add a new vmk0 NIC and attach it to the Management Network port group:

esxcli network ip interface add --interface-name=vmk0 --portgroup-name="Management Network"

Confirm that the MAC address has changed:

esxcli network ip interface list

If your port groups have been customized, list them to find the correct name:

esxcli network vswitch standard portgroup list

Assign an IPv4 Address

If this host is brand new or recently reset, then the rest of the configuration can take place in the Direct Console User Interface (DCUI). As long as the MAC address of vmk0 is no longer matching the physical address, this will still work. The DCUI can be easier to manage since there is no SSH to script it out, yet.

If you want to remain in the command line, continue along here.

Configure the IP settings for the new interface:

esxcli network ip interface ipv4 set --interface-name=vmk0 --ipv4=<IP_Address> --netmask=<Netmask> --type=static

Verify the address assignment:

esxcli network ip interface ipv4 address list

Verify Management Tag

Ensure that vmk0 is tagged for management traffic:

esxcli network ip interface tag get --interface-name=vmk0

Configure a Default Gateway

Set the default gateway:

esxcli network ip route ipv4 add --gateway=172.16.2.254 --network=default

Test Connectivity

At this point, the host should be reachable again. Test by:

ping <Gateway_IP_Address>
ping <vCenter_IP_or_Management_Station>

Finally, open the ESXi Host Client in a web browser to confirm access.

Conclusion

This configuration ensures the host has a unique, system-generated MAC address on vmk0, which prevents MAC conflicts in environments with multiple hosts or when using DRS. While it’s manageable with a single host, in clustered setups the default behavior often causes connectivity issues until ARP tables refresh. By recreating vmk0, we avoid these frustrating interruptions.