Skip to content

Matt Adam

Tech Blog

Menu
  • Home
  • Home Lab
    • Home Lab
    • Home Lab with NSX-T
  • Kubernetes
    • Openshift
    • K3s
    • Tanzu
      • Tanzu – TKGs (WCP)
      • Tanzu – TKG (TKGm)
    • Avi Kubernetes Operator (AKO)
  • About
  • Privacy Policy
Menu

Home Lab – NSX-T Troubleshooting

Posted on July 13, 2022September 15, 2022 by Matt Adam

If you’ve made it this far and you’re still seeing errors in your lab (as I did) then there are a few things you should look for.

  • DNS
  • NTP
  • MTU

Table of Contents

  • DNS
  • NTP
  • MTU
    • Physical ESXi and Nested ESXi
    • vyos Router
    • In NSX-T

DNS

I would ssh into all the components we’ve created (esxis, nsx-mgr, edge, vyos, etc…) and do some digs and nslookups. If any of them fail, then there is a DNS problem. Also test outbound connectivity, dig google.com and see if you get a response. There are a ton of guides out there on troubleshooting DNS issues, but definitely get this resolved before you move on.

NTP

NTP can cause quite a few issues in nested environments. Make sure NTP is set on all the esxi hosts, physical and nested. NSX Manager, edge, vyos, dns. Everything! And test it!

#To test ntp: ssh into the device and run

ntpq -p
You should see a similar output. If you do not see delays, offset, jitter, etc. Or if they’re all 0. NTP is not working correctly.

MTU

And finally MTU. NSX is surprisingly picky about MTU..

From VMware: NSX-T leverages the Generic Network Virtualization Encapsulation (Geneve) protocol, a
network virtualization tunneling protocol used to establish tunnels across transport
nodes to carry overlay traffic. Transport nodes include VM and physical-based Edges,
ESX hosts, and KVM Hypervisors, all of which require at least one Geneve Termination
End Point (TEP). With encapsulation technologies, like Geneve, it is essential to
increase the maximum transmission unit (MTU) supported both on transport nodes and
the physical network underlay. This article looks at steps to validate MTU in an NSX-T
Environment.
Read More

From Me: Set everything to MTU 9000 and you don’t have to worry about anything.

If you’re using an unmanaged switch, like I am, then google it to make sure it supports jumbo frames. Since we have 2 supermicro esxi hosts, any network devices that it passes through must also support jumbo frames.

Physical ESXi and Nested ESXi

MTU can be set in 2 places, on the virtual switch and on the VMKernel Nics

On the virtual Switches
on the VMKernel NICs

One way that you can test this, is by running a command vmkping on the hosts (physical and nested)

# Example: On physical host1 you can vmkping your other physical host2
# -d don't fragment
# -s send up to a 8972 MTU

vmkping -d -s 8972 192.168.3.5
This is what a successful ping looks like.

vyos Router

Make sure MTU is set at 9000

In NSX-T

Check the Tier 0 router, go to interfaces, and make sure the interface is 9000 MTU.
You can also modify the Global Gateway Config and set MTU to 9000. Networking -> Settings -> Global Networking Config

2 thoughts on “Home Lab – NSX-T Troubleshooting”

  1. Robbi Levesque says:
    April 21, 2023 at 11:14 am

    When applying an MTU change in the interface, when are these taken into consideration? Is this change life or do we have to restart EDGE and ESX?

    Thanks

    Reply
    1. Matt Adam says:
      May 3, 2023 at 4:04 pm

      I don’t remember restarting any edges/esx hosts. As soon as you make the change, it goes into effect. Same as the vyos router.

      Reply

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Recent Posts

  • Upgrading the ESXi Node from vSphere7 to vSphere 8
  • Setting up the Kubernetes Dashboard
  • Running a DNS server in K3s
  • Raspberry Pi Kubernetes Cluster
  • Pod Routing: NodePort, ClusterIP, NodePortLocal

About

My name is Matt Adam and I’m a Product Line Manager at VMware.

I support the NSX Advanced Load Balancer (Avi Networks) with a focus on containers and Kubernetes. I have a background in load balancing, automation, development, and public cloud.

© 2023 Matt Adam | Powered by Minimalist Blog WordPress Theme