WAN Dynamics received a message from a customer late one evening that no network services company wants to get: “We performed a code upgrade and broke connectivity to our disaster recovery site”. This customer had a datacenter interconnect (DCI) with BGP routing and VXLAN between the datacenters but after the EOS upgrade there was reachability issues between them. I jumped in to assist and at first glance, things looked good. The switch interfaces and BGP sessions were up and there was full connectivity from VTEP to VTEP in the datacenter interconnect. That said, MAC addresses were not being learned across VXLAN. Hmmm, wonder why that would be? I decided to check the switch logs for any potential clues as to what could be happening. Aha! in the log buffer there are messages about VXLAN being disabled:
VXLAN decapsulation has been disabled on
Ethernet48 because it carries both default VRF and non- default VRF traffic
Ethernet47 because it carries both default VRF and non- default VRF traffic
Ethernet46 because it carries both default VRF and non- default VRF traffic
This is due to a modification in VXLAN that Arista made to correct a potential security issue explained in Arista Security Advisory 0055 from December 2020: “On impacted devices, malformed packets could be incorrectly forwarded across VRF boundaries when non-default VRFs are configured. This issue affects UDP traffic, and will fail to complete the three-way handshake for TCP traffic.” So this is actually expected behavior when upgrading to any of the following versions of EOS or later on select hardware platforms and when leveraging multiple VRFs on a single interface:
In order to enable VXLAN once again you would need to configure the following on the VTEP interfaces which would disable the VXLAN filter on all Ethernet interfaces:
vxlan decapsulation filter interface multiple-vrf disabled
If you only want to disable the filter on specific Ethernet interfaces, you would use the following instead:
vxlan decapsulation filter interface multiple-vrf disabled EthernetX EthernetY
After adding the configuration and running the command “
show vxlan address-table” we then saw the MAC address entries we were expecting and service was restored. Whew!
In future code releases this command will remain, so expect to work around it from here on out. This was a little bit of a surprise to the customer so we wanted to put this article out to potentially avoid the same surprise for others!
Thanks for reading and if you like this, check out some of my other articles on Arista EOS and all the cool things you can do with it.