WAN Dynamics Senior Network Engineer Kevin Giusti explains the dual datacenter DMZ configuration with VXLAN, VARP, VRF & OSPF.
Based on business needs, dual datacenter networks will evolve from an “active/standby” disaster recovery configuration that has an active primary datacenter carrying workloads and a site sitting on standby over to a “active/active” datacenter environment with workloads spread across both centers. Typically in the datacenters there will be several demilitarized zone (DMZ) networks for various Internet access functions. With this design there usually will then be a requirement to extend the DMZ networks across the datacenter interconnect (DCI).
Virtual Extensible Local Area Network (VXLAN, https://tools.ietf.org/html/rfc7348) is a great technology to solve this problem. However, enabling a VXLAN based DCI is only part of the equation to make the solution work appropriately. Some challenges encountered in an “active/active” datacenter DMZ internet access approach are:
- Devices in the secondary datacenter will trombone across the DCI in order to reach their default gateway which works but is inefficient and adds latency.
- If the active firewall in primary datacenter has a hardware failure, all clients on the DMZ networks will not be able to reach their default gateway and will not be able to route causing an outage.
- High Address Resolution Protocol (ARP) churn will occur in a failover event.
The following are techniques to make efficient and reliable use of VXLAN in a dual “active/active” datacenter environment utilizing Virtual ARP (VARP), Virtual Routing and Forwarding (VRF) and a dynamic routing protocol, in our case Open Shortest Path First (OSPF):
- Create a VRF instance for each DMZ VLAN in the switches. This will create a dedicated routing and forwarding table in the switch for the routes and segment them away from your internal private routes.
- Use VARP on the VLAN interfaces so that the default gateway for clients is the data center core switch. VARP allows the same IP address to be used on an interface in multiple switches not to mention multiple datacenters.
- Use a routing protocol such as OSPF to make the 4 datacenter core switches and the two firewalls all form an adjacency within the DMZ VLAN and VRF.
- Increase OSPF cost on the firewall links that should not be forwarding traffic under normal circumstances. Here we increased the cost on the firewall in the secondary data center so that it will only be used if there is a failure in the primary data center.
The following are example config snippets of what this might look like:
#PRIMARY DATACENTER
vrf definition SFTP-DMZ
rd 65004:18
ip routing vrf SFTP-DMZ
#Primary datacenter Core Switch 1
interface vlan 200
description SFTP-DMZ
vrf forwarding SFTP-DMZ
ip address 192.168.170.2/24
ip virtual-router address 192.168.170.1
router ospf 10 vrf SFTP-DMZ
passive-interface default
no passive-interface Vlan100
network 192.168.170.2/32 area 100
router-id 192.168.170.2
#Primary datacenter Core Switch 2
interface vlan 200
description SFTP-DMZ
vrf forwarding SFTP-DMZ
ip address 192.168.170.3/24
ip virtual-router address 192.168.170.1
router ospf 10 vrf SFTP-DMZ
passive-interface default
no passive-interface Vlan100
network 192.168.170.3/32 area 100
router-id 192.168.170.3
#SECONDARY DATACENTER
#Secondary datacenter Core Switch 1
interface vlan 200
description SFTP-DMZ
vrf forwarding SFTP-DMZ
ip address 192.168.170.6/24
ip virtual-router address 192.168.170.1
router ospf 10 vrf SFTP-DMZ
passive-interface default
no passive-interface Vlan100
network 192.168.170.6/32 area 100
router-id 192.168.170.6
#Secondary datacenter Core Switch 2
interface vlan 200
description SFTP-DMZ
vrf forwarding SFTP-DMZ
ip address 192.168.170.7/24
ip virtual-router address 192.168.170.1
router ospf 10 vrf SFTP-DMZ
passive-interface default
no passive-interface Vlan100
network 192.168.170.7/32 area 100
router-id 192.168.170.7
Once implemented each DMZ network may take in more routes than required from the network. Since traffic should only be sent from each core to the active firewall, a distribute list can be applied to cut down on superfluous routes. The configuration on one of the core switches would be similar to this to only allow a default route:
ip prefix-list OSPF-IN seq 10 permit 0.0.0.0/0
route-map OSPF-IN permit 10
description match OSPF routes
match ip address prefix-list OSPF-IN
router ospf 10 vrf SFTP-DMZ
distribute-list route-map OSPF-IN in
In the switches, you should only see a default route originating from the firewalls. The following is output similar to what you should expect to see:
PrimaryDC-Core1#show ip route vrf SFTP-DMZ
VRF name: SFTP-DMZ
Codes: C - connected, S - static, K - kernel,
O - OSPF, IA - OSPF inter area, E1 - OSPF external type 1,
E2 - OSPF external type 2, N1 - OSPF NSSA external type 1,
N2 - OSPF NSSA external type2, B I - iBGP, B E - eBGP,
R - RIP, I L1 - ISIS level 1, I L2 - ISIS level 2,
A B - BGP Aggregate, A O - OSPF Summary,
NG - Nexthop Group Static Route, V - VXLAN Control Service
Gateway of last resort:
O E1 0.0.0.0/0 [110/11] via 192.168.170.4, Vlan200
C 192.168.170.0/24 is directly connected, Vlan200
Shutting the interfaces on the primary data center firewall to force the OSPF adjacency to fail and then the other default route is learned.
PrimaryDC-Core1#show ip route vrf SFTP-DMZ
VRF name: SFTP-DMZ
Codes: C - connected, S - static, K - kernel,
O - OSPF, IA - OSPF inter area, E1 - OSPF external type 1,
E2 - OSPF external type 2, N1 - OSPF NSSA external type 1,
N2 - OSPF NSSA external type2, B I - iBGP, B E - eBGP,
R - RIP, I L1 - ISIS level 1, I L2 - ISIS level 2,
A B - BGP Aggregate, A O - OSPF Summary,
NG - Nexthop Group Static Route, V - VXLAN Control Service
Gateway of last resort:
O E1 0.0.0.0/0 [110/11] via 192.168.170.8, Vlan200
C 192.168.170.0/24 is directly connected, Vlan200
Though at times complex, implementing active/active datacenter designs become easier with technologies such as VXLAN, VARP and VRFs. This configuration has been utilized in production and works very well. That said, recommendations and comments are always welcomed.
Thanks for reading!