Welcome back to
part 6!
We’re going to take a simple topology and configure VxLAN with EVPN. Just like part
5, this is a basic topology to keep things simple.
If you’ve come straight to this video, I would recommend going back and watching part
5 first. It explains a few extra things that we won’t cover in detail here.
We still have two switches, with a routed link. The routed link is the underlay network.
This time, we will have two tenants. They will have two hosts each, one on each switch.
There will also be some IP addressing overlap on the hosts, to confirm that this won’t
be a problem.
One of our tenants will have a single VNI. The other will have two different VNI’s,
and will need routing between them.
This time we will use head-end replication, also known as ingress-replication. This is
just so you can see how it’s configured. Multicast would also work well.
We’ll also see how to add anycast gateway, and ARP suppression.
This time, we’ll focus on getting the underlay up and running first. This is the IP network,
which doesn’t know about VxLAN.
We’re going to configure the routed link, MTU size, and OSPF.
We’ll start by getting the OSPF feature enabled, and increasing the maximum MTU.
Next, we’ll get the routed link up, and advertised into OSPF.
As we’re not using multicast this time, we will only need one loopback interface.
This is still where the VTEP gets its IP address from.
And of course, repeat everything on the second switch.
Now that’s done, we can verify that the neighbours come up… And there it is.
Ready to move on to the overlay.
Now it’s time for things to get interesting. We’re going to enable the features we need
for the overlay, configure the anycast gateway virtual MAC address, and perform the base
configuration of BGP.
We start by enabling the overlay features. We need the BGP feature for peering, and nv
overlay evpn adds the EVPN address family.
Interface-vlan is the same feature we use when we want to create VLAN SVI’s. In this
case, we use it to create a virtual anycast gateway interface.
The other two features were covered in part 5, so I won’t go over them again here.
Anycast Gateway uses the same vMAC on all switches. We assign this with the fabric forwarding
anycast-gateway-mac command. Just remember to be sensible when you choose a MAC address,
so you don’t cause a conflict.
The VTEP is quite similar to before. Now though, we have the command host-reachability protocol
bgp. This enables control plane learning on the VTEP.
Each switch needs to run an instance of BGP. We specify the other switch as a neighbour.
In the real world, you may have many neighbours, or a few route-reflectors.
We’re going to use loopback 0 as the source for BGP communication. This is the same IP
address that’s applied to the VTEP.
Under the l2vpn evpn address family, we need to enable sending communities. This is needed
so we can send route target information to our neighbours.
And finally, the corresponding config is added on the second switch.
There’s a fair bit of work to get the first tenant up and running. This includes creating
a layer-3 VNI and a VRF, configuring Route Distinguishers and Route Targets, configuring
the VTEP, ARP suppression, head-end replication, the anycast gateway, and BGP EVPN.
As we work through this, remember that some of the config is for layer-3, and some is
for layer-2. This can get confusing. I’ll try to explain it the best I can, but don’t
be afraid to ask questions in the comments.
Let’s get started with creating our first tenant.
The first thing that we need when we create a tenant is a layer-3 VNI. This serves two
purposes. It allows routing on the local switch, and it can be attached to a VRF.
This tenant uses VNI 900001 as their L3VNI. We associate it with VLAN 101.
Here, we’re creating the tenant’s VRF. The first thing we do is add the L3VNI.
We also need to set the route distinguisher. If you’re not familiar with RD’s, they
are used to keep the VRF unique in the BGP database.
The RD value can be manually specified, which is what I recommend in a mixed vendor environment.
In an all-Nexus environment, I recommend letting the switch assign it automatically.
Route-Targets are used to both import and export routes and evpn addresses. The auto
keyword lets the switch select its own unique value.
If you haven’t seen this before, let me give you a simplified description. ‘Exporting’
marks the routes from the local VRF in the BGP database, with the route target value.
The routes in BGP are shared with other neighbours. Neighbours receive the routes, and look for
the Route Target values that they want to import. Imported routes are added into an
appropriate VRF.
In short, this is a way to share routes and MAC addresses between peers, while still associating
them with a tenant.
An SVI is created for VLAN 101. This is the VLAN associated with the L3VNI, and is associated
with the tenant’s VRF.
The ip forward command is used to enable this interface to forward traffic. This essentially
turns on routing.
VTEP configuration is a bit different now. We need to add the tenant’s VNI’s, including
the L3VNI. The associate-vrf keyword tells the VTEP that this VNI is used for routing.
The suppress-arp command enables ARP suppression. If you remember back to part 4, this enables
the local switch to answer a hosts ARP request without flooding.
The ingress-replication command tells the VTEP to use head-end replication for BUM traffic.
As you can see, it’s a very simple configuration. This replaces the need for multicast in this
lab.
Let’s head back over to BGP. We go into the tenant’s VRF, and their IPv4 address
family. Then we use the advertise l2vpn evpn command to allow external routes to be advertised
as EVPN routes within the VRF.
This won’t make a lot of difference in our lab, as we don’t have any external routes
to advertise in. I’ve included the command for your reference.
Incidentally, there is little difference if you want to use IPv6. Just remember to use
the IPv6 address family.
Now we’re just creating our VLANs and VNI’s for our hosts. An SVI is created for VLAN
1000, so we can configure the anycast gateway. Notice that the IP address is the same on
both switches. This lets any of the switches act as the default gateway for the hosts.
The evpn section is used to advertise MAC address information through BGP. The MAC Routes
also need to be kept unique, so they still use route targets. This is very similar to
how the layer-3 information was shared earlier.
And finally, we configure our host ports. I configured one of the ports incorrectly
here, but due to the magic of video editing, I have invisibly fixed that up.
That’s our first VNI for our first tenant. There’s a fair bit of work to get the initial
config up and running.
We’re now adding the second VLAN and VNI to this tenant. This should all be fairly
familiar to you now.
The basic procedure is; add VLAN and associate VNI, create an SVI for the anycast gateway,
add the VNI to the VTEP, and add the VNI into EVPN.
Now that the first tenant is ready, we can start the next one. Much of this config will
be the same, so we’ll move through it fairly quickly.
I’ll also avoid showing you the host port configuration. I think you understand that
fairly well by now.
We’ll create a VRF for this tenant, using layer-3 VNI 900002. This is attached to VLAN
102, and ip forwarding is enabled.
This tenant uses VNI 6000 for their hosts. This is attached to VLAN 900. As before, the
anycast gateway address is added to this SVI.
Even though this tenant doesn’t need to route between VNI’s, the anycast gateway
is still the default gateway for external networks, like the internet or a WAN.
The L2VNI and L3VNI are added to the existing VTEP. We don’t need to create new VTEPs
for each tenant.
Switch 2 has the same configuration in this case. As you can see, adding a new tenant
is not as complex as setting up the first one.
Now the two tenants have been configured, let’s verify that it’s all working. The
commands we’re going to use here are useful when troubleshooting. I suggest bookmarking
this, and referring back when you need it.
We’re jumping right in with show bgp l2vpn evpn summary. This command shows us our BGP
neighbours. If you’re used to BGP, you should feel comfortable here.
As usual, we can see the neighbour, their AS number, and their state. If the state is
IDLE, you have problems.
We can see VTEPs with the show nve peers command. Notice here on switch 1 that this now shows
that the learning type is ‘CP’ for control plane.
On the second switch we’re using show nve vni. This shows us the VNI’s that are associated
with the VTEP. We also see that control plane learning is configured.
The multicast group column would normally show the multicast group that the VNI is bound
to. This is for the BUM traffic that we discussed in part 4.
In our case, this shows as UnicastBGP. This is because we configured ingress replication
instead of multicast on these VNI’s. The L3VNI’s are used for routing, so the multicast
column is not relevant to them.
Let’s get on one of our hosts. In this lab I’m using a router to simulate a host.
First we see that there are no pings dropped while we learn about destination addresses.
This is for two reasons. One, the local switch already knows all the IP to MAC bindings,
and two, ARP suppression causes the host’s ARP request to be answered very quickly.
If we try to ping an address owned by the other tenant, we can see that this fails.
This proves that we have successfully separated our customer’s traffic.
The same is true in reverse. Tenant-2 has access to their resources, but not to tenant-1’s
resources.
Back on ARP suppression, we can see the IP to MAC bindings that the switches are returning
to the hosts, with the show ip arp suppression-cache detail command.
This even shows the ports that the hosts are connected on. The L flag shows that the host
is connected locally. The R flag shows that the host has been learned from a remote VTEP.
We can use show vxlan to easily see VLAN to VNI mappings.
As you know well by now, MAC addresses are shared through BGP. To see the MAC’s that
have been learned, we can look at the BGP database with show l2route evpn mac all.
The next-hop column is interesting here. This shows the IP of the remote VTEP that is used
to get to this address.
If the Rmac flag is set, this indicates that this an L3VNI, used for routing.
We can change this command slightly to show l2route evpn mac-ip all. This gives another
way to see MAC to IP bindings, the VNI they’re on, and the VTEP they’re behind.
If BGP is listed next to an entry, this entry has been learned from a remote VTEP. HMM is
Host Mobility Manager. In this context, this means that the address has been learned on
the local switch.
The big one is show bgp l2vpn evpn. There’s a lot of data in this one.
This is how you see all the details in the BGP database. We can see that MAC addresses
have been learned, along with the IP that it maps to.
The route distinguisher is also shown. This essentially represents the tenant’s VRF.
This is the automatically assigned value that we configured earlier.
And that’s all for the VxLAN series. I hope you’ve enjoyed watching the videos and have
learned something useful, or at least interesting.
Please share what you thought of the series in the comments below. If you liked it, please
share and subscribe.
Thanks for staying with me through this series