It’s time for things to get really interesting. Welcome back to part 4.
We know that VxLAN is an overlay technology. It creates tunnels over an IP underlay network.
The ingress VTEP encapsulates the traffic, and sends it to the egress VTEP. The egress
VTEP decapsulates the traffic, so it can be delivered to its ultimate destination.
But have you wondered, how does the ingress VTEP find the egress VTEP?
There could be many VTEPs. How does the switch find the right one? Also, how does it learn
what the destination MAC address are?
There’s two ways to approach address learning. The first is called Data Plane Learning. This
is the traditional method for learning about addresses. It’s similar to traditional ethernet
in many ways.
The other is control plane learning, which is a newer and more sophisticated approach.
This method uses BGP to share MAC address information. This is similar to the way BGP
learns and shares routes.
But before getting too deep into this, we need to think about the two different traffic
types that we will encounter.
The first is unicast traffic. This is where traffic is sent to a specific single destination.
The other is called BUM Traffic. This needs special handling, which needs a closer look...
BUM traffic! Yes, it’s a strange name, especially in my little corner of the world. But that
aside, it is something that we need to know about for VxLAN.
BUM is actually an acronym, which stands for Broadcast, Unknown unicast, and Multicast.
Collectively, these are traffic types that need special handling.
Simply put, it’s any traffic that goes to more than one destination. ARP is an example
of BUM traffic.
In a traditional ethernet network, BUM traffic may be flooded to many destinations. VxLAN
needs to be smarter in its approach. Otherwise, it simply wouldn’t scale well in a large
network.
There are two possible ways that VxLAN can handle BUM traffic. These are Multicast and
Head end replication.
Multicast is probably the most common solution. Each VNI is mapped to a single multicast group.
Each multicast group may map to one or more VNI’s.
When a VTEP comes online, it uses IGMP to join the multicast groups for the VNI’s
that is uses. If there is a VNI that the VTEP does not use, it does not need to join that
group.
When a VTEP needs to send BUM traffic, it will send it only to the relevant multicast
group. This is one method of VTEP discovery.
This is a very efficient method. It may be complicated, depending on your multicast infrastructure.
We’ll have a look at how this is configured in part 5 of this series.
Head end replication is an alternative to multicast. But, only it is only available
if you use BGP EVPN.
When BUM traffic arrives, the VTEP creates several unicast packets and sends one to each
remote VTEP that supports the VNI.
This is not as efficient as multicast, and certainly doesn’t scale as well. However,
it’s much simpler if you don’t have a multicast infrastructure. It’s fine for
about 20 VTEPs or less
In the early days, VxLAN relied on data plane learning. This is also known as flood and
learn, and is very similar to regular ethernet.
Let’s take a moment to think about how Ethernet works. When a frame needs to be sent, the
sender needs to find the destination address.
To find the destination, the sender floods the frame out. The correct device will respond
to the sender, all others will ignore the frame.
When the sender gets the response, it will add the address to its cache. Or perhaps ‘cash’
if you’re from a different part of the world
Eventually, the cache will expire, and flooding will happen again.
VxLAN Data plane learning is very similar. It’s a little better, as flooding only occurs
to multicast group members.
However, data plane learning has a serious limitation. There is no built-in support for
routing. It is only used for ‘bridging’ devices at layer-2.
To reach the outside world, you will need an external router as your gateway.
If you want to route between VNI’s, you also need an external router, which will cause
traffic to ‘hairpin’.
From a security perspective, keep in mind that VTEPs are not authenticated. There’s
nothing built-in to prevent a rogue VTEP in the network.
In most cases, control plane learning is recommended. Data plane learning is still worth investigating,
as its simpler to understand.
We’ll have a look at configuring this in part 5 of this series.
Learning through the data plane has several drawbacks. Control plane learning is much
more functional and efficient.
Control plane learning means that switches learn MAC addresses before they’re needed.
This works in the same way as a routing protocol. Switches peer with each other using BGP, and
share the addresses that they know about.
This uses the EVPN address family. If you’re not familiar with address-families, they are
ways for BGP to carry reachability information for different protocols.
There are IPv4 and IPv6 address families, VPN address families for MPLS, and others.
You’ll get to see this in action in part 6.
But for now, let me explain how this all works. Each switch runs BGP, and peers with other
switches over the IP network.
The normal BGP rules still apply here, so you will need either a full mesh or route-reflectors.
Some or all of these switches will contain VTEPs. This means that each switch automatically
learns where all the other VTEPs are in the network.
This also provides VTEP authentication. When VTEPs are learned through BGP, they are added
to a whitelist. All other VTEPs are untrusted. BGP authentication may be used to prevent
rogue peers.
Host MAC addresses are added into the local BGP process. They are discovered when they
start up or send GARP messages.
Their MAC address is then shared with all BGP peers.
So, when a host sends a frame to another host, the switch looks up the MAC address in BGP.
It immediately knows which VTEP to send the traffic to without flooding.
At this point, you might be wondering… how does ARP fit into the picture? When you use
control plane learning, you are also using ARP Suppression.
This doesn’t mean that ARP is entirely gone. The hosts on your network have no idea that
VxLAN or EVPN exist, so they will still send ARP requests when they need to find another
host.
When the ARP request reaches the local switch, it is not flooded out as normal.
Flooding is not needed, because the local switch has already learned where the destination
host is, including the IP to MAC address mapping.
Rather than flooding the ARP message, the local switch sends an ARP response back to
the source host.
So then, what about BUM traffic? Is it still relevant when control plane learning is used?
Most definitely yes! Applications still send broadcast and multicast traffic. Unknown Unicast
traffic may even appear if there are silent hosts in the network.
To handle BUM traffic, as already discussed, you can use multicast or head end replication.
The general recommendation is to use multicast, as it is the most scalable option.
There is a major advantage to using BGP EVPN. It supports Integrated Routing and Bridging.
Unlike data plane learning, you don’t need to use an external router.
VNI’s can be configured as either a Layer-2 VNI, or a layer-3 VNI. Both layer-2 and layer-3
information is carried in BGP.
A layer-2 VNI is used for bridging. This is when traffic is all kept within the same LAN
segment.
A layer-3 VNI is used for routing, when traffic needs to leave a layer-2 VNI.
Layer-3 VNI’s are technically optional, but if you want to route on the local switch,
you need them.
VTEPs only need to know about the layer-2 VNI’s that they service locally. On the
other hand, all VTEPs need to know about all layer-3 VNI’s. This is to support a feature
called Anycast Gateway.
When IRB is used, each switch acts as a default gateway for hosts in a VNI. Rather than give
each switch a different IP address, they all have the same IP, and the same virtual MAC
address.
There is no need for hello messages or timers like HSRP and VRRP have.
This means that all hosts in a VNI can have the same default gateway, regardless of which
switch they’re connected to. This helps with virtual machine mobility.
To support multi tenancy, layer-3 VNI’s are attached to a VRF. If you’ve used MPLS,
this will feel familiar.
This means that many VNI’s can be associated with a customer or tenant. Routing tables
are kept separate by using Route Targets and Route Distinguishers.
We’ll have a look at how this is configured in part 6 of this series.
In the fifth part of this series, we’ll start on configuration with VxLAN bridging.
I’d also really like to get your feedback on this video. Please leave me a comment so
I know what you thought.
I’ll see you in Part 5