The title explains pretty well what this post is about. After playing with VXLAN’s in the previous posts, the next obvious step these days is to automate it. And to do this I’ll be using Ansible.
So what I am trying to automate is creating the VLANs, clag (similar to Cisco vPC), vrr(similar to VRRP), BGP, evpn , vxlan on Cumulus switches.
See the previous 2 posts on these tasks:
https://latebits.com/2020/01/31/vxlans-on-cumulus/
https://latebits.com/2020/02/03/routing-vxlans/
To do this, first, I had to put everything in some order. So switch ports and IPs have changed since the last posts. This is because in order to automate something and also be scalable you need to put some rules in place and create something like a blueprint. Because for example, adding a new switch to your design should be easy to do. It should be a matter of changing some parameters, not the script.
But first, the new design:
You see above and also in the configuration, later, that some IPs have changed (loopbacks, client IP’s, anycast IPs, AS numbers), and the switch ports have now a specific order in how they are connected.
So for example all the spine switches use port 1,2,3 to connect to each leaf switch. The leaf switches also have port 1,2 connected to each spine switch, ports 5 and 6 are used for clag peering and port 3 is used for the southbound connection. Loopback IP of the spine is now in the 10.5.5.x range and Vlans are in 192.168.100.x for VLAN 100 and in 192.168.200.x for VLAN 200.
To configure all the switches I’ve used a management network. Each switch has its management port (eth0) attached to a Management switch. This switch I’ve bridged it via a Cloud object to my PC, where I’ve created a Loopback on the same subnet with all the switches. In this way, I can run my Ansible script from my PC.
The Ansible module i’ve used is this (it’s the only one that’s available): https://docs.ansible.com/ansible/latest/modules/nclu_module.html#nclu-module
My inventory files look like this:
[leaf]
leaf1 ansible_ssh_host=10.0.0.1 ansible_ssh_user=cumulus ansible_ssh_pass=CumulusLinux! id=1
leaf2 ansible_ssh_host=10.0.0.2 ansible_ssh_user=cumulus ansible_ssh_pass=CumulusLinux! id=2
leaf3 ansible_ssh_host=10.0.0.5 ansible_ssh_user=cumulus ansible_ssh_pass=CumulusLinux! id=3
[spine]
spine1 ansible_ssh_host=10.0.0.4 ansible_ssh_user=cumulus ansible_ssh_pass=CumulusLinux! id=1
spine2 ansible_ssh_host=10.0.0.6 ansible_ssh_user=cumulus ansible_ssh_pass=CumulusLinux! id=2
You will see that I use that “id” variable in the scripts below.
So let’s start with the spine switches (spine.yml) :
- hosts: spine
gather_facts: no
vars_files:
- group_vars/vars
tasks:
- name: bgp
nclu:
commands:
- add hostname 'Spine{{id}}'
- add loopback lo ip address '10.5.5.{{id}}/32'
- add bgp autonomous-system '6555{{id}}'
- add bgp neighbor 'swp1-{{leafnumber}}' interface remote-as external
- add bgp l2vpn evpn neighbor 'swp1-{{leafnumber}}' activate
- add bgp network '10.5.5.{{id}}/32'
commit: true
The script looks pretty easy, there is not much that can be explained. You see that I use the “id” for every device to define the AS number for example and the loopback IP. Also, I use a variable called “leafnumber”.
This and the “spinenumber”can be found in a file called “vars” under the “group_vars” folder, and its content looks like this:
#this is for spine switches
leafnumber: 3
#this is for leaf switches
spinenumber: 2
In this file i define the number of leaf switches and the number of spine switches.
Next is the leaf.yml file. But first i need to share with you another var file: group_vars/leaf/vars
#this is for leaf switches
clagid: 1
vlans:
vlan100:
id: '100'
number: '01'
vlan200:
id: '200'
number: '02'
Here I define how many clag(vPC,Mlag) will be configured, and also the VLANs. Vlans are a dictionary, and each VLAN has an id and a number. The id I am using it as VLAN id and the number I am using is for the anycast MAC address.
So let’s start with the clag and the VLANs:
---
- hosts: leaf
gather_facts: no
vars_files:
- group_vars/vars
tasks:
- name: add clag
nclu:
commands:
- add hostname 'Leaf{{id}}'
- add clag port bond bond'{{clagid}}' interface swp3 clag-id '{{clagid}}'
commit: true
- name: add clag role
nclu:
template: |
{% if id % 2 == 0 %}
add clag peer sys-mac 44:38:39:FF:01:01 interface swp5-6 secondary
{% else %}
add clag peer sys-mac 44:38:39:FF:01:01 interface swp5-6 primary
{% endif %}
atomic: true
As one switch has to be the primary and one the secondary in clag, I am using the id of the device for this. So if the id is an odd number, that switch will be the primary.
- name: create vlans
nclu:
template: |
{% for item in vlans.values() %}
add vlan "{{item.id}}"
add bond bond"{{clagid}}" bridge trunk vlans "{{item.id}}"
add vlan "{{item.id}}" ip address 192.168."{{item.id}}".25"{{id}}"/24
add vlan "{{item.id}}" ip address-virtual 00:00:5e:00:01:"{{item.number}}" 192.168."{{item.id}}".254/24
{% endfor %}
atomic: true
As you see I am using the “item.id and item.number ” which are the id and number from the VLANs dictionary to define the VLAN id, IP, and mac.
Next is the BGP,evpn part:
- name: bgp evpn
tags: bgp
nclu:
commands:
- add loopback lo ip address '10.1.1.{{id}}/32'
- add bgp autonomous-system '6500{{id}}'
- add bgp neighbor 'swp1-{{spinenumber}}' interface remote-as external
- add bgp l2vpn evpn neighbor 'swp1-{{spinenumber}}' activate
- add bgp l2vpn evpn advertise-all-vni
- add bgp network '10.1.1.{{id}}/32'
commit: true
I am using the spine number variable for defining how many ports the BGP will run on (1-2). The id of the device is used for AS, loopback IP.
Each clag pair needs to share the same anycast IP for the loopback (used for VTEP):
### create anycast for each lag pair
- name: Vtep anycast
tags: anycast
nclu:
template: |
{% set y = id + 1 %}
{% if id % 2 == 0 %}
add loopback lo clag vxlan-anycast-ip 10.10.10."{{id}}"
add bgp network 10.10.10."{{id}}"/32
{% else %}
add loopback lo clag vxlan-anycast-ip 10.10.10."{{y}}"
add bgp network 10.10.10."{{y}}"/32
{% endif %}
atomic: true
This was a little hard to do (till I found one solution). So I am using the id of the device for this. If the id is an odd number the anycast IP will be id+1, if not it will be the id. This way each pair will have the same IP.
For example, if id=3 then the anycast IP will be 10.10.10.4, and for id=4 , the same.
The last part is the vxlan’s:
- name: vxlan
tags: vxlan
nclu:
template: |
{% for item in vlans.values() %}
add vxlan vni"{{item.id}}" vxlan id "{{item.id}}"
add vxlan vni"{{item.id}}" bridge access "{{item.id}}"
add vxlan vni"{{item.id}}" bridge learning off
add vxlan vni"{{item.id}}" stp bpduguard
add vxlan vni"{{item.id}}" stp portbpdufilter
add vxlan vni"{{item.id}}" vxlan local-tunnelip 10.1.1."{{id}}"
add vxlan vni"{{item.id}}" bridge arp-nd-suppress on
{% endfor %}
atomic: true
Here I’ve used the VLANs dictionary, but this time only the “item.id”.
As you may have noticed I am using templates (jinja) on some tasks. They are a great way of using control structures like if, for, math, comparisons, logic, etc. ( https://jinja.palletsprojects.com/en/master/templates/ )
So to conclude, you can use these files to add a new spine/leaf switch, add a new VLAN and vxlan. All you need to do is to edit the var files. You can also run only the tasks that you need. (if a task does not have a tag , i am sure you can add one).
For example, if you want to create a new vxlan and VLAN, you only have to edit the group_vars/leaf/vars file to add a new VLAN and run the leaf.yml file with the tag for the vxlan task only.
like this: ansible-playbook leaf.yml -i inventory -vv –tags “vxlan”
And that’s it.
You can find the yml/var files and the GNS project files, here:
https://github.com/czirakim/Ansible.VxLAN.Cumulus
