Ansible.VxLAN.Cumulus

The title explains pretty good what this post is about. After playing with VXLAN’s in the previous posts, the next obvious step these days is to automate it. And to do this i’ll be using Ansible.

So what i am trying to automate is creating the vlans, clag (similar to Cisco vPC), vrr(similar to VRRP), bgp, evpn , vxlan on Cumulus switches.
See the previous 2 posts on these tasks:
https://latebits.com/2020/01/31/vxlans-on-cumulus/
https://latebits.com/2020/02/03/routing-vxlans/

To do this, first i had to put everything in some order. So switch ports and ip’s have changed since last posts. This is because in order to automate something and also be scalable you need to put some rules in place and create something like a blueprint. Because for example , adding a new switch to your design should be easy to do. It should be a matter of changeing some parameters , not the script.

But first , the new design:

You see above and also in the configuration, later, that some ip’s have changed (loopback’s, client ip’s, anycast ip, AS numbers) ,and the switch ports have now a specific order in how they are connected.
So for example all the spine switches use ports 1,2,3 to connect to each leaf switch. The leaf switches also have ports 1,2 connected to each spine switch, ports 5 and 6 are used for clag peering and port 3 is used for southbound connection. Loopback ip of the spine are now inthe 10.5.5.x range and Vlans are in in 192.168.100.x for vlan 100 and in 192.168.200.x for vlan 200.

To configure all the switches i ‘ve used a management network. Each switch has it’s management port (eth0) attached to a Management switch. This switch i’ve bridged it via a Cloud object to my PC , where i’ve created a Loopback on the same subnet with all the switches. In this way i can run my Ansible script from my PC.

The Ansible module i’ve used is this (it’s the only one that’s available): https://docs.ansible.com/ansible/latest/modules/nclu_module.html#nclu-module

My inventory files looks like this:

[leaf]
leaf1 ansible_ssh_host=10.0.0.1 ansible_ssh_user=cumulus ansible_ssh_pass=CumulusLinux! id=1
leaf2 ansible_ssh_host=10.0.0.2 ansible_ssh_user=cumulus ansible_ssh_pass=CumulusLinux! id=2
leaf3 ansible_ssh_host=10.0.0.5 ansible_ssh_user=cumulus ansible_ssh_pass=CumulusLinux! id=3

[spine]
spine1 ansible_ssh_host=10.0.0.4 ansible_ssh_user=cumulus ansible_ssh_pass=CumulusLinux! id=1
spine2 ansible_ssh_host=10.0.0.6 ansible_ssh_user=cumulus ansible_ssh_pass=CumulusLinux! id=2

You will se that i use that “id” variable in the scripts below .

So let’s start with the spine switches (spine.yml) :

- hosts: spine
  gather_facts: no
  vars_files:
    - group_vars/vars
  tasks:
    - name: bgp
      nclu:
        commands:
             - add hostname 'Spine{{id}}'
             - add loopback lo ip address '10.5.5.{{id}}/32'
             - add bgp autonomous-system '6555{{id}}'
             - add bgp neighbor 'swp1-{{leafnumber}}' interface remote-as external
             - add bgp l2vpn evpn neighbor 'swp1-{{leafnumber}}' activate
             - add bgp network '10.5.5.{{id}}/32'
        commit: true       

The script look pretty easy , there is not much that can be explained. You see that i use the “id” for every device to define the AS number for example and the loopback ip. Also i use a variable called “leafnumber”.
This and the “spinenumber”can be found in a file called “vars” under the “group_vars” folder, and it’s content looks lie this:

#this is for spine switches
leafnumber: 3

#this is for leaf switches
spinenumber: 2

In this file i define the number of leaf switches and the number of spine switches.

Next is the leaf.yml file. But first i need to share with you another var file: group_vars/leaf/vars

#this is for leaf switches
clagid: 1

vlans:
  vlan100:
    id: '100'
    number: '01'
  vlan200:
    id: '200'
    number: '02'

In here i define how many clag(vPC,Mlag) will be configured, and also the vlans. Vlans are a dictionary , and each vlan has an id and a number. The id, i am using it as vlan id and the number i am using for the anycast MAC address.

So let’s start with the clag and the vlans:

---
- hosts: leaf
  gather_facts: no
  vars_files:
    - group_vars/vars
  tasks:
    - name: add clag
      nclu:
        commands:
             - add hostname 'Leaf{{id}}'
             - add clag port bond bond'{{clagid}}' interface swp3 clag-id '{{clagid}}'
        commit: true

    - name: add clag role
      nclu:
        template: |
          {% if id % 2 == 0 %}
          add clag peer sys-mac 44:38:39:FF:01:01 interface swp5-6 secondary
          {% else %}
          add clag peer sys-mac 44:38:39:FF:01:01 interface swp5-6 primary
          {% endif %}
        atomic: true

As one switch has to be the primary and one the secondary in clag, i am using the id of the device for this. So if the id is an odd number, that switch will be the primary.

- name: create vlans
      nclu:
        template: |
          {% for item in vlans.values() %}
          add vlan "{{item.id}}"
          add bond bond"{{clagid}}" bridge trunk vlans "{{item.id}}"
          add vlan "{{item.id}}" ip address 192.168."{{item.id}}".25"{{id}}"/24
          add vlan "{{item.id}}" ip address-virtual 00:00:5e:00:01:"{{item.number}}" 192.168."{{item.id}}".254/24
          {% endfor %}
        atomic: true

As you see i am using the “item.id and item.number ” which are the id and number from the vlans dictionary to define the vlan id, ip and mac.

Next is the bgp,evpn part:

    - name: bgp evpn
      tags: bgp
      nclu:
        commands:
             - add loopback lo ip address '10.1.1.{{id}}/32'
             - add bgp autonomous-system '6500{{id}}'
             - add bgp neighbor 'swp1-{{spinenumber}}' interface remote-as external
             - add bgp l2vpn evpn neighbor 'swp1-{{spinenumber}}' activate
             - add bgp l2vpn evpn advertise-all-vni
             - add bgp network '10.1.1.{{id}}/32'
        commit: true

I am using the spine number variable for defining how many ports the bgp will run on (1-2). The id of the device is used for AS, loopback ip.

Each clag pair needs to share the same anycast ip for the loopback (used for VTEP):

### create anycast for each lag pair
    - name: Vtep anycast
      tags: anycast
      nclu:
        template: |
          {% set y = id + 1 %}
          {% if id % 2 == 0 %}
          add loopback lo clag vxlan-anycast-ip 10.10.10."{{id}}"
          add bgp network 10.10.10."{{id}}"/32
          {% else %}
          add loopback lo clag vxlan-anycast-ip 10.10.10."{{y}}"
          add bgp network 10.10.10."{{y}}"/32
          {% endif %}
        atomic: true

This was a little hard to do (till i found one solution). So i am using the id of the device for this. If the id is an odd number the anycast ip will be id+1, if not it will be the id. This way each pair will have the same ip.
For example if id=3 than the anycast ip will be 10.10.10.4 and for id=4 , the same.

The last part are the vxlan’s:

  - name: vxlan
      tags: vxlan
      nclu:
        template: |
          {% for item in vlans.values() %}
          add vxlan vni"{{item.id}}" vxlan id "{{item.id}}"
          add vxlan vni"{{item.id}}" bridge access "{{item.id}}"
          add vxlan vni"{{item.id}}" bridge learning off
          add vxlan vni"{{item.id}}" stp bpduguard
          add vxlan vni"{{item.id}}" stp portbpdufilter
          add vxlan vni"{{item.id}}" vxlan local-tunnelip 10.1.1."{{id}}"
          add vxlan vni"{{item.id}}" bridge arp-nd-suppress on
          {% endfor %}
        atomic: true

In here i’ve used the vlans dictionary, but this time only the “item.id”.

As you may have noticied i am using templates (jinja) on some tasks. They are a great way of using control structure like if, for, math, comparisons, logic etc. ( https://jinja.palletsprojects.com/en/master/templates/ )

So to conclude, you can use these files to add a new spine/leaf switch, add a new vlan and vxlan. All you need to do is to edit the var files. You can also run only the tasks that you need. (if a task does not have a tag , i am sure you can add one).

For example if you want to create a new vxlan and vlan, you only have to edit the group_vars/leaf/vars file to add a new vlan, and run the leaf.yml file with the tag for the vxlan task only.
like this : ansible-playbook leaf.yml -i inventory -vv –tags “vxlan”

And that’s it.

You can find the yml/var files and the GNS project files, here:
https://github.com/czirakim/Ansible.VxLAN.Cumulus

About the author

Mihai is a Network Aficionado with more than 10 years experience