Build your Network programmability Environement in your Labtop via Pycharm to Automate your Network via Python

Always Network Engineers in a dilemma of How to build their Network Programmable Environment :

They read a lot but few who take steps towards the Hands-on.

That Article will decrease that gap.

- How to start write Python Code on your Windows machine ?

- What are the Tools you need to start developing your scripts ?

- How to Create your first project in that tool ?

Your Labtop or PC should be like the below one

1. Python either 2 or 3 .

2. Python interpreter (I am using Pycharm) 

3. If your Labtop has connection to real Network so it's ok. 

    if you need to connect to your Simulated Environment

    So you will need to go for either

    - EVE-NG 

    - GNS3

  and both of GNS3 or EVE-NG will be hosted as VMs on either VMware workstation,     

    Vmware vSphere or virtual box.

 1) Python

Python 2 is installed by default on most of the OSs now 

you can open your command prompt and check that by typing python.

if Not , go for the that URL to download Python 

  Download Python

Python interpreter Installation and let's Open the First Project

Let's begin with Python interpreter at Which we will write over it our Python Code .

1.     How to download the Python Interpreter jetbrains Pycharm ? 

         click on the below hyperlink to download it's community edition

          Download Pycharm (python interpreter)

      2. How to Create project on Pycharm.

- Open Pycharm  , you will see a page like the below

click on create New project

 - choose the directory that you will host your Project  and give your project a name and    

     choose the interpreter either  python 2 or 3

    here i choose python 2  and my project name is  : MyfirstPythonproject

finally click Create

 - your project appear now at the environment

- Right Click on the project itself then choose New > Python file

- Assign for your file a name and click ok
   No need here to write the .py extension of the file as it is already by choosing the file          type.

- Congrats 😉 you can type python right now.

   let's type your first Code

This code to print the GigabitEthernet interfaces with no shutdown below it to open all the interfaces inside my  Network Device.

here i  make a simple for loop to iterate from 0 to 9

to open the first ten interfaces at my line card at slot 0  

so i wll print the string "GigabitEthernet0/"  and the variable will be after the   /

 but because we are iterating over integer number , so in the print we can't add string to integer.

so we will change the type of the variable i firstly , and the variable becomes y now .

Now let's run the script ,

- After running the Code the output will appear in the below half of the Screen from the Run tab

  to adust that we can right click on Run and choose Windowed mode 

the output will appear in another Screen

For sure you can copy it and paste it directly to your Network device 😉

In the Coming blog post i will show you how to run from Pycharm a python Code towards The Network Devices Topology inside the EVE-NG simulator.

Be ready and download the EVE-NG from the below link 😉

       Download EVE-NG (formerly unetlab)

Happy labbing and keep tuned.

Regards,
Mostafa Hassan Ahmed

Cisco ASR9K Configuration via Python , JINJA and YAML

Network Automation becomes a mandatory now and Network engineers should adapt themsleves to write a code.

I started  from more than a year  reading in multiple python libraries like paramiko , netmiko , napalm and Ciscoconfparse and i learned how to loop over devices to push and pop configuration from network devices  and parse the output via regex 
then i took a break for the CCIE SP and now am back again 😉 


This Article will go through : 

1. What are the Network Devices Configuration Templates ?
2. What are the methods to merge the variables of each device with the template ? 
3. What are jinja and YAML ? 
4.  Live example on generating configuration files for Cisco ASR9K devices by the use of jinja2 and YAML python libraries.  

What are the Network Devices Configuration Templates ?

Any Network device's configuration has a static part for each service 
for e.g. the ospf configuration in ASR9k is like the below 


==============================
router ospf  1
 log adjacency changes
 router-id 100.100.100.100
 area 0
  interface Loopback0
  !
  interface GigabitEthernet0/0/0/0
  !
  interface GigabitEthernet0/0/0/1
  !
  interface GigabitEthernet0/0/0/2
  !
  interface GigabitEthernet0/0/0/3
  !
 !
!
=========================================
   
you will find that find we have that always the variable part is the process id , router-id , area number and the interfaces. 

and the fixed part for sure the other things. 

Configuration templating means that we want to define the fixed part in a seperate place and the variable parts in another place. 

then merging them via a method. 

in that Case the fixed part will be the jinja file 
the variables of each devices will be defined in the YAML file. 


What are the methods to merge the variables of each device with the template ? 

 we have 2 methods to do that 

> via Python Code
> via Ansible module 

here in that post we will cover the python method and 
Ansible one will be covered in the coming post. 


What are jinja and YAML ? 

Jinja :

it is a Python library for creating configuration based on templates. Jinja2 defines a templating language with which templates are created. 

jinja files extensions  .j2 or .txt


YAML : 

YAML is a human friendly data serialization standard for all programming languages. It is not a markup language. It is more human-friendly compared to XML or JSON. Python has a YAML library to parse the YAML files into dictionaries. Once the dictionaries are created, these can be used to create the configuration from templates using Jinja2.

YAMl files extensions  .yml or .yaml
 
Now let's cut it short and go to the  example :

Prerequisites

1. you have to install python on your system
2. you have to install jinja2 and yaml libraries

 you should reach to the below status to complete  if you wanna practice the example.

Now you can go to your IDE or text editor  (Pycharm , Sublime , Atom , .... ) to create the project
which will consists of those files

1. Devices_meta_data.yaml
2. ASR9K-temlate.j2
3. ASR9K-generate-config.py

Python Code

from jinja2 import Environment, FileSystemLoader import yaml import os with open("Devices_meta_data.yaml", 'r') as stream: Devices = (yaml.load(stream)) THIS_DIR = os.path.dirname(os.path.abspath(__file__)) print THIS_DIR j2_env = Environment(loader=FileSystemLoader(THIS_DIR), trim_blocks=True) for device, data in Devices.iteritems(): configuration = j2_env.get_template('template.j2').render(data) with open(os.path.join(THIS_DIR, device + ".txt"), 'w') as config_file: config_file.write(configuration)

Click on the below photo to see it in your browser which declare what is happening at each part of the code.

Here is the YAML Code of 2 ASR9K devices at each one we are definning
> hostname 
> interfaces
> ospf 
> bgp 
> mpls 

To apply connectivity to the other routers at the topology ,
for sure also we can do that for other ios routers but to make it simple here we applied that just for ASR9K devices.

YAML Code 

ASR9K-1:
  hostname: XR1-0
  interfaces:
    loopback0:
      ipaddr: 100.100.100.100
      sm: 32
    GigabitEthernet0/0/0/0:
      ipaddr: 70.70.111.1
      sm: 24
    GigabitEthernet0/0/0/1:
      ipaddr: 70.70.112.1
      sm: 24
    GigabitEthernet0/0/0/2:
      ipaddr: 70.70.113.1
      sm: 24
    GigabitEthernet0/0/0/3:
      ipaddr: 70.70.114.1
      sm: 24
    GigabitEthernet0/0/0/4:
      ipaddr: 70.70.115.1
      sm: 24
    GigabitEthernet0/0/0/5:
      ipaddr: 70.70.100.1
      sm: 24
    GigabitEthernet0/0/0/6:
      ipaddr: 70.70.123.1
      sm: 24

  ospf:
    ospf_id: 100.100.100.100
    area : 0
    interfaces:
      - loopback0
      - GigabitEthernet0/0/0/0
      - GigabitEthernet0/0/0/1
      - GigabitEthernet0/0/0/2
      - GigabitEthernet0/0/0/3
      - GigabitEthernet0/0/0/4
      - GigabitEthernet0/0/0/5
      - GigabitEthernet0/0/0/6
  mpls:
    mpls_id: 100.100.100.100
    interfaces:
      - loopback0
      - GigabitEthernet0/0/0/0
      - GigabitEthernet0/0/0/1
      - GigabitEthernet0/0/0/2
      - GigabitEthernet0/0/0/3
      - GigabitEthernet0/0/0/4
      - GigabitEthernet0/0/0/5
      - GigabitEthernet0/0/0/6
  bgp:
    bgp_id: 100.100.100.100
    local_as: 1
    neighbors:
      ABR-1:
        neighbor_address: 11.11.11.11
        neighbor_as: 1
      ABR-2:
        neighbor_address: 12.12.12.12
        neighbor_as: 1
      ABR-31:
        neighbor_address: 13.13.13.13
        neighbor_as: 1
      ABR-32:
        neighbor_address: 23.23.23.23
        neighbor_as: 1
      ABR-4:
        neighbor_address: 14.14.14.14
        neighbor_as: 1
      ABR-5:
        neighbor_address: 15.15.15.15
        neighbor_as: 1
      ASBR1-0:
        neighbor_address: 10.10.10.10
        neighbor_as: 1

ASR9K-2:
  hostname: XR2-0
  interfaces:
    loopback0:
      ipaddr: 200.200.200.200
      sm: 32
    GigabitEthernet0/0/0/0:
      ipaddr: 80.80.111.1
      sm: 24
    GigabitEthernet0/0/0/1:
      ipaddr: 80.80.112.1
      sm: 24
    GigabitEthernet0/0/0/2:
      ipaddr: 80.80.113.1
      sm: 24
    GigabitEthernet0/0/0/3:
      ipaddr: 80.80.114.1
      sm: 24
    GigabitEthernet0/0/0/4:
      ipaddr: 80.80.115.1
      sm: 24
    GigabitEthernet0/0/0/5:
      ipaddr: 80.80.100.1
      sm: 24

  ospf:
    ospf_id: 100.100.100.100
    area : 0
    interfaces:
      - loopback0
      - GigabitEthernet0/0/0/0
      - GigabitEthernet0/0/0/1
      - GigabitEthernet0/0/0/2
      - GigabitEthernet0/0/0/3
      - GigabitEthernet0/0/0/4
      - GigabitEthernet0/0/0/5
  mpls:
    mpls_id : 200.200.200.200
    interfaces:
      - loopback0
      - GigabitEthernet0/0/0/0
      - GigabitEthernet0/0/0/1
      - GigabitEthernet0/0/0/2
      - GigabitEthernet0/0/0/3
      - GigabitEthernet0/0/0/4
      - GigabitEthernet0/0/0/5
  bgp:
    bgp_id: 200.200.200.200
    local_as: 1
    neighbors:
      ABR-1:
        neighbor_address: 11.11.11.11
        neighbor_as: 1
      ABR-2:
        neighbor_address: 12.12.12.12
        neighbor_as: 1
      ABR-31:
        neighbor_address: 13.13.13.13
        neighbor_as: 1
      ABR-32:
        neighbor_address: 23.23.23.23
        neighbor_as: 1
      ABR-4:
        neighbor_address: 14.14.14.14
        neighbor_as: 1
      ABR-5:
        neighbor_address: 15.15.15.15
        neighbor_as: 1
      ASBR1-0:
        neighbor_address: 10.10.10.10
        neighbor_as: 1

Hints about YAML  :
- Begin any YAML file with ---
- No TABs no TABs no TABs  just spaces.
- Take Care of the indentation.

let's Zoom in the YAML and what  if we wanna map it to python

it's more better right ?  Not a lot of curl prackets and Commas.

YAML

Python

ospf:     ospf_id: 100.100.100.100     area : 0     interfaces:       - loopback0       - GigabitEthernet0/0/0/0       - GigabitEthernet0/0/0/1       - GigabitEthernet0/0/0/2       - GigabitEthernet0/0/0/3       - GigabitEthernet0/0/0/4       - GigabitEthernet0/0/0/5       - GigabitEthernet0/0/0/6

{   'ospf': {   'area': 0,                 'interfaces': [   'loopback0',                                   'GigabitEthernet0/0/0/0',                                   'GigabitEthernet0/0/0/1',                                   'GigabitEthernet0/0/0/2',                                   'GigabitEthernet0/0/0/3',                                  'GigabitEthernet0/0/0/4',                                   'GigabitEthernet0/0/0/5',                                   'GigabitEthernet0/0/0/6'],                 'ospf_id': '100.100.100.100'                } }

So we will complete in the same manner each section of configuration for each device.

Kepping in mind that all devices are exist in the Same YAML File.


Now let's jump to the Jinja part . Jinja not Ninja 😉


Jinja Code 

hostname {{hostname}}

{%if interfaces%}
{%for intf , int_config in interfaces.items()%}
interface {{intf}}
  ipv4 address {{int_config['ipaddr']}}/{{int_config['sm']}}
  no shutdown
!
{%endfor%}
{%endif%}

{%if ospf %}
router ospf 1
  router-id {{ospf['ospf_id']}}
  area {{ospf['area']}}
{%for ospf_intf in ospf['interfaces']%}
    interface {{ospf_intf}}
    !
{%endfor%}
{%endif%}

{%if bgp %}
router bgp {{bgp['local_as']}}
  bgp router-id {{bgp['bgp_id']}}
  address-family ipv4 unicast
  !
{%for neighbor , neighbor_info in bgp['neighbors'].items()%}
  neighbor {{neighbor_info['neighbor_address']}}
   remote-as {{neighbor_info['neighbor_as']}}
   update-source Loopback0
   address-family ipv4 unicast
    route-reflector-client
   !
  !
!
{%endfor%}
{%endif%}


{%if mpls %}
mpls ldp
 router-id {{mpls['mpls_id']}}
 address-family ipv4
 !
{%for interface in mpls['interfaces']%}
 interface {{interface}}
 !
{%endfor%}
!
end
{%endif%}

Jinja file is simple just contain the fixed the part of the configuration and
we are calling the variables in the YAML  ((accessing dictionaries)) which is in green colour in the above .

also it is a normal text file with python conditional expressions and iteration embedded  in it

like if and for  , but here it should be between {%   %} unlike Python.



the below image shows now side by side both 3 files

Finally after running the python code we will get the 2 Configuration files of both devices and they will appear in the place that we predefined in the python code.

ASR9k-1

ASR9k-2

hostname XR1-0 interface loopback0   ipv4 address 100.100.100.100/32   no shutdown ! interface GigabitEthernet0/0/0/6   ipv4 address 70.70.123.1/24   no shutdown ! interface GigabitEthernet0/0/0/5   ipv4 address 70.70.100.1/24   no shutdown ! interface GigabitEthernet0/0/0/4   ipv4 address 70.70.115.1/24   no shutdown ! interface GigabitEthernet0/0/0/3   ipv4 address 70.70.114.1/24   no shutdown ! interface GigabitEthernet0/0/0/2   ipv4 address 70.70.113.1/24   no shutdown ! interface GigabitEthernet0/0/0/1   ipv4 address 70.70.112.1/24   no shutdown ! interface GigabitEthernet0/0/0/0   ipv4 address 70.70.111.1/24   no shutdown ! router ospf 1   router-id 100.100.100.100   area 0     interface loopback0     !     interface GigabitEthernet0/0/0/0     !     interface GigabitEthernet0/0/0/1     !     interface GigabitEthernet0/0/0/2     !     interface GigabitEthernet0/0/0/3     !     interface GigabitEthernet0/0/0/4     !     interface GigabitEthernet0/0/0/5     !     interface GigabitEthernet0/0/0/6     ! router bgp 1   bgp router-id 100.100.100.100   address-family ipv4 unicast   !   neighbor 10.10.10.10    remote-as 1    update-source Loopback0    address-family ipv4 unicast     route-reflector-client    !   ! !   neighbor 15.15.15.15    remote-as 1    update-source Loopback0    address-family ipv4 unicast     route-reflector-client    !   ! !   neighbor 14.14.14.14    remote-as 1    update-source Loopback0    address-family ipv4 unicast     route-reflector-client    !   ! !   neighbor 13.13.13.13    remote-as 1    update-source Loopback0    address-family ipv4 unicast     route-reflector-client    !   ! !   neighbor 12.12.12.12    remote-as 1    update-source Loopback0    address-family ipv4 unicast     route-reflector-client    !   ! !   neighbor 11.11.11.11    remote-as 1    update-source Loopback0    address-family ipv4 unicast     route-reflector-client    !   ! !   neighbor 23.23.23.23    remote-as 1    update-source Loopback0    address-family ipv4 unicast     route-reflector-client    !   ! ! mpls ldp  router-id 100.100.100.100  address-family ipv4  !  interface loopback0  !  interface GigabitEthernet0/0/0/0  !  interface GigabitEthernet0/0/0/1  !  interface GigabitEthernet0/0/0/2  !  interface GigabitEthernet0/0/0/3  !  interface GigabitEthernet0/0/0/4  !  interface GigabitEthernet0/0/0/5  !  interface GigabitEthernet0/0/0/6  ! ! end

hostname XR2-0 interface loopback0   ipv4 address 200.200.200.200/32   no shutdown ! interface GigabitEthernet0/0/0/5   ipv4 address 80.80.100.1/24   no shutdown ! interface GigabitEthernet0/0/0/4   ipv4 address 80.80.115.1/24   no shutdown ! interface GigabitEthernet0/0/0/3   ipv4 address 80.80.114.1/24   no shutdown ! interface GigabitEthernet0/0/0/2   ipv4 address 80.80.113.1/24   no shutdown ! interface GigabitEthernet0/0/0/1   ipv4 address 80.80.112.1/24   no shutdown ! interface GigabitEthernet0/0/0/0   ipv4 address 80.80.111.1/24   no shutdown ! router ospf 1   router-id 100.100.100.100   area 0     interface loopback0     !     interface GigabitEthernet0/0/0/0     !     interface GigabitEthernet0/0/0/1     !     interface GigabitEthernet0/0/0/2     !     interface GigabitEthernet0/0/0/3     !     interface GigabitEthernet0/0/0/4     !     interface GigabitEthernet0/0/0/5     ! router bgp 1   bgp router-id 200.200.200.200   address-family ipv4 unicast   !   neighbor 10.10.10.10    remote-as 1    update-source Loopback0    address-family ipv4 unicast     route-reflector-client    !   ! !   neighbor 15.15.15.15    remote-as 1    update-source Loopback0    address-family ipv4 unicast     route-reflector-client    !   ! !   neighbor 14.14.14.14    remote-as 1    update-source Loopback0    address-family ipv4 unicast     route-reflector-client    !   ! !   neighbor 13.13.13.13    remote-as 1    update-source Loopback0    address-family ipv4 unicast     route-reflector-client    !   ! !   neighbor 12.12.12.12    remote-as 1    update-source Loopback0    address-family ipv4 unicast     route-reflector-client    !   ! !   neighbor 11.11.11.11    remote-as 1    update-source Loopback0    address-family ipv4 unicast     route-reflector-client    !   ! !   neighbor 23.23.23.23    remote-as 1    update-source Loopback0    address-family ipv4 unicast     route-reflector-client    !   ! ! mpls ldp  router-id 200.200.200.200  address-family ipv4  !  interface loopback0  !  interface GigabitEthernet0/0/0/0  !  interface GigabitEthernet0/0/0/1  !  interface GigabitEthernet0/0/0/2  !  interface GigabitEthernet0/0/0/3  !  interface GigabitEthernet0/0/0/4  !  interface GigabitEthernet0/0/0/5  ! ! end

Hope that this example is clear and please if there is some thing not explained well let me know to clarify it in detail.

Happy Labbing 😉

Regards ,
Mostafa Hassan Ahmed