Hey Folks !

Do checkout this introductory video on Cisco Model Driven Telemetry on my new YouTube channel.

This video aims to present an overview . Do give your suggestion through comments! Thanks

In a previous article we saw that Computers/machines communicating with each other might use different encoding/data formats.

The three formats that are heavily used in the networking world for the exchange of data are JSON, XML and YAML.

Objects

So all the languages described above are used in denoting what is called objects. Let's deep dive a bit. An object is a container for information. It is how we pass information between machines through API. So we need a way to describe the object, don't we? A way to describe the characteristics of the object.

The multiple characteristics can always be denoted by key-value pairs.

Say for example let's take an object which denotes an employee. An employee might have the following attributes. An ID number, a name, a department etc. All these characteristics can be described as a key and value pair.

Let us look into the details of the object language one by one:-

XML ( eXtensible Markup Language )

Amongst the discussed formats, XML is the heaviest data type; very difficult to comprehend by humans. It is often used when lots of data is exchanged between machines. If the XML data is very small then we can attempt to understand it with some effort.

But as the structure grows heavier, it becomes highly difficult to comprehend and interpret.

Semantics

XML has been the language for the machine to machine communications for a very long time. It is very similar to HTML. Some of the key features of XML are:-

• It's a markup language.
• XML uses a system of tags to denote key-value pairs
• Tags are used in the format <tags></tags>
• The value is contained inside a tag pair <key>value</key>
• XML is case sensitive in the tag usage.

Let us look at a sample XML file to understand the semantics described above.

<employee>
  <name>Arjuna</name>
  <location>
    <city>Delhi</city>
    <state>Delhi</state>
  </location>
  <roles>Network Engineer</roles>
  <roles>Blogger</roles>
</employee>

JSON ( JavaScript Object Notation)

If we compare the painfully heavy nature of XML with JSON, JSON is very light and more human-readable. But why was JSON needed in the first place? Imagine you are running an application on your phone. The application might fetch data from the server. Now, what format will you choose for the data exchange? Will you send HTML? Will you send plain text? Now the plain text will increase complications at the client side. How would the client know about the data it has received? So in this object-oriented world, we need a structured data exchange between the client-server architecture. In this case, JSON comes to the rescue. It is a notation that will help applications running Javascript easily parse and get the value of objects.

Will it help only Javascript users?

No, it has become a common object notation language that is now being used by multiple languages. Many come with built-in or custom libraries to parse and use JSON objects.

Semantics

Some key points to keep in mind while constructing a JSON file.

• The syntax makes use of {} (curly braces) , [] (square braces and) , comma.
• Whitespaces don't matter.
• The Tag concept is not used here.

We can open a JSON object by using {} and go on to list the key-value pairs. Both the key and value are to be surrounded by quotes " if there is a need to denote the value as a string. Multiple attributes of an object are separated from each other using ','.

Let us look at the same data we represented using the YAML file above through JSON constructs:-

{
    "employee": {
        "name": "Arjuna", 
        "location": {
            "city": "Delhi", 
            "state": "Delhi"
        },
        "roles": [
            "network engineer",
            "blogger"
        ]
    }
}

Notice how the square braces are used to denote an array in JSON ( Compared to the hyphens we used in YAML)

YAML ( YAML Aint Markup Language)

YAML language was designed with one and the only thing in mind.

Human readability

It is designed to be a very clean language. As a developer, you can use two extensions for YAML files. They are yml and yaml. Both are acceptable. We see YAML being converted to XML and JSON to make it mode machine-readable in a few use cases. YAML is mostly used to define configurations. It uses indentation to structure or define the objects. So people who are familiar with Python can understand YAML structure very easily.

Semantics

Some key points to keep in mind while constructing a YAML file.

• YAML is not a markup language.
• There is no concept of tags.
• Whitespaces matter in YAML. Let us look at a sample YAML file.

---
employee:
  name: Arjuna
  location:
    city: Delhi
    state: Delhi
  roles:
    - network engineer
    - blogger

All data of a particular object uses the same indentation level. Key-value pairs are separated by the use of colon ":". "_ _ _" mentions the start of a YAML file.

In YAML, typically two spaces are used for indentation, but we can use any form of indentation we are comfortable with.

Remember indentation using TABS is not supported in YAML. Use 2 space / 4 space indentations.

YAML assumes the data type of the attributes based on the content. This applies to numbers strings etc. So there is no explicit need to append " to strings. Also note that there is no comma "," at the end of the value of a key-value pair. To denote the values forming part of a list use the dash/ hyphen "-" symbol.

Hope this article gave an introduction to various encoding languages used in API communication. Bye.

Introduction

Evolution of Software

We all are encountering different types of Software in our day to day lives. The development process for each of this software is different and unique. Every software solves a separate purpose. Earlier we used to have big fat applications which used to serve a single purpose, but nowadays we see everything migrating towards Web apps or Mobile Apps. We see most of them integrated with Social media. We see many apps making use of SSO of popular social media apps like Facebook Linkedin etc. It is only fair to say there is heavy reuse of code everywhere and no one wants to re-invent the wheel

Why the hype behind API?

APIs help separate the modules by the functionality.

You interface with an API because it solves a particular problem for you. Take for example the Google Maps API. You interface with it if you want geolocation functionality.

Imagine the several new web applications that are being developed day in day out. Now there are several modules implemented in the backend to give that nice user interface to you. At the backend, a module might take care of authentication, another might take care of database etc. Now all the interaction between the modules is possible because of API. It allows programmers to develop their own module and expose the functionality through an API and not be concerned about what other programmers have developed.

You can develop and test your implemented function through API.

Distributed Computing

Typically all these modules might be implemented in a single system/server or they might even be distributed. Take the example of google maps API. Your application might reside on a server in India. The maps API could point to a server in California. So the app implemented by you in your server has to extend an API call to the California server, fetch the response and then present it to your clients in a nice GUI. Can we say this to be a distributed system? we sure can. We need to take care of the networking requirements we implement such distributed systems.

Documentation of APIs

The documentation for API forms key for people to efficiently use and implement the API in their code. The doc should mention clearly how you are expecting the data input, what is the output format etc. Also, the language you expect the API server and client to speak is very important. Shall we call it encoding? Most of the API design uses a standard encoding schema.

We shall see more about the encoding schemas in upcoming articles

Let us zoom through some of the commands and use cases for PIP the python package manager.

pip install "xyz"

Use the install command to install any of the modules that you need. Earlier you could search for the packages using the search command. It has now been discontinued. So you might get an error when you try to use their API for searching packages.

pip list

You can use the pip list command to list all the packages that you currently have installed.

pip uninstall "xyz"

You can use the uninstall command to uninstall any of the modules that you wish to remove.

pip list --outdated

You can use the list command with the outdated option to get a list of all outdated packages. You will get to know what version you have installed versus the latest one.

So how do you update the outdated packages?

pip install -U "xyz"

The install command with the -U option will let you upgrade the module to the latest version.

What's the use of the freeze command?

pip freeze

The freeze command will output all the modules installed in a format that is easier for the user to compile and make a requirements file. Suppose you need to ship the code to somebody. You need to tell them that these are the requirements that you need to have before you run my code. In that case, you will do a pip freeze; copy the contents and send him the requirements txt file.

pip freeze > requirements.txt

The above command will get the output from pip freeze and put the contents into the requirements text file.

Suppose you have sent Mr X this text file. How will he install all the requirements on his computer? Will he do it one by one?

pip install -r requirements.txt

The -r modifier will instruct the pip to install all the modules mentioned in the requirements text file.

How to download modules through pip?

There might be a use-case where you need to download the package from one computer and install it on another. We can use pip to download the files and save the files on a directory of choice using the following command.

pip download "xyz" --dest "C:/path.."

This will download all the necessary files in this folder. This folder can be zipped and shipped to the remote destination and there it can be installed using the requirements.txt file method that was discussed previously.

Hope you found this snippet useful for working with pip the python package manager.

Hey, folks let's look at the usage of "__name__" in this snippet. Let us try to print the variable from a python program called code1.py

print(__name__)

On running this we get,

Python loads a few Environment variables before it starts executing the code that you have given it to execute.  The "__name__" variable is one such special variable. Here we ran the code directly by executing code1.py and therefore python has set the "__name__"  variable to main.

Let us now import the code1.py python module into another python program called code2.py. Now we all know that python runs the complete code when we import a module into another module. Now let's test how the "__name__" variable behaves when the code1.py is imported in code2.py

import code1

code2.py has only the above-mentioned piece of code. So, let us run this and compare it with what we got when we execute code1.py

Look at that!. We now have code1 as the output when we ran code2.py.

INFERENCE

The "__name__" variable is initialised with main if the code is directly run by python. On the contrary when the code is imported and internally run by python; the "__name__" is initialised with the name of the imported module. That is why we got code1 as the output of the print function when it was imported by code2.py.

USECASE

if __name__ == "__main__":
	{
    "Place the code that you want to run if this module is executed directly"
    }
else:
	{
    "Place the code that you want to run if this module is imported"
    }

A check condition using an if statement such as the one mentioned in the above example can be incorporated whenever we want certain things to be done only when the module is executed directly. Whatever code we put inside this check statement will not be executed when this module is imported into another module.

Drop-in your comments if you find this snippet useful. Bye!.

In Part 1 of this article, we took a look at how to set up your base VM and get the shell script ready for your docker container. In this article let's look quickly at how to install Nginx, set up the HTTPS certificates with Certbot and LetsEncrypt and check our mail settings for our newly created blog.

Installing Nginx is straightforward. Nginx will act as a frontend intercepting requests forwarded to our server and forwarding it inturn to our docker container which is listening on a custom port.

sudo apt-get install nginx

Once installed let us create a file inside the sites-enabled folder of Nginx.

/etc/nginx/sites-enabled/yourdomain.xyz

server {

	server_name <YOUR DOMAIN>.com www.<YOUR DOMAIN>.com;

	location ~ /.well-known {
		allow all;
		break;
	}

	location / {
		proxy_pass http://127.0.0.1:2368;
		proxy_set_header X-Real-IP $remote_addr;
		proxy_set_header HOST $http_host;
		proxy_set_header X-Forwarded-Proto https;	
		proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
	}
}

Set up the directories that will be used by LetsEncrypt for server validation.

mkdir /var/www/.well-known
chmod 755 /var/www/.well-known

Install Certbot and the plugin for Nginx.

sudo apt install certbot python3-certbot-nginx

Let Certbot do its job. Remember in the previous article we had looked into setting up Cloudflare for our domain. You would want to temporarily pause Cloudfare in this domain so that the proxy features or disabled. You can also do this by enabling maintenance mode. This would help Certbot during the checks and allow it to complete the process. Take care of this proxy thing during renewals also.

sudo certbot --nginx -d example.com -d www.example.com

Replace your domain in the above command.

MAIL GUN API credentials

Once the Ghost settings interface is up and running you can key in your Mailgun API credentials

Note this is in addition to the SMTP credentials that we had configured in Part 1.

ZOHO setup. Now we have configured the mail dispatch settings that will help us send emails from our domain ( newsletters etc). But it will not let us receive any emails. That's why it is recommended to configure a separate sub-domain for Mailgun that will take care of our sending requirements. To have a mail server setup for our root domain we can take help from Zohos always free tier. Their website offers complete instructions on setting up your DNS records and other settings. Check it out here.

I think by now you would have given the docker container we built in the previous article a spin. It would be listening on the custom port we configured. Configure your firewall settings accordingly.

Yay!. We now have HTTPS configured, set up proxying through Cloudflare, our bulk email setting should work now (now that we have configured mail gun), we have an email service for us through ZOHO and that's pretty much it. I think we pretty much have everything to get a basic domain and a website up and running. There is one thing left. Backups. You can use your own means to back up the ghost_blog folder that we created to back up the content in total including images and stuff.

Ghost also gives you an option to backup the contents by downloading a JSON file. You can choose to do that also. But it doesn't include images and other contents. Should you bump into any issue setting up the blog after following the instructions let me know in the comments.  

In this post, let us try to further concepts introduced earlier. We started looking at the way a yang data model is structured. In this article let's take a sample yang model and try to understand it part by part.

MODULES, PREFIXES

We first start with the modules. A module can comprise multiple sub-modules. Every module needs to be uniquely identified by a namespace. This will help to differentiate this yang model from many other models that may reside in the system/device. The prefix is sort of a shorthand notation we would use to reference this namespace subsequently in this module.

ADMINISTRATIVE CONTENT IN YANG HEADER

The module header can include administrative content like organisation, contact information etc. The revision field can help us uniquely identify the revision of the module. Including revision- fields can help prevent any mismatch between the devices that are talking NETCONF protocol. How? During the initial hello exchange the devices involved in NETCONF exchange their data models and capabilities and any mismatch in revision in the yang header can be easily identified by the devices.

IMPORT / INCLUDES

Then there is the concept of imports and includes. Import statements are used when we want to reference definitions from another yang file. It is used to just reference; so the entire body of the referenced file is not pulled into the referencing yang module.  This is especially useful when we write a module that supposedly contains a common theme and we can reference it in multiple other modules for clarity.

On the other hand, include statements are used to pull in the parts of the yang file spread over multiple files. The included sub-modules are stiched and pulled together into the referencing module when an include statement is used.

What's the need for a sub-module?

Sub-modules allow the writer to split a big complex model design into several small sub-modules. All such sub-modules combine and contribute to the module that calls the sub-modules using the previously introduced "include" statement.

Notice how the sub-module does not have the name-space declaration, instead has a belongs-to statement. A sub-module can belong to only a single module.

DATA TYPES SUPPORTED IN YANG

Yang supports most of the common data types integer, unsigned integer etc. All supported base types have been defined in RFC 6020.  Some of the supported data types are:-

The final element of yang that stores the data (ex leaf) can store a base type or a derived type. A derived type can be defined by a typedef or grouping. We will not be looking into groupings at this point in time.

TYPEDEF

Typedefs are used to define a new derived data type using the base type with some restrictions. Let us see this example to understand better:-

In this typedef, the author has tried to impose a restriction on the values the defines datatype "per cent" could take using the range keyword. Later we see the leaf "completed" taking the type of the newly defined "per cent" using the type keyword. Several modifiers can be used to play around with the restrictions. If we come across a yang file we can always google to find out the meaning of that modifier or keyword to understand the restrictions it brings to the table. When one derived data type references another derived datatype previously defined, the restrictions of the new derived type are applied on top of the restrictions posed by the mother derived data type.  

UNION STATEMENT

If we require our derived data types to be either of two base data types then we can mark such restrictions using a union statement. Let's take a look at this example:-

In the above example, we try to define a new datatype called "threshold" which can hold values that either belongs to the unsigned integer base type with restriction of (0 to 100) or it can be of type "enumeration" which can hold one of the enum type called "disabled".

Apart from the base types defined in RFC 6020,  the working group thought they should build a module for the commonly used objects of the networking world. This module is named "ietf-yang-types". So anybody who wishes to use these definitions in their modules can go ahead and import them. Remember the "ietf-interfaces" module that we took a look at in the initial part of this article. Notice how it imports the ietf-yang-types module and assigns a prefix "yang" to it. This the convention mostly followed when one imports the "ietf-yang-types" module:-

You can take a look at the "ietf-yang-types" module here and go through the derived datatypes the writers have created.

When we want to utilise one of the derived types from the imported module we should append the prefix for that module followed by a colon ":" to uniquely reference the definition from that module. Example:-

type yang:counter64

What are leaves?

A leaf node contains simple data like an integer or a string.  It has exactly one value of a particular type and no child nodes.

The YANG language provides two constructs that can hold data namely the leaves and leaf lists. First, we see an example of a leaf definition and an XML instance of the leaf called "enabled".

leaf enabled {
	type boolean;
	default true;
}

<enabled>false</enabled>

A leaf node can have atmost only one instance but a leaf-list can have multiple instances. Here we see an example of leaf-list definition followed by an example XML instance:-

leaf-list cipher {
	type string;
}

<cipher>blowfish-cbc</cipher>
<cipher>3des-cbc</cipher>

Then what are non-leaf constructs?

Non-leaf type constructs are those that can have references to child nodes but cannot take in a value. Only leaves can take in value. So what are the non-leaf constructs? They are containers and lists.

Let us see about containers. A container mode is used to only group some other nodes (child nodes). So what are all the options for child nodes? A container can contain leaves, leaf lists, the container itself or lists(we haven't seen lists yet). Remember, a container can only have child nodes but cannot take a value on its own. Let us take a look at a definition of container followed by a sample instance.

container system {
         container login {
             leaf message {
                 type string;
                 description
                     "Message given at start of login session";
             }
         }
     }

See how the container named "system" references another container named "login". This is a valid definition. Further, the container "login" references a leaf child node that will contain the value. Let us see a sample XML instance of the container defined above:-

<system>
       <login>
         <message>Good morning</message>
       </login>
</system>

See how the string "Good morning" is contained within the leaf named message.

So then what is the other non-leaf type? It's the list. If you are familiar with python you can think of a dictionary object at the back of your mind to augment the understanding of this yang construct. In the likes of a container a list cannot have value but can have child objects.  The child objects can be a leaf or list or a container (any number of them) . In a python dictionary, how do you uniquely identify an object inside the dict object? We use a key to reference an object and get its value. Similarly, the list construct also allows us to define the key attribute.

Let's quickly look at an example to understand better:-

 list user {
         key "name";
         leaf name {
             type string;
         }
         leaf full-name {
             type string;
         }
         leaf class {
             type string;
         }
     }

In this list called user, we have defined multiple child objects (In this case three leaves have been defined). But out of those, the leaf called "name" has been defined to be the key attribute for this list. Let's take a look at a sample XML instance:-

<user>
       <name>glocks</name>
       <full-name>Goldie Locks</full-name>
       <class>intruder</class>
</user>
<user>
       <name>snowey</name>
       <full-name>Snow White</full-name>
       <class>free-loader</class>
</user>

In this case, multiple entries of this list can be uniquely identified by the "name" attribute.

We have come a long way in understanding the basic constructs, data types and parts of the yang modelling language. It is always tough to learn theory, isn't it ? The last two parts of this series have been heavily theoretical but I am sure it will help us in understanding the working aspects of Cisco NSO. We have not dwelled into the complete intricacies of yang modelling language; we will look into it further in future if there is a need.

In the next article, we will look into some practical aspects of NSO, start getting our hands dirty. Do let me know your suggestions and comments. If you come across any technical inaccuracies please point them out, I will do my best to correct them as soon as possible. It will help many others in the community. Thanks.  

In the previous article, we saw, in brief, the parts that made up the NSO. It looked something like this:-

We have the northbound interfaces with which the operator could interact (small blue boxes at the top). We have the core components of NSO like the Service and Device managers which sort of helped us in abstracting the different services and devices we use in day to day life. And finally, we have the NED which helps us in processing the southbound interaction with the network devices. If the Network device did not natively support NETCONF then NED helps in modelling the interaction with the device either through native CLI or SNMP etc.

How do we intend to consume these NorthBound Interfaces?

The NSO CLI (Yeah NSO has its own CLI) and the WEB UI is primarily meant for us Network Engineers. It is more human-oriented.

The NETCONF and RESTCONF are used primarily for interaction with other platforms, think of machine to machine interaction. (OSS/BSS). Example:-NSO receiving instructions from Ansible to configure a bunch of devices.

SNMP might be used for monitoring the NSO system itself. Again these are possible use cases, but everyone uses the Northbound Interfaces differently. If you have a use case that differs do comment.

It's time to get into YANG & NETCONF

The RFC 3535 insisted on developing a programmatic interface for device configuration. We need to be able to differentiate between data involved in configuring devices (Think config statements) vs data involving the state (Think show commands) of the network devices. The RFC also insisted on the ability to configure services (say for example L2VPN)  without being all concerned about the devices involved (say the PE devices or the CE devices. Given the fact that the config that goes into these devices for successfully configuring the L2VPN service might be different ). And most importantly if there is a problem when committing the config changes to devices I need to be able to revert those changes in an appropriate order to restore the state.

THE NETCONF PROTOCOL and YANG - MODEL DRIVEN PROGRAMMABILITY

Simply put, NETCONF is the transport protocol for interacting programmatically with our network devices. The protocol and its intricacies figured in RFC 4741 & RFC 6241. A slight google search would reveal that there is a gap of 5 years between the RFCs. Also, the RFC for YANG RFC 6020 was released somewhere in 2010 which was out there because the community felt there was a need to streamline the data that is carried by the transport protocol (NETCONF) when interacting with devices. So, after the initial RFC for NETCONF in 2006, some changes were incorporated to NETCONF to include the modalities of YANG. The modified version of NETCONF RFC was out in 2011.

That begs the question " Is NETCONF the only protocol that could carry/transport YANG data to and out of devices ?" Protocols like RESTCONF and gRPC (It is again an open-source project from Google) could also be used to transport YANG data, but it depends upon the use case. Like gRPC is now heavily being used in the network community to transport Streaming telemetry data. The latest versions of IOS XR do support telemetry with gRPC to transport live statistics of the device(through YANG data models of course).

It helps to picture mentally picture the stack of what all can interact with YANG data. Here is a picture that helps us understand:-

So NETCONF is the protocol that carries YANG data. Drawing parallels to SNMP, NETCONF can be compared with the SNMP (the protocol itself) and YANG can be compared to the MIBs.

YANG - Yet Another Next Generation (Data/Data Model/Modelling language)

Let us ask ourselves a few questions to understand the YANG.

Why do I need a data model?

Data models typically describe our objects of interest called Entities. Suppose we own a Grocery store. We might have several products. So products are entities. There are so many ways to describe a product. They are the attributes of the product. Suppose I wish to standardise how I want to describe a product. Why do I need to standardise in the first place? Say I am procuring stuff from a wholesale dealer. He might describe his products in a different manner. When conveying to him there can be a misunderstanding. So between the grocery store and the wholesale dealer when data is exchanged for procurement wouldn't it be beneficial if they agree on the attributes about their entities of interest (products). Does it make sense?

So if we are talking about an interface of a network device (that is our entity) we need to make sure the attributes are agreed upon. How do we describe those attributes? How do we say " okay, an interface has the following attributes - It has speed in Mbps, bandwidth in Mbps, Input error count etc" - the YANG modelling language in this case helps us describe the attributes. Once we pen down the attributes it becomes a YANG data model. The actual value of the attributes we described earlier becomes the YANG data.

As a network engineer, what can we use YANG for?

To model device data and to model service data. An example of device data would be modelling interface data. An example of service data modelling would the one that describes a VRF or say a Layer 3 VPN.

Where do the models come from?

YANG models can be of two types. The industry-standard ones that are written by bodies such as the IETF( Github repo for yang models), ITU etc and the vendor-specific ones that are written by the vendors specific to their devices or Operating systems.

How do we represent a YANG data model?

A yang model can be described in the YANG language but most commonly it is described in an XML or JSON format. When we are looking from a NETCONF point of view the encoding supported is just the XML.

How to look into the tree structure of a yang file?

Pyang module helps us visualise the tree structure. Lots of tutorials available for Pyang visualisation. This brief video by Knox on YANG includes looking up tree structure using pyang.

How is a YANG model structured?

It is a hierarchical model of nodes. We can have a container to group the nodes. If I need to sequentially identify a set of nodes I put them in what's called a list. A node might have multiple attributes. So, each individual attribute is denoted by a leaf. A leaf has to have an associated data type.  Whoa. Let's take a pause and analyse this picture to understand the components of the hierarchical model. We will take a deeper dive into the YANG model in the next article, now that we have understood the need.

Again this has just been an introduction to NETCONF and YANG and there are more pieces in understanding the structure and the protocol itself. There might also be a question in our mind; "Is this series about NSO or YANG?"; why do we need so much of NETCONF and YANG for NSO. I think a clear understanding of NETCONF and YANG will help in understanding various concepts of modelling used in NSO.

Please do drop in your comments and suggestions.  

You will find a lot of literature everywhere on what is Cisco's NSO tool. Briefly, NSO aims to provide an orchestration layer; which can help ease our interaction with both traditional and SDN enabled devices. It has support for multiple vendors and can interact with network devices in multiple ways. We will learn about the features that NSO brings along gradually. The series follows no definite pattern. We will try to understand the need for NSO and then dive into the practical aspects. Let us go over some fundamental topics that we need to know before we jump to NSO.

NETWORK MANAGEMENT STANDARDISATION

We have heard about NETCONF, RESTCONF, and YANG in many presentations. We usually brush them aside thinking now is not the time. The usual thought is "My organization will take time in implementing such changes; my devices don't support them" etc. We give all sorts of excuses to learn them because we network engineers have been heavily comfortable with CLI. But smart people were focused on standardizing the way we manage networks.

RFC 3535

This blog post by the tail-f team gives a very brief read on how the RFC by the standards body IETF became the catalyst for developing the robust NETCONF protocol and how YANG became the popular data modelling language. This mention is just to take you through history; there is no need to know too much of NETCONF and YANG stuff.

Also, we need to understand the drawbacks of our existing tools (CLI, SNMP). CLI has always been proprietary stuff. You spend years and tears (when typing I wanted to write years and years, but later I thought tears are more appropriate) learning the CLI of one vendor; you are tied to it. Say you are a Cisco expert. A company that acquires you to manage their devices can have you on their payroll only to manage Cisco devices. What if later their purchase team says there are more features in Juniper devices (just an example) and deploys Juniper devices as part of an upgrade. They will have to spend more cost on acquiring resources, network engineers to manage Junos devices. Bottom-line - CLI is expensive.

SNMP has been very successful in allowing us to poll statistics from devices but honestly, it has not been successful in letting us do config changes. I may be wrong but the farthest  I have seen do are ACL changes on interfaces. So no CLI and no SNMP. What next?

CLI Scripting

I think we all by now would have tried our hands in CLI scripting.  Elementary python knowledge would arm us with the firepower to burst some show commands to our devices, wait for a timer to expire, parse the reply from devices using the very many string functions that come with python and get some usable output. Or even more, we tried our best to enter into the config terminal and push some config changes. We then had questions like was it implemented successfully? How do I push to multiple devices at the same time; should I use concepts of multi-threading etc. It is plagued with problems. But for anyone to understand the problems of Adhoc scripting you should get your hands dirty in it. Try using libraries such as paramiko to help jumpstart your ssh sessions and try building some use cases.

NSO - NETCONF & YANG

We spoke about the drawbacks of CLI and SNMP briefly to understand why NSO uses NETCONF and YANG to interact with devices southbound to get onboard in the standardization train. It doesn't mean NSO will not use CLI. We will discuss the multiple ways NSO interacts with devices in future. NSO can interact with both physical and virtual devices; can interact with multiple vendor implementations; act as sort of an SDN controller for multiple technologies like Service Provider, Data Centers etc. Standardizing the interaction between a controller and a network device which we saw earlier --> delivers, aids in the platform-vendor-agnostic features of NSO.

NEED FOR CONFIG DATABASE

We all have felt the need for a config database. Haven't we? When we see our software counterparts using git in and out we have felt why network implementations have not ingested such concepts. We saw some of these concepts come up in the latest routers  where we are asked "Do you want to commit?" Take for example IOS XR. We can guess where this concept comes from now I guess. Still, it was the responsibility of the individual devices to maintain the database and roll back ids. You could go back in the config state based on the IDs. But what about a central database in sync with the device configurations. We didn't get that. The closest we could get was Network Config Management tools that scraped the config from the devices at fixed intervals of time (say a day) and did the job of storing it in a database as a backup.

The problems with such config backups are many. Imagine you want to go back to a state say the 05th of May. You open the config tool and it gives you the config that was available on the 05th of May. There could be multiple changes done after the 05th of May on that device. How do you as a network administrator approach this problem of reverting the state to that of 05th May? We open our notepads copy the config, compare it line by line and apply our mind to negate the config and restore the config as it was on 05th of May. Keep this in the back of your mind as we progress along.

Cisco NSO has something called the CDB (Configuration Database) which is an in-memory database (If you want to know more about in-memory check this link here) and uses the YANG model for transactions. This will serve as our single Source of Truth (SoT). NSO will try to query our devices periodically; try to fetch the config and maintain an in-sync config state in the CDB. Whoa! When it queries the devices it can use multiple means; it can use CLI, it can use NETCONF but NSO does its dutiful job of maintaining the CDB; most importantly an in-sync CDB.

NORTHBOUND VS SOUTHBOUND

Northbound interactions comprise all interactions between the user and NSO. NSO supports multiple technologies for this northbound interaction. Some include REST , NETCONF , CLI (NSO has its own CLI), Web UI etc.

Southbound takes care of all interaction between our NSO and network devices. NED(Network element drivers) are written for multiple vendors and multiple OSs. For example, there is NED for Cisco ios and another for Cisco ASA.

CORE COMPONENTS

NSO consists of the following core components:-

A service manager, a device manager and a Configuration Database.

The device manager has the role of simplifying device interactions; so obviously talks to devices via previously discussed NEDs. CDB is the database where all the latest and greatest config lies. It is like an amplified git repo for network config. The service manager is the one we have to wrap our brains on. It has the task of applying service changes. Think of service as a Layer 3 VPN config. The service layer would handle it for you. Between the service layer and the device layer, we have a mapping mechanism that sorts of tells the device layer what to apply to the end devices.

This is the end of part 1. We will gradually build our NSO concepts in upcoming posts.

We all have longed a website for ourselves, haven't we? In this post, we look at how do we set things up to have a Ghost blog up and running on an Ubuntu server hosted on GCP free tier in no time. I have consolidated information from various articles of similar nature. All such sources have been given due credits in the reference section

GOOGLE CLOUD FREE TIER

Step 1 - Set up a fully functional GCP account; with all payment info. If you are just starting with GCP the differences between free credit and always free tier can confuse you. Free credits give you the initial boost whereas always free tier allows you to run specific resources for free for a fixed number of hours in a month. In our case, we are interested in the Compute Engine resource. We should be choosing the resource from the region and zone entitled to always free tier.

GCP free tier allows us to run an f1-micro instance for free. Under 'General Purpose' category select N1 Series and then select f1-micro.

Select the desired OS; In my case, I have opted for Ubuntu 21.04 OS. Be careful when selecting the type of disk. The default option might be the balanced type ; but it doesn't come under the free tier criteria. We need to select the standard persistent disk and we can go up to 30GB.

Once the VM is up and running we should do the following basic steps to prepare our VM

• Note down the Public IP of the VM that you have spun up.
• Generate your key pair using putty gen. Add your public key data to GCP meta data.
• SSH into your VM using your private key.
• Perform apt-get update and apt-get upgrade

CREATE SWAP FILE FOR BETTER PERFORMANCE

As you might reckon, we have chosen an f1-micro for our blog which has limited RAM available. It is recommended to create a swap file for better performance.

Courtesy: http://www.theappliedarchitect.com/

You would want to make the changes to the swap file permanent.

sudo nano /etc/fstab

Edit the file and append /swapfile swap swap defaults 0 0 at the end of the file.

INSTALL DOCKER

Step 2. The easiest way to get docker for your VM is to utilize the convenience script prepared by the docker team.

You can also choose to do the installation manually by following the steps mentioned on this page.

PREPARING YOUR DOMAIN

Step 3. You most probably have a registered domain with you. Quickly head to the area where you can key in the DNS servers or nameservers for the domain. Many domain registrars might charge an additional cost for letting you key in TXT /MX records, so it would be beneficial to change the name servers entirely to some free service like Cloudflare. If you are comfortable changing the records from the DNS server of the registrar; you can choose to do so.

In my case, I have opted to create a free account in Cloudflare and pointed my domains name servers to that of Cloudflare's. This will take some time to propagate. Cloudflare will let you know through an email when your domain is ready. During setup, you can choose caching options i.e if you want to proxy your sites traffic through Cloudflare's CDN or just opt for its DNS services. I have opted for the proxy as this would reduce the latency drastically. You can check that for yourself once you have done this configuration.  If everything is okay you should have your landing page in Cloudflare looking like this:-

CREATE A MAILGUN FREE ACCOUNT

Step 3. After hosting a blog, you might want to send bulk emails; dispatching your well-written articles to everyone. The integration with Mailgun is the easiest. but should you chose to go with any other provider you are welcome to modify the code. After creating a free account you need to add appropriate records to your DNS server. Detailed documentation on the DNS records required for Mailgun can be accessed here. The document is self-explanatory. You will be required to add TXT records to be able to send mails from your domain. MX records are required to receive mail. ( We will not be using Mailgun to receive emails for our root domain) . I will explain how I had ZOHO always free tier set up for sending and receiving emails for our root domain. Once you key in all the relevant records and allow them to propagate; you will receive an email from Mailgun stating that you are now ready to start sending emails.  

PREPARE CONFIG FILES FOR GHOST

Step 4. Make a new folder for Ghost. This would contain all the resources relating to our blog, making it easy for us to back up.

mkdir ghost_blog

Step 5. Prepare config files. I should give credit to http://www.theappliedarchitect.com/ from where I got the template for this config file. I have done some modifications to the template to also include the SMTP setting for our Mailgun account. You can get your SMTP credentials under domain settings in the Mailgun dashboard.

This config file should be inside the ghost_blog folder that we have created already. The port settings can be modified according to your needs. I have opted to leave it at 2368 as I will be using a Nginx front end before the docker container.

I will reference a shell script prepared by http://www.theappliedarchitect.com/ that will help us do basic docker command functions from the shell. This will be useful to restart the container after reloads etc.

This script can be placed outside the ghost_blog folder. If you read through the shell script you will understand why.  Note how I have used HTTPS everywhere. This would come in handy when you complete the HTTPS setup with Nginx, Let's Encrypt and Certbot. Before you run this script and kick start your docker container we need few more things installed and set up.   In part 2 of this article, we will look into setting up Nginx, dispatching our mails through mail gun (through API & SMTP); setting up the member option in GHOST which will enable your visitors to sign up as members and receive your newsletters.

Hope it was a good read. Please let me know through comments if it helped setting up your blog. Bye.

Let us be real. We net-workers refer lots of documentation on the Internet to do our day-to-day jobs. We try to browse and peruse through tons and tons of PDFs, OEM documentation and research about our use cases. Often, we do not find clear illustrations, that jumps straight to the point; but when we find them out it offers a very cool experience to every implementer/technologist.

The joy of finding a blog / article where the author had thought just like we did is surreal. This blog aims to (at-least attempt to) give articles that are precise, concise and real-world implementation oriented.

While the traditional networking constructs in various domains is transitioning to Software-Defined bandwagon; many forget that in-spite of the fancy GUI and automation codes& scripts our final endeavor is to route and switch. Let us route and switch them packets! Happy reading!