Ansible Tower on Vagrant

I am still on the re-install apps land on the macOS, and this is a mini guide on how to install Ansible Tower using Vagrant for demo/trial usage only.

The first step is to install Vagrant if not already installed for other means. Vagrant relies on interactions with 3rd party systems, known as “providers”, to provide Vagrant with resources to run development environments.  I am running VirtualBox

To verify the installation of both Vagrant and VirtualBox:

vagrant --version

vboxmanage --version

Once the installation of both Vagrant and VirtualBox are completed, Ansible Tower can be initialized by creating a Vagrantfile with default instructions in the current directory as follows:

vagrant init ansible/tower

vagrant up

The process takes a few minutes the first time, and once complete:

vagrant ssh

The vagrant ssh command will give you your admin password and the Tower log-in URL ( This is using the default (basic) settings in Vagrantfile and it can be edited further, including a more specific name for the ansible VM.

To verify the Ansible version:

ansible --version

At the moment, there are two trial/demo licenses available: one with enterprise features such as LDAP and Active Directory support, System Tracking, Audit Trails, Surveys, and one limited to 10 nodes and no expiration date, however, it does not include the enterprise features just listed. The open source alternative (or non-enterprise version) with no node limitation is the project AWX.

Below is the main (default) dashboard of Ansible Tower:

ansible tower

And here is a nice walk-through on the GUI: Ansible Tower demo.

Tip: if by any chance can not be accessed the first time, check the routing table (ip r) and interfaces (ip a show) to see if is listed on the Vagrant VM. If not listed, reinstall everything. 

Apstra in a Whiteboard

As an occasional and very ordinary writer, I think this topic deserves a “direct from the source” approach. I am definitely using credits from my blank slate bucket. In other words, nothing I write would be better than reading from who explained in a whiteboard at Networking Field Day 19 so simple and easy to understand what the Apstra AOS (Apstra Operating System) is all about and its building blocks.

Besides doing a great whiteboard session, @_vCarly also published an outline of that very busy morning at Apstra’s NFD19 Experience. It is a detailed narrative that goes from the reference architecture made of the AOS server sitting at the orchestration (or management) layer and agents installed on each individual switch (supporting modern leaf-spine designs or extensible to other environments) to the building blocks: logical device, rack type, template, blueprint, interface map, device profile, resources, and managed devices. They are all interconnected and it makes more sense when delving into the whiteboard. There is no better way to get a clear understanding of Apstra other than watching the original video followed by her narrative.


On a side note, this is someone who I met in person for the first time, but who I have known for a while through videos as part of my initial Cisco ACI learning journey. I just wish my whiteboards were that decent and inspirational.

In addition to the whiteboard session, other highlights were around the ServiceNow integration delivered jointly with Network to Code, an overview and demo of Day 2 Operations via IBA (Intent-Based Analytics) with a write-up here as well, and a demo on AOS for additional context. The original videos are available at the Networking Field Day 19 portal.

NSX-T Logical Routers

Between a few VMworld 2018 sessions and a recent NSX-T Bootcamp, I believe I collected enough information to describe at a high level the new logical routing scheme within NSX-T. The interest is also being driven by an internal project.

The intent is to continue to “route as close as possible to the source” as all routing and switching is being done at the host level in software within the NSX overlay architecture, while the underlay infrastructure provides only transport and external connectivity.

Logical Routers Components

NSX-T has two logical routers components, namely the Services Router (SR) and the Distributed Router (DR). As the names imply, SR is where centralized services are provisioned such as NAT, DHCP, VPN, Perimeter Firewall, Load Balancing, etc., and DR performs distributed routing across all hosts participating in a given transport zone. This is very similar to the Distributed Logical Router (DLR) in NSX-v, except that there is no need for a DLR Control VM or dynamic routing protocols between DR and centralized services.

Apart from the logical router components being named SR and DR, the actual logical router naming convention configured within the NSX-T Manager is “Tier 0” and “Tier 1” router as described further when deploying single or two-tier routing/topologies.

Figure 1 is a conceptual diagram illustrating the SR and the DR placement within the NSX domain, for north-south (external networks) and east-west (internal networks) traffic respectively.

Screen Shot 2018-12-14 at 11.24.51 PMFigure 1 – Conceptual Design

Single Tier Topology

In a single tier topology, both SR and DR are known as Tier 0 Logical Router. In this architecture, upon creation of a Tier 0 Router with downlink interfaces to logical switches, Tier 0 Distributed Routers are automatically pushed to all Transport Nodes (compute hosts) participating in a transport zone. A Tier 0 DR instance is also automatically added to the Edge Node with the SR, which is instantiated the moment a service is enabled. By default, the link between the SR and the DR uses the subnet and is auto-plumbed by NSX-T Manager. A default route is created on the DR with the next-hop pointing to the SR, and the connected routes of the DR are programmed on the SR with a next-hop pointing to the DR.

For east-west traffic, a packet coming from a virtual machine behind a DR to a virtual machine in another logical switch (same or different compute host) is routed at the local DR, same goes for the returning traffic, which is routed at the local DR first. For north-south traffic that traverses the Edge Node, the packet is also routed at the local DR first, and the returning traffic is routed at the local DR residing in the Edge Node before it is encapsulated and sent back to the source.

There is a lot that happens in the background, but from the perspective of the Tier 0 DR router, the logical switches are directly connected as south-bound switches, and the logical switches only see a single logical router or single routing construct upstream. Figure 2 depicts the physical and logical view with color-coded routers to indicate which one is SR, which one is DR within a single tier topology.

Screen Shot 2018-12-14 at 11.25.02 PM

Figure 2 – Single Tier Topology

Two-Tier Topology

In a two-tier (or multi) routing topology, the fundamentals of SR and DR remains the same, but the logical routers are named Tier 0 and Tier 1 routers and both are instantiated on the hypervisors of each transport node in a fully distributed architecture. The “RouterLink” between Tier 0 and Tier 1 routers is automatically configured with a /31 IP in the subnet range of when the Tier 1 is connected to the Tier 0 router, same auto-plumbing process in the backend by NSX-T Manager. There is no routing protocol running between Tier 0 and Tier 1 routers. The NSX management plane knows about the connected routes on Tier 1 and creates static routes on the Tier 0 router with a next-hop in the subnet range of

As with the single tier topology, the Edge Node also has instances of SR and DR locally, or Tier 0 and Tier 1. The major difference is that even though Tier 1 is being “distributed” across all Transport Nodes, it has tenant isolation from the other Tier 1 routers across the NSX domain. A Tier 1 can be removed from the environment without affecting any other tenant, completely independent (or isolated) given the multi-tenancy nature. Specific services can also be enabled on a Tier 1 router such as Load Balancing or NAT.

If there is a need for inter-tenant connectivity, traffic between tenants traverse the local Tier 1 as well as the Tier 0 routers in the transport node, and packets are routed locally by the Tier 1 before it hits the wire via Geneve encapsulation. Returning traffic is also routed at the local Tier 1 at the remote tenant. For north-south traffic that traverses the Edge Node, the packet is also routed at the local Tier 1 first, and the returning traffic is routed at the local Tier 0 and Tier 1 instantiated at the Edge Node.

Figure 3 depicts the physical and logical view with color-coded routers to indicate which one is Tier 0, which one is Tier 1 within a two-tier topology contained in each tenant.

Screen Shot 2018-12-14 at 11.25.10 PM

Figure 3 – Two-Tier Topology

Which One?

The decision of when and which topology to use will come down to business requirements. In a multi-tenancy environment, of any kind, two-tier routing has its place by providing tenant isolation and independent control over network and security policies. For some, it may simplify management, while for others, it may add complexity. The single tier topology is as “simple” as it could be. If there is no interest (or requirement) in separating routing domains for tenants/groups, then only Tier 0 can be deployed.

Recommended Sessions

These are the VMworld sessions which add more in-depth details with packet walk of the logical routing in NSX-T:

  • NET1127BU: NSX-T Data Center Routing Deep Dive
  • NET1561BU: Next Generation Reference Design with NSX-T Data Center (part 1)
  • NET1562BU: Next Generation Reference Design with NSX-T Data Center (part 2)


Thanks to my friend @LuisChanu who provided invaluable inputs and guidance.

VCSA 6.7 on VMware Fusion

I am in the midst of reinstalling a lot of applications on my laptop after going through a complete reinstall of the macOS a few weeks ago, as direct upgrade to Mojave did not work for reasons I will probably never know.

In this process, I am now having to install all my virtual machines back into Fusion as part of my mini tiny home lab. The first one is the vCenter Server Appliance. After a few hours going through the “known” process of updating the .vmx file with “guestinfo” prior to power it on and not getting anywhere, I decided to ask google and luckly found the solution here posted by Moussa. At the moment this is the only one with guidelines for v6.7 that I managed to find. All others relate to v6.5 and still uses the manual editing of the .vmx file. More about earlier versions here.

For VCSA v6.7, here is the step by step summary:

1. Extract the .iso file and load the .ova under the vcsa folder as virtual machine
2. Choose the deployment option (in my case, tiny)
3. Edit the “Networking Configuration” section. This removes the need to edit the .vmx file manually as in previous versions. Sample below:

Screen Shot 2018-11-22 at 3.01.28 PM

4. The network settings can be changed at the initial boot from the default “bridge networking”, if required.
5. Once it is deployed, root password to be set, and remaining configuration to be done via the Appliance Management UI at [https://vcsaip:5480].
6. Important: during the “Set Up” process, the system name needs to match the “Host Network Identity”. If they do not match, it will not start.

Sample below (using IP instead of FQDN):

Screen Shot 2018-11-22 at 3.39.33 PM

Upon completion, VCSA is accessible at [https://vcsaip] with SSO details.

Note to self: always check google first.

Illumio Micro-segmentation at Scale

At Networking Field Day 19, Illumio launched the PCE Supercluster enhancement to their Adaptive Security Platform (ASP) solution, which will allow for a federated multi-region micro-segmentation architecture with centralized policy management and global visibility at scale.

Previously, Illumio presented at Networking Field Day 12, where the focus was an introduction to ASP and their policy model approach to micro-segmentation. Below is a recap of the base architecture and how PCE Supercluster comes into play.

Architecture Recap

The Illumio ASP is delivered in software only, agent-based, and it supports multiple operating systems, containers, network switches (via API calls), and cloud environments (via Security Groups), which is a competitive advantage for enterprises running a multitude of operating systems within compute and networking and are looking for policy consistency across domains, including cloud-based.

The architecture is composed by the Virtual Enforcement Node (VEN), a lightweight agent installed on workloads residing in any data center or cloud, and the Policy Compute Engine (PCE), which is the central brain that collects all the telemetry information from the VENs, visualizes it via real-time application dependency maps which is another must for any micro-segmentation strategy, and then calculates and recommends the optimal firewall rules or security controls based on contextual information about the environment, workloads, and processes. These rules are transmitted back to the VENs, which in turn program hosts, access lists or security groups depending on what is in the scope. The PCE can be deployed via SaaS or on premises.

The diagram below illustrates the major components along with the Application Dependency and Vulnerability maps which are displayed in the PCE.


PCE Supercluster

The PCE Supercluster is extending the ASP capability to provide global application visibility and federated security policies at scale across regions. According to Illumio, it is designed for enterprise-scale and globally distributed data centers. It provides organizations with global visibility into the connections and flows across its multiple data centers and enables them to centralize policies across federated PCEs. Compared to a single PCE, a PCE Supercluster provides multiple independent PCE failure domains and support for a significantly greater number of workloads.

In a PCE Supercluster deployment, one region or site acts as the leader while others act as members. The leader is defined as the master for the policy model (white-listed), and the members contain replicas of the policy model. In other words, all PCEs in the PCE Supercluster have the same information. The policy provisioning is always through the leader. All traffic between PCEs are encrypted via TLS.

Illumio leverages a role-based access control (RBAC) model to assign Application Owners, Audit, Security, and IT Ops the least required privilege they need to perform their jobs. This helps preventing unwanted changes across PCEs.

Below is a 3 regions deployment sample, which was also the topology used during the demo sessions.



Illumio’s presented 5 demos covering Global Visibility and Policy Propagation, Global Policy Portability in the case of Application Disaster Recovery, Intra-Region PCE Resiliency, PCE Supercluster Disaster Recovery in the case of Inter-Region, and Vulnerability-based Segmentation. All these demos are available at the Tech Field Day portal.


This was my first actual exposure to their solution and value proposition, and it has the potential to shake things around in a time where visibility and micro level segmentation are becoming so critical to augment perimeter security and prevent spread of breaches inside data centers and cloud environments. Illumio has a great development potential as well, considering the traffic data being gathered which could lead to sophisticated analytics.

Illumio’s Team:

PJ Kirner, CTO and Founder
– Wendy Yale, VP of Marketing
– Matthew Glenn, VP Product Marketing
– Anand Ghody, Technical Marketing Engineer

Matthew and Anand presented the demos and they were so well prepared and extremely creative. For each of the demo presented, a new t-shirt was used with the name on it. The synergy and communication were the highlights and gave us a great example (and reminder) of why team work is so important, and how perception is everything. It is definitely worth spending time getting to know what they are doing and their innovations.

docker for mac – base installation

As with most things, I need to try it out to actually understand how it works. This is a mini version based on the official getting started guide from Docker. There are two community versions available to be selected after installation: stable and edge.

Stable Edge
The Stable version is fully baked and tested, and comes with the latest general availability release of Docker. The Edge version offers cutting edge features and comes with experimental features turned on.

Since I am an absolute beginner, I am going with the stable version. This is the second time I am installing Docker, the difference this time is that I am documenting the installation and more serious about making use of it (eventually). It is an easy-to-install process with complete development environment for building, debugging, and testing “dockerized” apps on a Mac, and no need for Docker Toolbox depending on the macOS version.

The .dmg package will install the following components: Docker Engine, Docker CLI client, Docker Compose, and Docker Machine. The versions can be checked by adding –version:

$ docker --version

$ docker-compose --version

$ docker-machine --version

Prerequisite:  a DockerID created at the Docker Store. This is the same login used to access Docker Cloud, Docker Store, and Docker Hub services.

Docker Architecture

The illustration below depicts the Docker client-server architecture, where “docker” is the client when we input docker commands such as docker build, docker run and so on, and “docker daemon” is the server listening to docker requests and responsible to manage docker objects (images, containers, networks, and volumes). The docker registry is where images are stored, either private or in public registries such as Docker Cloud or Docker Hub – where most base images are available for consumption.

docker architecture

Docker Test

$ docker run hello-world

$ docker image ls

If it works, a message will be displayed indicating all the steps performed by Docker, such as the client contacted the daemon, the daemon then pulled the “hello-world” image from the registry Docker Hub (default) and created a new container from that image which runs the message displayed on the terminal. With the docker image ls, the image can also be seen. If by any chance it does not work the first time, docker login fixes it.

Done, this is how we can install and test Docker for Mac, and have it available for further use.


With the newer releases of Docker for Mac, both Edge and Stable, a standalone (single node) Kubernetes server is included as well so we can deploy Docker workloads on K8s, which is something else I want to play with later on.

the beginning

During my first participation as a delegate at the Networking Field Day 19, I decided to start a blog and try something different. Over the years, I became more of a reader, socially speaking. In here, among other ordinary things, I want to write about tecnologies that I have been exposed to and also new technologies I am learning about. I hope this helps others who are also trying new things, as I have been inspired by a few colleagues in the industry who I admire and been following for years.

To be continued…