1. Prerequisite

This lab is created with below software version:

• NSX Advance Load Balancer (AVI Networks) v18.2.7
• Openshift v4.2.16

The resources required for Controller as well as Service Engines can be found here for Controller and here for Service Engine

The diagram for the setup is as below:

2. Setup Authentication to OpenShift

• Create Service Account for AVI

  {
  "apiVersion": "v1",
  "kind": "ServiceAccount",
  "metadata": {
    "name": "avi"
    }
  }
  

  oc create -f sa.json
  

• Add Service Account to Cluster-Admin role

  oc adm policy add-cluster-role-to-user cluster-admin -z avi
  

• Extract token for use in AVI cloud configuration

  oc serviceaccounts get-token avi
  

3. Configure IPAM, DNS Profile, and Provision DNS Virtual Server

• Configure DNS Virtual Server More detailed information about AVI DNS architecture can be found here

  DNS Virtual server responsible to answer DNS queries for delegated domain. The delegated domain is ingress.openshift4.corp.local and this is used for ingress traffic.

  To create the DNS Virtual Service, Advanced Virtual Service is required. Select “Create Virtual Service”

  Select “System-DNS” as the application profile

  AVI-DNS Virtual Service will be shown in the dashboard. The Virtual IP address is used as NS record for domain delegation purpose.

• Configure IPAM Profile IPAM will be used to maintain ingress IP addresses and will be represented as Virtual Server. Below is the configuration of IPAM Profile

• Configure DNS Profile

4. Configure Openshift Cloud in AVI

• Choose Openshift/Kubernetes Cloud, and then fill in the information as below:

• Choose to deploy Service Engine manually, I’m going to deploy Service Engine as a VM instead of as a pod.

• Add IPAM and DNS profile

• After integration to Openshift, you should see the namespaces in Openshift cluster registered as Tenant

5. Create Service Engine VM for Openshift Cloud and Register to Controller

Deploying Service Engine as a VM has to be done manually, however the Openshift cloud is not “no-orchestrator” mode, hence the token has to be generated manually as well. This token will be used by Service Engine to register itself into the Openshift cloud.

To generate a token for Openshift, I will do API call to generate the token. However, HTTP basic authentication has to be enabled for this API query.

Detailed info about how to enable HTTP basic auth can be found here

• Enable basic authentication

  $ shell
  [admin:192-168-110-71]: > configure systemconfiguration
  

  Updating an existing object. Currently, the object is:
  +----------------------------------+------------------------------------+
  | Field                            | Value                              |
  +----------------------------------+------------------------------------+
  | uuid                             | default                            |
  | dns_configuration                |                                    |
  |   server_list[1]                 | 192.168.110.10                     |
  <truncated>
  | portal_configuration             |                                    |
  |   enable_https                   | True                               |
  |   redirect_to_https              | True                               |
  |   enable_http                    | True                               |
  |   sslkeyandcertificate_refs[1]   | System-Default-Portal-Cert         |
  |   sslkeyandcertificate_refs[2]   | System-Default-Portal-Cert-EC256   |
  |   use_uuid_from_input            | False                              |
  |   sslprofile_ref                 | System-Standard-Portal             |
  |   enable_clickjacking_protection | True                               |
  |   allow_basic_authentication     | False                              |
  |   password_strength_check        | True                               |
  <truncated>
  +----------------------------------+------------------------------------+
  

  [admin:192-168-110-71]: systemconfiguration> portal_configuration
  
  [admin:192-168-110-71]: systemconfiguration:portal_configuration> allow_basic_authentication
  Overwriting the previously entered value for allow_basic_authentication
  
  [admin:192-168-110-71]: systemconfiguration:portal_configuration> exit
  
  [admin:192-168-110-71]: systemconfiguration> exit
  

  +----------------------------------+------------------------------------+
  | Field                            | Value                              |
  +----------------------------------+------------------------------------+
  | uuid                             | default                            |
  | dns_configuration                |                                    |
  <truncated>
  | portal_configuration             |                                    |
  |   enable_https                   | True                               |
  |   redirect_to_https              | True                               |
  |   enable_http                    | True                               |
  |   sslkeyandcertificate_refs[1]   | System-Default-Portal-Cert         |
  |   sslkeyandcertificate_refs[2]   | System-Default-Portal-Cert-EC256   |
  |   use_uuid_from_input            | False                              |
  |   sslprofile_ref                 | System-Standard-Portal             |
  |   enable_clickjacking_protection | True                               |
  |   allow_basic_authentication     | True                               |
  |   password_strength_check        | True                               |
  <truncated>
  +----------------------------------+------------------------------------+
  

• Do API Call to generate the token

  curl -X GET -u 'admin:VMware1!' 'https://avi-controller01.corp.local/api/securetoken-generate?cloud_uuid=cloud-61b9151a-b184-4dd6-82f9-7ca94fa1d665' --insecure | grep auth_token
  

  "auth_token": "1cd039a6-4fa8-4b48-937e-ce722888be04"
  

• Deploy Service Engine OVA into vSphere

  Wait until the initialization completed and fully registered in the controller

Caveats

• Condition vSphere Cloud is in Write mode Openshift Cloud is in Write mode Service Engine is deployed in vSphere

• Symptomp When the Service Engine VM deployed manually for Openshift Cloud, the SE will be tagged for vSphere Cloud and configured for vSphere cloud.

• Workaround Convert vSphere Cloud into “no-orchestrator” mode, then manually deploy Service Engine into Openshift Cloud

Sources

https://avinetworks.com/docs/latest/avi-vantage-openshift-installation-guide/