Installing AVI Kubernetes Operator (AKO) on Openshift Container Platform 4.5
1. Topology
In this lab environment, I have 2 sites, Site-01a and Site-01b. In each site I have:
- 1x AVI Controller
- 1x vCenter Server
- 1x Openshift Cluster with 3x master node and 3x worker node
- 1x Linux host to bootstrap and do most of the stuff
There are several Service Engines created to do the load balancing job. There are two Service Engine Groups, one group for DNS and the other group for the ingress controller.
Here is the topology I used in this lab environment.
2. Requirement
Software Requirement
The setup was created with below software version:
Prerequisites
-
AVI Controller has to be setup with vCenter cloud
-
Make sure the Port Group of the Openshift nodes is configured in IPAM profile and the network has IP pool
-
If the Pod CIDRs are not routable, we need to create VRF context object in AVI for kubernetes controller and configure Port Group network with VRF context.
3. Install Helm
The guide on installing Helm can be found here
curl https://baltocdn.com/helm/signing.asc | sudo apt-key add -
sudo apt-get install apt-transport-https --yes
echo "deb https://baltocdn.com/helm/stable/debian/ all main" | sudo tee /etc/apt/sources.list.d/helm-stable-debian.list
sudo apt-get update
sudo apt-get install helm
4. Installing AKO
Step 1: Get values.yaml file from avinetworks github
mkdir ako
cd ako
wget https://raw.githubusercontent.com/avinetworks/avi-helm-charts/master/charts/stable/ako/values.yaml
Step 2: Create the avi-system namespace
kubectl create ns avi-system
Step 3: Add AKO repository
helm repo add ako https://avinetworks.github.io/avi-helm-charts/charts/stable/ako
Step 4: Search for available charts
helm search repo
NAME CHART VERSION APP VERSION DESCRIPTION
ako/ako 1.2.1 1.2.1 A helm chart for Avi Kubernetes Operator
Step 5: Install AKO
helm install ako/ako --generate-name --version 1.2.1 -f values.yaml --set configs.controllerIP=<avi-controller-ip> --set avicredentials.username=<avi-ctrl-username> --set avicredentials.password=<avi-ctrl-password> --namespace=avi-system
Step 6: Verify the installation
$ oc get all -n avi-system
NAME READY STATUS RESTARTS AGE
pod/ako-8ff7fbdc-sb55x 1/1 Running 0 21d
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/ako 1/1 1 1 21d
NAME DESIRED CURRENT READY AGE
replicaset.apps/ako-8ff7fbdc 1 1 1 21d
5. Verification
After AKO installation, a new Virtual Service is created on the AVI Controller. This is Layer-7 Ingress service.
6. Deploy Demo App
Online Boutique is a cloud-native microservices demo application. Online Boutique consists of a 10-tier microservices application. The application is a web-based e-commerce app where users can browse items, add them to the cart, and purchase them.
This application can be found on its Github Page
The example apps above does not include ingress service, so I add ingress service as per yaml below:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: frontend-ingress
labels:
app: avi-gslb
spec:
rules:
- host: shop.apps.corp.local
http:
paths:
- path: /
pathType: Prefix
backend:
serviceName: frontend
servicePort: 80
I have ingress on subdomain shop.apps.corp.local and it is redirecting to frontend service.
below are the result:
$ oc get pods
NAME READY STATUS RESTARTS AGE
adservice-86bc987ccd-x2n2q 1/1 Running 1 11d
cartservice-c877477df-qz2rp 1/1 Running 7 5d13h
cartservice-c877477df-vjzzj 1/1 Terminating 2 11d
checkoutservice-cf8cf75db-45qzc 1/1 Running 0 11d
currencyservice-6c596c8df8-bw22b 1/1 Running 1 11d
emailservice-d86586496-8xgjh 1/1 Running 1 11d
frontend-d6f4f8984-fft7g 1/1 Running 0 15h
frontend-d6f4f8984-htcx2 1/1 Running 0 15h
frontend-d6f4f8984-n85jw 1/1 Running 0 15h
frontend-d6f4f8984-rpggn 1/1 Terminating 0 11d
frontend-d6f4f8984-v5z9z 1/1 Running 0 15h
frontend-d6f4f8984-zc8jv 1/1 Running 0 15h
loadgenerator-558d6c8d85-4cwt8 1/1 Terminating 6 11d
loadgenerator-558d6c8d85-bvww2 1/1 Running 2 5d13h
paymentservice-7cb9cfd8b8-5jqrq 1/1 Running 0 5d13h
paymentservice-7cb9cfd8b8-n7zlv 1/1 Terminating 1 11d
productcatalogservice-7bb4c9868-96khz 1/1 Running 0 5d13h
productcatalogservice-7bb4c9868-lcmj2 1/1 Terminating 0 11d
recommendationservice-df4dc9bfb-759qh 1/1 Running 5 11d
redis-cart-659df7674c-8djjz 1/1 Running 0 5d13h
redis-cart-659df7674c-9khtb 1/1 Terminating 0 11d
shippingservice-df95d5484-86gqs 1/1 Terminating 0 11d
shippingservice-df95d5484-vh5pr 1/1 Running 0 5d13h
holuser@ubuntu-01a:~$ oc get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
adservice ClusterIP 172.30.153.106 <none> 9555/TCP 11d
cartservice ClusterIP 172.30.61.177 <none> 7070/TCP 11d
checkoutservice ClusterIP 172.30.130.82 <none> 5050/TCP 11d
currencyservice ClusterIP 172.30.129.103 <none> 7000/TCP 11d
emailservice ClusterIP 172.30.53.143 <none> 5000/TCP 11d
frontend ClusterIP 172.30.49.167 <none> 80/TCP 11d
frontend-external LoadBalancer 172.30.19.147 192.168.243.21 80:31315/TCP 11d
paymentservice ClusterIP 172.30.206.8 <none> 50051/TCP 11d
productcatalogservice ClusterIP 172.30.191.141 <none> 3550/TCP 11d
recommendationservice ClusterIP 172.30.98.64 <none> 8080/TCP 11d
redis-cart ClusterIP 172.30.250.199 <none> 6379/TCP 11d
shippingservice ClusterIP 172.30.221.246 <none> 50051/TCP 11d
$ oc get routes
NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD
frontend-ingress-k4x5c shop.apps.corp.local ... 1 more / frontend http None
I can look at specific pod in frontend deployment, this should be my pool member
$ oc get pods -o wide | grep frontend
frontend-d6f4f8984-6vnzj 1/1 Running 0 15m 10.130.2.25 ocp01-grq8j-worker-w26bz <none> <none>
Below is my Virtual Service construct in my controller. The ingress configured in the application is registered as host URL in my Virtual Service. There is a HTTP rule to specify this in HTTP header. All created automatically from AKO.
Source
https://github.com/avinetworks/avi-helm-charts/blob/master/docs/AKO/README.md#parameters