There are many cluster networking options for Kubernetes. 2 of the most popular are:
Flannel is a simple and easy way to configure a layer 3 network fabric designed for Kubernetes. It is also used by default by Oracle Container Services for use with Kubernetes (aka Kubernetes on Oracle Linux) and by Oracle Container Engine (OKE).
Calico provides both a layer 3 networking and a network policy engine. Its policy engine can also be used together with flannel.
In this post, we will deploy Calico for network pod policy.
Manually installing Calico
If you have manually created the OKE Cluster using the cli or the OCI console, obtain the kubeconfig file and set your KUBECONFIG environment variable:
Download the Calico policy-only manifest for the Kubernetes API datastore:
By default, the pod CIDR block on OKE is 10.244.0.0/16. Set this as an environment variable:
Then replace the default pod CIDR block value (192.168.0.0/16) in the calico.yaml. You can skip this step if your pod CIDR block is 192.168.0.0/16.
sed -i -e "s?192.168.0.0/16?$POD_CIDR?g" calico.yaml
If your cluster consists of more than 50 worker nodes, then you need to do 1 additional step:
sed -i -e 's/typha_service_name:\s"none"/typha_service_name: calico-typha/g' calico.yaml
Apply the manifest:
kubectl apply -f calico.yaml
Calico also recommends a minimum of 3 replicas in production environment and 1 replica per every 200 node:
kubectl -n kube-system scale --current-replicas=1 --replicas=3 deployment/calico-typha
The installation steps and other recommendations can be viewed on the Calico website.
Installing Calico when provisioning with terraform-oci-oke module
If you are provisioning your cluster with the terraform-oci-oke module, there is an option to automate its installation. Set the following variables in your terraform.tfvars file
create_bastion = "true"
install_calico = "true"
and run terraform apply again:
terraform apply -auto-approve
The calico installation script in terraform-oci-oke also handles the cases when you have more than 50 nodes in your cluster and and the number of replicas needed are calculated and scaled to accordingly.
If you want to test calico as a network pod policy engine, there are some very excellent recipes here. You should be able to take them all for a spin.
Alternatively, you can also try the tutorials on the Calico website.