Cost optimization

GKE

.https://cloud.google.com/solutions/best-practices-for-running-cost-effective-kubernetes-applications-on-gke

Disable non-prod add-ons in kubernetes

# Disable Horizontal Pod Autoscaling
gcloud container clusters update <cluster-name> \
  --update-addons=HorizontalPodAutoscaling=DISABLED

# Disable Kube DNS
kubectl scale --replicas=0 kube-dns-autoscaler \
  --namespace=kube-system

# Limit Kube DNS scaling
kubectl scale --replicas=0 deployment/kube-dns-autoscaler \
  --namespace=kube-system

kubectl scale --replicas=1 deployment/kube-dns \
  --namespace=kube-system

Pod Disruption Budget

Multitenant clusters

• Utilize multiple namespaces to group & isolate teams or workloads in a GKE cluster

• Role-based access control

• Kubernetes resource quotas

• Configure monitoring dashboards to view resource usage by namespace

Bin packing

.https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/

• Make sure the workload fit well inside the machine size

• We can create multiple node pools and use either nodeSelector or NodeAffinity to select which node your pod must run

GKE autoscaling strategies

Lab: https://www.qwiklabs.com/focuses/15636?parent=catalog

Flow

Scalability dimensions

Configuring pod disruptions - https://kubernetes.io/docs/tasks/run-application/configure-pdb/

Commands in google cloud shell

    1  gcloud config set compute/zone us-central1-a
    2  gcloud container clusters create scaling-demo --num-nodes=3 --enable-vertical-pod-autoscaling
    3  kubectl get deployment
    4  kubectl apply -f php-apache.yaml
    5  kubectl get deployment
    6  kubectl autoscale deployment php-apache --cpu-percent=50 --min=1 --max=10
    7  kubectl get hpa
    8  gcloud container clusters describe scaling-demo
    9  gcloud container clusters describe scaling-demo | grep verticalPodAutoscaling
   10  gcloud container clusters describe scaling-demo | grep ^verticalPodAutoscaling
   11  gcloud container clusters describe scaling-demo | grep ^verticalPodAutoscaling -A 1
   12  gcloud container clusters describe scaling-demo | grep ^verticalPodAutoscaling -A 2
   13  kubectl create deployment hello-server --image=gcr.io/google-samples/hello-app:1.0
   14  kubectl get deployment
   15  kubectl set resources deployment hello-server --requests=cpu=450m
   16  kubectl describe pod hello-server
   17  kubectl describe pod hello-server | sed -n "/Containers:$/,/Conditions:p/"
   18  kubectl describe pod hello-server | sed -n "/Containers:$/,/Conditions:/p"
   19  kubectl apply -f hello-vpa.yaml
   20  kubectl describe vpa hello-server-vpa
   21  kubectl apply -f hello-vpa.yaml
   22  kubectl scale deployment hello-server --replicas=2
   23  kubectl get pods -w
   24  kubectl get vpa
   25  kubectl describe pod hello-server | sed -n "/Containers:$/,/Conditions:/p"
   26  gcloud beta container clusters update scaling-demo --enable-autoscaling --min-nodes 1 --max-nodes 5
   27  kubectl get deployment -n kube-system
   28  gcloud beta container clusters update scaling-demo --autoscaling-profile optimize-utilization
   29  kubectl get vpa
   30  kubectl create poddisruptionbudget kube-dns-pdb --namespace=kube-system --selector k8s-app=kube-dns --max-unavailable 1
   31  kubectl create poddisruptionbudget prometheus-pdb --namespace=kube-system --selector k8s-app=prometheus-to-sd --max-unavailable 1
   32  kubectl create poddisruptionbudget kube-proxy-pdb --namespace=kube-system --selector component=kube-proxy --max-unavailable 1
   33  kubectl create poddisruptionbudget metrics-agent-pdb --namespace=kube-system --selector k8s-app=gke-metrics-agent --max-unavailable 1
   34  kubectl create poddisruptionbudget metrics-server-pdb --namespace=kube-system --selector k8s-app=metrics-server --max-unavailable 1
   35  kubectl create poddisruptionbudget fluentd-pdb --namespace=kube-system --selector k8s-app=fluentd-gke --max-unavailable 1
   36  kubectl create poddisruptionbudget backend-pdb --namespace=kube-system --selector k8s-app=glbc --max-unavailable 1
   37  kubectl create poddisruptionbudget kube-dns-autoscaler-pdb --namespace=kube-system --selector k8s-app=kube-dns-autoscaler --max-unavailable 1
   38  kubectl create poddisruptionbudget stackdriver-pdb --namespace=kube-system --selector app=stackdriver-metadata-agent --max-unavailable 1
   39  kubectl create poddisruptionbudget event-pdb --namespace=kube-system --selector k8s-app=event-exporter --max-unavailable 1
   40  kubectl get nodes
   41  gcloud container clusters update scaling-demo     --enable-autoprovisioning     --min-cpu 1     --min-memory 2     --max-cpu 45     --max-memory 160
   42  kubectl run -i --tty load-generator --rm --image=busybox --restart=Never -- /bin/sh -c "while sleep 0.01; do wget -q -O- http://php-apache; done"
   43  kubectl get pods
   44  kubectl run -i --tty load-generator --rm --image=busybox --restart=Never -- /bin/sh -c "while sleep 0.01; do wget -q -O- http://php-apache; done"
   45  history

Optimizing virtual machine choices

.https://cloud.google.com/solutions/best-practices-for-running-cost-effective-kubernetes-applications-on-gke#choose_the_right_machine_type

GKE workload optimization

Modes of operation of Kubernetes

Logs

exclusion rules

As the metric server needs to run somewhere, as with each new cluster update the metrics server gets updated and adds to lag in functioning of autoscaling. For this release, it is better to own the metric deployment configuration:

Use kpt for configuration with Anthos.