Lesson 2.5: Managing Rollouts in Kubernetes Deployments

Rolling Update

Rolling updates are a deployment strategy in Kubernetes that allows you to update an application with zero downtime. Instead of taking down all instances of the application at once, Kubernetes gradually replaces old Pods with new ones, ensuring that the application remains available throughout the update process.

How Rolling Updates Work

Create New Pods: Kubernetes creates new Pods with the updated configuration (e.g., a new container image).
Gradual Replacement: Kubernetes replaces old Pods with new ones one by one (or in small batches), ensuring that the desired number of Pods is always running.
Health Checks: Kubernetes ensures that the new Pods are healthy and ready to serve traffic before terminating the old Pods.
Completion: Once all old Pods are replaced with new ones, the rolling update is complete.

Key Benefits of Rolling Updates

Zero Downtime: The application remains available during the update process.
Gradual Rollout: Updates are applied incrementally, reducing the risk of introducing bugs or issues.
Rollback: If something goes wrong, you can easily roll back to the previous version.

Example Scenario

Let’s say you have a Deployment running 3 replicas of an nginx application. You want to update the container image from nginx:1.26.3 to nginx:1.27.4

Current State: 3 Pods running nginx:1.26.3

[root@master ~]# cat deploy.yml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deploy
  labels:
    env: demo
spec:
  template:
    metadata:
      name: nginx
      labels:
        env: demo
    spec:
      containers:
      - name: nginx
        image: nginx:1.26.3
  replicas: 3
  selector: 
    matchLabels:
      env: demo 
 
[root@master ~]# kubectl get deploy
NAME           READY   UP-TO-DATE   AVAILABLE   AGE
nginx-deploy   3/3     3            3           31s
 
[root@master ~]# kubectl get pods
NAME                            READY   STATUS    RESTARTS   AGE
nginx-deploy-69d557476b-729z4   1/1     Running   0          19s
nginx-deploy-69d557476b-g6fkd   1/1     Running   0          19s
nginx-deploy-69d557476b-m9xbq   1/1     Running   0          19s
 
# Checking the installed version
[root@master ~]# kubectl describe pod nginx-deploy-69d557476b-729z4
...
Image:  nginx:1.26.3
...

Update the Deployment: Change the container image to nginx:1.27.4 in the Deployment YAML or use the kubectl set image command.

[root@master ~]# cat deploy.yml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deploy
  labels:
    env: demo
spec:
  template:
    metadata:
      name: nginx
      labels:
        env: demo
    spec:
      containers:
      - name: nginx
        image: nginx:1.27.4
  replicas: 3
  selector: 
    matchLabels:
      env: demo 
 
[root@master ~]# kubectl apply -f deploy.yml 
deployment.apps/nginx-deploy configured

Rolling Update Process:

Kubernetes creates a new ReplicaSet with the updated image (nginx:1.27.4).
Gradually scales up the new ReplicaSet while scaling down the old one.
Ensures that the desired number of Pods (e.g., 3) is always running.

[root@master ~]# kubectl get pods
NAME                            READY   STATUS              RESTARTS   AGE
nginx-deploy-69d557476b-729z4   1/1     Running             0          6m14s  # old-version
nginx-deploy-69d557476b-g6fkd   1/1     Running             0          6m14s  # old-version
nginx-deploy-69d557476b-m9xbq   1/1     Running             0          6m14s  # old-version
nginx-deploy-6c4cfb5d6d-594h7   0/1     ContainerCreating   0          1s     # new-version
[root@master ~]# kubectl get pods
NAME                            READY   STATUS              RESTARTS   AGE
nginx-deploy-69d557476b-729z4   1/1     Running             0          6m17s  # old-version
nginx-deploy-69d557476b-g6fkd   0/1     Terminating         0          6m17s  # old-version
nginx-deploy-69d557476b-m9xbq   0/1     Terminating         0          6m17s  # old-version
nginx-deploy-6c4cfb5d6d-594h7   1/1     Running             0          4s     # new-version
nginx-deploy-6c4cfb5d6d-npm2j   0/1     ContainerCreating   0          0s     # new-version
nginx-deploy-6c4cfb5d6d-w65qm   1/1     Running             0          1s     # new-version
[root@master ~]# kubectl get pods
NAME                            READY   STATUS    RESTARTS   AGE
nginx-deploy-6c4cfb5d6d-594h7   1/1     Running   0          6s     # new-version
nginx-deploy-6c4cfb5d6d-npm2j   1/1     Running   0          2s     # new-version
nginx-deploy-6c4cfb5d6d-w65qm   1/1     Running   0          3s     # new-version
 
[root@master ~]# kubectl describe pod nginx-deploy-6c4cfb5d6d-594h7
...
Image:  nginx:1.27.4
...

Rollback a Rolling Update

If something goes wrong during the update, you can roll back to the previous version:
Rolling updates ensure zero downtime during application updates.
Kubernetes Deployments handle rolling updates automatically.
You can customize the update behavior and roll back if needed.

[root@master ~]# kubectl rollout undo deployment/nginx-deploy 
 
deployment.apps/nginx-deploy rolled back
[root@master ~]# kubectl get pods
NAME                            READY   STATUS              RESTARTS   AGE
nginx-deploy-69d557476b-pdjxq   0/1     ContainerCreating   0          2s   # version-1.26.3
nginx-deploy-6c4cfb5d6d-594h7   1/1     Running             0          21m  # version-1.27.4
nginx-deploy-6c4cfb5d6d-npm2j   1/1     Running             0          21m  # version-1.27.4
nginx-deploy-6c4cfb5d6d-w65qm   1/1     Running             0          21m  # version-1.27.4
[root@master ~]# kubectl get pods
NAME                            READY   STATUS        RESTARTS   AGE
nginx-deploy-69d557476b-478rj   1/1     Running       0          2s   # version-1.26.3
nginx-deploy-69d557476b-8shjn   1/1     Running       0          1s   # version-1.26.3
nginx-deploy-69d557476b-pdjxq   1/1     Running       0          4s   # version-1.26.3
nginx-deploy-6c4cfb5d6d-npm2j   1/1     Terminating   0          21m   # version-1.27.4
[root@master ~]# kubectl get pods
NAME                            READY   STATUS    RESTARTS   AGE
nginx-deploy-69d557476b-478rj   1/1     Running   0          4s   # version-1.26.3
nginx-deploy-69d557476b-8shjn   1/1     Running   0          3s   # version-1.26.3
nginx-deploy-69d557476b-pdjxq   1/1     Running   0          6s   # version-1.26.3

Rollout History

Viewing Rollout Revisions

[root@master ~]# kubectl rollout history deployment/nginx-deploy
deployment.apps/nginx-deploy 
REVISION  CHANGE-CAUSE
2         <none>
3         <none>

Inspecting Details of a Specific Revision

[root@master ~]# kubectl rollout history deployment/nginx-deploy --revision=2
deployment.apps/nginx-deploy with revision #2
Pod Template:
  Labels:	env=demo
	pod-template-hash=6c4cfb5d6d
  Containers:
   nginx:
    Image:	nginx:1.27.4
    Port:	<none>
    Host Port:	<none>
    Environment:	<none>
    Mounts:	<none>
  Volumes:	<none>
  Node-Selectors:	<none>
  Tolerations:	<none>
 
[root@master ~]# kubectl rollout history deployment/nginx-deploy --revision=3
deployment.apps/nginx-deploy with revision #3
Pod Template:
  Labels:	env=demo
	pod-template-hash=69d557476b
  Containers:
   nginx:
    Image:	nginx:1.26.3
    Port:	<none>
    Host Port:	<none>
    Environment:	<none>
    Mounts:	<none>
  Volumes:	<none>
  Node-Selectors:	<none>
  Tolerations:	<none>

Rollback to a Specific Revision

If you want to roll back to a specific revision, use:

[root@master ~]# kubectl rollout undo deployment/nginx-deploy --to-revision=2
deployment.apps/nginx-deploy rolled back
 
[root@master ~]# kubectl describe pod nginx-deploy-6c4cfb5d6d-6dw8l | grep Image:
    Image:          nginx:1.27.4

Add Change-Cause to Track Changes in Revision

To make the rollout history more meaningful, you can add a CHANGE-CAUSE annotation to your Deployment. This can be done by adding the kubernetes.io/change-cause annotation in the Deployment YAML or using the kubectl apply command with the --record flag.

[root@master ~]# cat deploy.yml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deploy
  labels:
    env: demo
spec:
  template:
    metadata:
      name: nginx
      labels:
        env: demo
      annotations:
        kubernetes.io/change-cause: "Deployment with nginx:1.27.4-alpine"
    spec:
      containers:
      - name: nginx
        image: nginx:1.27.4-alpine
  replicas: 3
  selector: 
    matchLabels:
      env: demo 
 
[root@master ~]# kubectl apply -f deploy.yml 
deployment.apps/nginx-deploy configured
 
[root@master ~]# kubectl get pods
NAME                            READY   STATUS    RESTARTS   AGE
nginx-deploy-6dc6975759-gj5pb   1/1     Running   0          18s
nginx-deploy-6dc6975759-sjlfv   1/1     Running   0          6s
nginx-deploy-6dc6975759-vtgqt   1/1     Running   0          7s
 
[root@master ~]# kubectl describe pod nginx-deploy-6dc6975759-gj5pb | grep Image:
    Image:          nginx:1.27.4-alpine

Now, the CHANGE-CAUSE column will show the command or annotation:

[root@master ~]# kubectl rollout history deployment/nginx-deploy 
deployment.apps/nginx-deploy 
REVISION  CHANGE-CAUSE
3         <none>
4         <none>
5         Deployment with nginx:1.27.4-alpine

Example

# version nginx:1.16 
[root@master practice-2]# vim q3.yml 
[root@master practice-2]# cat q3.yml 
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: web-proj-268
  name: web-proj-268
spec:
  replicas: 1
  selector:
    matchLabels:
      app: web-proj-268
  template:
    metadata:
      labels:
        app: web-proj-268
      annotations:
        kubernetes.io/change-cause: "Deployment of nginx version 1.16"
    spec:
      containers:
      - image: nginx:1.16
        name: nginx
[root@master practice-2]# kubectl apply -f q3.yml 
deployment.apps/web-proj-268 created
 
[root@master practice-2]# kubectl rollout history deploy web-proj-268 
deployment.apps/web-proj-268 
REVISION  CHANGE-CAUSE
1         Deployment of nginx version 1.16
 
# Version nginx:1.17
 
[root@master practice-2]# vim q3.yml 
[root@master practice-2]# cat q3.yml 
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: web-proj-268
  name: web-proj-268
spec:
  replicas: 1
  selector:
    matchLabels:
      app: web-proj-268
  template:
    metadata:
      labels:
        app: web-proj-268
      annotations:
        kubernetes.io/change-cause: "Deployment of nginx version 1.17"
    spec:
      containers:
      - image: nginx:1.17
        name: nginx
[root@master practice-2]# kubectl apply -f q3.yml 
deployment.apps/web-proj-268 configured
 
[root@master practice-2]# kubectl rollout history deploy web-proj-268 
deployment.apps/web-proj-268 
REVISION  CHANGE-CAUSE
1         Deployment of nginx version 1.16
2         Deployment of nginx version 1.17
 
# version nginx:latest
 
[root@master practice-2]# vim q3.yml 
[root@master practice-2]# cat q3.yml 
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: web-proj-268
  name: web-proj-268
spec:
  replicas: 1
  selector:
    matchLabels:
      app: web-proj-268
  template:
    metadata:
      labels:
        app: web-proj-268
      annotations:
        kubernetes.io/change-cause: "Deployment of nginx version latest"
    spec:
      containers:
      - image: nginx:latest
        name: nginx
[root@master practice-2]# kubectl apply -f q3.yml 
deployment.apps/web-proj-268 configured
 
[root@master practice-2]# kubectl rollout history deploy web-proj-268 
deployment.apps/web-proj-268 
REVISION  CHANGE-CAUSE
1         Deployment of nginx version 1.16
2         Deployment of nginx version 1.17
3         Deployment of nginx version latest
 
# Undo to version 1.16 --to-revision=1
[root@master practice-2]# kubectl rollout undo deployment web-proj-268  --to-revision=1
deployment.apps/web-proj-268 rolled back
[root@master practice-2]# kubectl get pods 
NAME                            READY   STATUS    RESTARTS   AGE
web-proj-268-78c5764bc8-ldvwh   1/1     Running   0          7s
[root@master practice-2]# kubectl describe pod web-proj-268-78c5764bc8-ldvwh | grep -i image 
    Image:          nginx:1.16
 
# Undo to version 1.17 --to-revision=2
[root@master practice-2]# kubectl rollout undo deploy web-proj-268 --to-revision=2
deployment.apps/web-proj-268 rolled back
[root@master practice-2]# kubectl get pods 
NAME                            READY   STATUS    RESTARTS   AGE
web-proj-268-7b99bdd4f4-fdn5d   1/1     Running   0          5s
[root@master practice-2]# kubectl describe pod web-proj-268-7b99bdd4f4-fdn5d | grep -i image: 
    Image:          nginx:1.17
 
# undo to version latest --to-revision=3
[root@master practice-2]# kubectl rollout undo deploy web-proj-268 --to-revision=3
deployment.apps/web-proj-268 rolled back
[root@master practice-2]# kubectl get pods 
NAME                            READY   STATUS    RESTARTS   AGE
web-proj-268-6c9ff76bc8-zncv4   1/1     Running   0          3s
[root@master practice-2]# kubectl describe pod web-proj-268-6c9ff76bc8-zncv4 | grep -i image: 
    Image:          nginx:latest 
  
[root@master practice-2]# kubectl rollout history deploy web-proj-268 
deployment.apps/web-proj-268 
REVISION  CHANGE-CAUSE
4         Deployment of nginx version 1.16
5         Deployment of nginx version 1.17
6         Deployment of nginx version latest