Skip to content
System Design

Kubernetes interview questions and answers

42 questions, 42 with code. About 3 h 19 min to read them all.

By TestLab · Updated

Plan for an interview
42 questions

Track what you know and what to revise.

Sign in to track progress

You can open a few full answers without signing in. Sign in to read all

1

What is Kubernetes, and why is it used for container orchestration?

Kubernetes is an open-source platform that runs containers across a cluster of machines and keeps them in the state you declare. You describe the desired state (for example “three replicas of this image, reachable on port 80”), and Kubernetes controllers keep comparing it with the actual state and fix any difference: they restart crashed containers, replace pods on failed nodes, roll out new versions and route traffic only to healthy instances. It is used because running more than a handful of containers by hand, with scripts, does not scale and does not heal itself.

Read full answer

Applies toKubernetes 1.29+ on managed services (EKS, GKE, AKS) or self-hosted clusters

The Problem It Solves

Docker runs a container on one machine. A production service needs more:

  • Placing containers on machines with enough CPU and memory.

  • Restarting them when they crash and moving them when a machine dies.

  • Giving a changing set of containers one stable address and spreading traffic across them.

  • Rolling out a new version without downtime, and rolling back when it misbehaves.

  • Scaling out under load and back in when it drops.

  • Injecting configuration and credentials without rebuilding images.

Kubernetes handles each of these with a built-in object instead of custom tooling.

The Declarative, Reconcile-Loop Model

  1. You submit a manifest (YAML) to the API server, which stores it in etcd.

  2. A controller watches that object type and compares desired with actual state.

  3. When they differ, it acts: a Deployment controller creates pods, the scheduler picks a node for each, and the kubelet on that node starts the containers.

  4. The loop never stops. If a node dies, the pod count drops below the desired three, and a replacement is created elsewhere.

This is why you change a cluster by editing manifests, not by issuing step-by-step commands.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: url-shortener
spec:
  replicas: 3
  selector:
    matchLabels:
      app: url-shortener
  template:
    metadata:
      labels:
        app: url-shortener
    spec:
      containers:
        - name: api
          image: registry.example.com/url-shortener:1.4.2
          ports:
            - containerPort: 8080
          resources:
            requests:
              cpu: 250m
              memory: 256Mi
            limits:
              memory: 512Mi
          readinessProbe:
            httpGet:
              path: /healthz
              port: 8080
kubectl apply -f deployment.yaml
kubectl get pods -l app=url-shortener
kubectl rollout status deployment/url-shortener

Core Building Blocks

Object What it gives you
Pod One or more containers sharing a network namespace and volumes
Deployment / ReplicaSet N identical, replaceable pods with rolling updates
StatefulSet Pods with stable names and their own storage
Service A stable virtual IP and DNS name over a changing set of pods
Ingress / Gateway API HTTP routing from outside the cluster
ConfigMap / Secret Configuration and credentials kept outside the image
HorizontalPodAutoscaler Replica count driven by CPU, memory or custom metrics

Trade-offs

Benefit Cost
Self-healing and rolling updates A control plane to run, upgrade and secure (or pay a provider for)
Portable across clouds and on-premises Many concepts to learn: networking, RBAC, storage classes
Consistent deployment model for every team YAML sprawl; usually needs Helm or Kustomize
Efficient bin-packing of workloads Debugging spans pods, nodes, DNS and the network plugin

When Not to Use It

  • A single small app or a team without operations experience: a PaaS, a managed container service or a couple of VMs behind a load balancer is cheaper to run.

  • Purely serverless, event-driven workloads that already fit a functions platform.

  • When nobody will own upgrades: Kubernetes minor versions are supported for roughly 14 months, so clusters need regular upgrades.

Interview Tip

Interviewers look for the reconcile loop in your answer, not a list of features. Say “desired state versus actual state, continuously reconciled by controllers”, and distinguish Docker (builds images and runs containers on one host) from Kubernetes (schedules and manages containers across many hosts). A common mistake is to call Kubernetes a container runtime; it delegates running containers to a runtime such as containerd through the CRI.

2

Describe the roles of master and worker nodes in the Kubernetes architecture.

A Kubernetes cluster has two kinds of machines. The control plane (historically called the “master”) makes the decisions: it stores the desired state, schedules pods and runs the controllers that reconcile it. Worker nodes do the work: they run the pods, report their health and route Service traffic. The control plane never runs your application containers by default; workers never decide where pods go.

Read full answer

Applies toKubernetes 1.29+ (kubeadm, managed services such as EKS, GKE, AKS)

Control Plane Components

Component Role
kube-apiserver The only entry point. Validates and stores every object, serves kubectl, controllers and kubelets. Stateless, so you run several behind a load balancer
etcd Consistent key-value store holding all cluster state. Uses Raft, so it needs a quorum
kube-scheduler Picks a node for each new pod: filters nodes that fit (resources, taints, affinity), then scores them
kube-controller-manager Runs the built-in controllers (Deployment, ReplicaSet, Node, Job, EndpointSlice…) that reconcile desired and actual state
cloud-controller-manager Talks to the cloud provider: load balancers, node lifecycle, routes

Worker Node Components

Component Role
kubelet Agent on every node. Watches the API server for pods assigned to it, starts them through the runtime, runs probes, reports status
Container runtime Pulls images and runs containers via the CRI (containerd or CRI-O; Docker Engine support through dockershim was removed in 1.24)
kube-proxy Programs iptables, IPVS or nftables rules so Service IPs reach pod IPs. Some network plugins replace it with eBPF
CNI plugin Gives each pod an IP and connects pods across nodes

How a Pod Gets Running

  1. kubectl apply sends a Deployment to the API server, which writes it to etcd.

  2. The Deployment and ReplicaSet controllers create pod objects with no node assigned.

  3. The scheduler sees unscheduled pods and binds each one to a node.

  4. The kubelet on that node sees the binding, asks the runtime to pull the image and start containers, and the CNI plugin assigns an IP.

  5. The kubelet runs readiness probes; once ready, the pod is added to the Service’s EndpointSlice and kube-proxy sends it traffic.

Every component talks to the API server, never directly to each other or to etcd (except the API server itself).

High Availability

  • Run three control plane nodes (or five for large clusters). etcd needs a majority: 3 members tolerate 1 failure, 5 tolerate 2. An even count adds no tolerance.

  • The API server is stateless and load-balanced; the scheduler and controller manager run on every control plane node but use leader election, so only one instance acts at a time.

  • If the whole control plane goes down, running pods keep running on workers, but nothing new is scheduled and failed pods are not replaced.

  • Spread workers across availability zones so losing a zone leaves capacity elsewhere.

Control plane nodes carry the taint node-role.kubernetes.io/control-plane:NoSchedule so ordinary pods stay off them.

kubectl get nodes -o wide
kubectl get pods -n kube-system
kubectl describe node worker-1 | grep -A5 Taints

Managed versus Self-Hosted

Aspect Managed (EKS, GKE, AKS) Self-hosted (kubeadm)
Control plane Run and upgraded by the provider You run, back up and upgrade it
etcd backups Provider You (etcdctl snapshot save)
Worker nodes Usually yours, often in managed node groups Yours
Cost Per-cluster fee plus nodes Machines plus your team’s time

Interview Tip

Say “control plane” rather than “master”; the term changed in the project and in node labels. Interviewers often follow up with “what happens if etcd loses quorum?” (the API becomes read-only or unavailable, but workloads keep running) and “what does the kubelet do if the API server is unreachable?” (it keeps existing pods running and restarts crashed containers locally). A common mistake is saying the scheduler starts containers; it only assigns a node, and the kubelet starts them.

3

How do Pods facilitate the management of containers in Kubernetes?

A Pod is the smallest unit Kubernetes schedules: one or more containers that are always placed on the same node and share a network namespace (one IP, talk over localhost), storage volumes and a lifecycle. Kubernetes manages pods, not individual containers, so containers that must run together, such as an app and a log-shipping sidecar, are scheduled, started, scaled and replaced as a unit.

Read full answer

Applies toKubernetes 1.29+

What Containers in a Pod Share

Shared Effect
Network namespace One pod IP; containers reach each other on localhost and must use different ports
Volumes An emptyDir or mounted volume can be read and written by every container
Node and lifecycle Scheduled together, deleted together, never split across nodes
Resource accounting Requests and limits add up per pod for scheduling

Each container still has its own image, filesystem, process space (unless shareProcessNamespace: true), probes and resource limits.

Patterns

  • Single container: the common case. One app per pod; scale by adding pods, not containers.

  • Sidecar: a helper that runs for the pod’s whole life: a log forwarder, a proxy (service mesh), a config reloader. Since Kubernetes 1.29 (beta, GA in 1.33) a sidecar is declared as an init container with restartPolicy: Always, so it starts before and stops after the main container.

  • Init containers: run to completion, in order, before the app starts: wait for a database, run a migration, fetch a file.

  • Adapter and ambassador: reshape output (metrics format) or proxy outbound connections.

Example

apiVersion: v1
kind: Pod
metadata:
  name: checkout
  labels:
    app: checkout
spec:
  initContainers:
    - name: wait-for-db
      image: postgres:16
      command: ["sh", "-c", "until pg_isready -h postgres -p 5432; do sleep 2; done"]
    - name: log-shipper
      image: fluent/fluent-bit:3.0
      restartPolicy: Always
      volumeMounts:
        - name: logs
          mountPath: /var/log/app
  containers:
    - name: app
      image: registry.example.com/checkout:2.1.0
      ports:
        - containerPort: 8080
      volumeMounts:
        - name: logs
          mountPath: /var/log/app
      resources:
        requests:
          cpu: 200m
          memory: 256Mi
        limits:
          memory: 512Mi
      readinessProbe:
        httpGet:
          path: /ready
          port: 8080
      livenessProbe:
        httpGet:
          path: /healthz
          port: 8080
        periodSeconds: 10
  volumes:
    - name: logs
      emptyDir: {}

How Kubernetes Manages a Pod

  1. The scheduler assigns the pod to a node with enough requested CPU and memory.

  2. The kubelet runs init containers in order, then starts the app containers.

  3. Probes drive behaviour: a failing readiness probe removes the pod from Service endpoints; a failing liveness probe restarts the container; a startup probe holds off liveness checks for slow starters.

  4. restartPolicy (Always, OnFailure, Never) decides whether exited containers restart. Restarts use exponential back-off, visible as CrashLoopBackOff.

  5. On deletion, each container gets SIGTERM, then SIGKILL after terminationGracePeriodSeconds (30 s by default).

Pod phases are Pending, Running, Succeeded, Failed and Unknown.

Pods Are Disposable

A pod is never moved or repaired. If its node fails, the pod is gone, and a controller creates a new pod with a new name and IP. That is why you almost never create bare pods: use a Deployment, StatefulSet, DaemonSet or Job, and put a Service in front so clients do not depend on pod IPs.

Trade-offs

Choice Benefit Cost
One container per pod Independent scaling and rollout Helpers must run elsewhere
Sidecar in the pod Shares network and files with the app Uses resources in every replica; scaling the app scales the sidecar
Two apps in one pod None for independent services They cannot scale or deploy separately; avoid

Interview Tip

Interviewers want to hear why the pod exists rather than scheduling containers directly: co-location plus shared network and storage for tightly coupled processes. A common mistake is putting a web app and its database in one pod; they have different scaling, storage and lifecycle needs and belong in separate workloads. Expect a follow-up on liveness versus readiness probes.

4

What types of Services exist in Kubernetes, and how do they facilitate pod communication?

A Service gives a changing set of pods one stable virtual IP and DNS name and load-balances connections across the pods that match its label selector. There are four types: ClusterIP (internal only, the default), NodePort (a port on every node), LoadBalancer (an external load balancer from the cloud provider) and ExternalName (a DNS alias to something outside the cluster). A headless Service (clusterIP: None) is a ClusterIP variant that returns pod IPs directly.

Read full answer

Applies toKubernetes 1.29+

Why Services Exist

Pods are replaced constantly, and each new pod gets a new IP. Clients cannot track those IPs themselves. A Service solves this:

  1. It selects pods by labels (app: orders).

  2. The EndpointSlice controller keeps a list of the ready pods’ IPs.

  3. kube-proxy (or an eBPF data plane) on each node programs rules that send traffic for the Service IP to one of those pods.

  4. CoreDNS publishes orders.shop.svc.cluster.local, so clients use a name, not an IP.

Request flow: Client pod → orders:80 (DNS → ClusterIP) → node rules → one ready pod on port 8080.

The Types

Type Reachable from How Typical use
ClusterIP Inside the cluster Virtual IP from the service CIDR Service-to-service calls
NodePort Outside, via <node-ip>:<port> Opens a port (30000–32767 by default) on every node, plus a ClusterIP Bare-metal, quick tests, behind your own LB
LoadBalancer Outside, via a provisioned LB Cloud controller creates an L4 load balancer that targets the nodes or pods Exposing one TCP/UDP service publicly
ExternalName Inside the cluster DNS CNAME to an external host; no proxying Pointing at a managed database by a cluster-local name
Headless (clusterIP: None) Inside the cluster DNS returns the pod IPs StatefulSets, client-side load balancing

Each type builds on the previous one: a LoadBalancer Service also has a NodePort and a ClusterIP.

Example

apiVersion: v1
kind: Service
metadata:
  name: orders
  namespace: shop
spec:
  type: ClusterIP
  selector:
    app: orders
  ports:
    - name: http
      port: 80
      targetPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: orders-public
  namespace: shop
spec:
  type: LoadBalancer
  selector:
    app: orders
  ports:
    - port: 443
      targetPort: 8443
kubectl get svc -n shop
kubectl get endpointslices -n shop -l kubernetes.io/service-name=orders
kubectl run tmp --rm -it --image=busybox:1.36 -n shop -- wget -qO- http://orders/healthz

Behaviour Worth Knowing

  • Only ready pods receive traffic; a failing readiness probe removes a pod from the EndpointSlice.

  • Load balancing is per connection, not per request. Long-lived HTTP/2 or gRPC connections stick to one pod; use client-side balancing, a headless Service or a mesh for even spread.

  • sessionAffinity: ClientIP pins a client to a pod, at the cost of uneven load.

  • externalTrafficPolicy: Local keeps the client source IP and avoids an extra hop, but only nodes running a pod receive traffic.

  • For HTTP routing by host or path, put an Ingress or Gateway API route in front of ClusterIP Services instead of one LoadBalancer per service, which costs one cloud load balancer each.

Choosing

  • Internal microservice: ClusterIP.

  • Public HTTP(S) for many services: ClusterIP plus Ingress/Gateway.

  • Public non-HTTP protocol (MQTT, a game server): LoadBalancer.

  • On-premises without a cloud LB: NodePort behind your own load balancer, or MetalLB to provide LoadBalancer support.

  • Stateful peers that address each other: headless.

Interview Tip

Explain the chain selector → EndpointSlice → kube-proxy rules → DNS, not just the type names. A frequent follow-up is “why does my gRPC service send all traffic to one pod?”, where the answer is connection-level balancing. A common mistake is thinking a Service is a process or a proxy pod; for ClusterIP it is a set of rules on every node.

5

Explain the functionality of Namespaces in Kubernetes.

A Namespace is a named scope inside one cluster that groups resources and gives them a separate set of names, permissions and quotas. Two teams can each have a Service called api in their own namespaces. Namespaces are the unit you attach RBAC, ResourceQuota, LimitRange and NetworkPolicy to, which makes them the basic tool for sharing a cluster between teams or environments. They do not isolate network traffic or nodes by themselves.

Read full answer

Applies toKubernetes 1.29+

What a Namespace Scopes

Namespaced (live inside one) Cluster-scoped (shared by all)
Pods, Deployments, StatefulSets, Jobs Nodes
Services, Ingresses, EndpointSlices PersistentVolumes, StorageClasses
ConfigMaps, Secrets, ServiceAccounts Namespaces themselves
Roles, RoleBindings ClusterRoles, ClusterRoleBindings
PersistentVolumeClaims, NetworkPolicies CustomResourceDefinitions, IngressClasses

kubectl api-resources --namespaced=true lists the first column for your cluster.

Built-in Namespaces

  • default: where objects go when you do not specify one.

  • kube-system: control plane add-ons such as CoreDNS and kube-proxy.

  • kube-public: readable by everyone, rarely used.

  • kube-node-lease: node heartbeat Lease objects.

What You Attach to a Namespace

apiVersion: v1
kind: Namespace
metadata:
  name: payments
  labels:
    team: payments
    pod-security.kubernetes.io/enforce: restricted
---
apiVersion: v1
kind: ResourceQuota
metadata:
  name: payments-quota
  namespace: payments
spec:
  hard:
    requests.cpu: "20"
    requests.memory: 40Gi
    limits.memory: 80Gi
    pods: "200"
---
apiVersion: v1
kind: LimitRange
metadata:
  name: payments-defaults
  namespace: payments
spec:
  limits:
    - type: Container
      defaultRequest:
        cpu: 100m
        memory: 128Mi
      default:
        memory: 256Mi
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: payments-devs-edit
  namespace: payments
subjects:
  - kind: Group
    name: payments-developers
    apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: edit
  apiGroup: rbac.authorization.k8s.io
  • RBAC: a RoleBinding grants rights only in its namespace; here the payments team can edit their own workloads and nothing else.

  • ResourceQuota: caps the total CPU, memory and object counts for the namespace.

  • LimitRange: sets default requests and limits for containers that omit them.

  • Pod Security Admission: the namespace label enforces the restricted pod security profile.

  • NetworkPolicy: restricts which pods may talk to which; without it, pods in all namespaces can reach each other.

DNS Across Namespaces

A Service api in namespace payments resolves as api from inside payments, and as api.payments or api.payments.svc.cluster.local from anywhere else.

kubectl create namespace payments
kubectl get pods -n payments
kubectl config set-context --current --namespace=payments
kubectl get all -A

Common Layouts

Layout Example Trade-off
Per team payments, search Clear ownership and quotas
Per environment staging, prod Simple, but staging and prod share nodes and the control plane
Per team and environment payments-staging Fine-grained, more objects to manage
Per tenant customer-123 Soft multi-tenancy; still one cluster

For strong isolation, such as separating production from development or untrusted tenants, use separate clusters. Namespaces share nodes, the kernel, cluster-scoped resources and the API server.

Interview Tip

The point interviewers check is that a namespace is a naming and policy boundary, not a security boundary. Say what you add to make it one: RBAC, quotas, NetworkPolicies, Pod Security and, if needed, dedicated nodes via taints. A common mistake is assuming pods in different namespaces cannot talk to each other. Also note that deleting a namespace deletes everything inside it.

6

Differentiate between Deployments, StatefulSets, and DaemonSets.

All three are controllers that keep pods running, but they answer different questions. A Deployment runs N interchangeable pods and is used for stateless services. A StatefulSet runs N pods with a stable identity: fixed names (db-0, db-1), stable DNS and a dedicated volume each, started and updated in order. A DaemonSet runs one pod per node (or per selected node) for node-level agents.

Read full answer

Applies toKubernetes 1.29+ (`apps/v1`)

Comparison

Aspect Deployment StatefulSet DaemonSet
Replica count You set replicas You set replicas One per matching node
Pod names Random suffix (api-7d9f-x2k4p) Ordinal (db-0, db-1) Random suffix per node
Identity after restart New pod, new name and IP Same name and DNS, same volume Recreated on the same node
Storage Usually none, or one shared claim volumeClaimTemplates: one PVC per pod, kept after deletion Usually hostPath or none
Start and update order Parallel, rolling with maxSurge/maxUnavailable Ordered by default (0, 1, 2…; updates from highest ordinal down) Rolling per node with maxUnavailable
Network identity Through a normal Service Needs a headless Service: db-0.db.ns.svc.cluster.local Usually none, or host ports
Scaling HPA friendly Possible, but data must rebalance Follows the node count
Typical workloads Web APIs, workers Databases, Kafka, ZooKeeper, Elasticsearch Log shippers, node exporters, CNI agents, CSI drivers

Deployment

A Deployment manages ReplicaSets. Each template change creates a new ReplicaSet and shifts pods to it gradually, which gives rolling updates and kubectl rollout undo.

StatefulSet

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: db
spec:
  serviceName: db
  replicas: 3
  selector:
    matchLabels:
      app: db
  template:
    metadata:
      labels:
        app: db
    spec:
      containers:
        - name: postgres
          image: postgres:16
          ports:
            - containerPort: 5432
          volumeMounts:
            - name: data
              mountPath: /var/lib/postgresql/data
  volumeClaimTemplates:
    - metadata:
        name: data
      spec:
        accessModes: ["ReadWriteOnce"]
        resources:
          requests:
            storage: 20Gi

Pod db-1 always mounts PVC data-db-1, even after rescheduling to another node. podManagementPolicy: Parallel drops ordered startup when the application does not need it. Kubernetes gives each replica a stable identity, but replication, failover and backups are the database’s job or an operator’s.

DaemonSet

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: node-exporter
  namespace: monitoring
spec:
  selector:
    matchLabels:
      app: node-exporter
  updateStrategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1
  template:
    metadata:
      labels:
        app: node-exporter
    spec:
      tolerations:
        - operator: Exists
      containers:
        - name: exporter
          image: quay.io/prometheus/node-exporter:v1.8.2
          ports:
            - containerPort: 9100

When a node joins, the DaemonSet controller adds a pod there; when it leaves, the pod goes too. The tolerations entry lets the agent run on tainted nodes, including control plane nodes. A nodeSelector or node affinity limits it to a subset, such as GPU nodes.

Choosing

  • Any replica can serve any request and holds no unique data: Deployment.

  • Each replica owns data or must be addressed by name by its peers: StatefulSet (or a managed database outside the cluster).

  • The pod exists to serve or observe the node it runs on: DaemonSet.

  • Work that runs to completion: none of these; use a Job or CronJob.

Interview Tip

Interviewers want the identity and storage distinction, not just “StatefulSet is for databases”. Mention the headless Service, volumeClaimTemplates, and that PVCs survive scale-down so data is not lost by accident. A common mistake is running a database as a Deployment with a shared ReadWriteOnce volume: a second replica cannot mount it, and a rolling update can deadlock while the old pod still holds the volume.

7

How do ReplicaSets ensure pod availability?

A ReplicaSet keeps a specified number of pods matching its label selector running at all times. Its controller continuously counts the matching pods: if there are fewer than replicas (a pod crashed for good, was deleted or its node died), it creates new ones from the pod template; if there are more, it deletes the extras. That reconcile loop, not any health check inside the ReplicaSet itself, is what keeps the service available.

Read full answer

Applies toKubernetes 1.29+ (`apps/v1`)

The Three Parts

Field Purpose
replicas Desired number of pods
selector Which pods count toward that number
template How to create a new pod; its labels must match the selector

The Reconcile Loop

  1. The controller watches pods and ReplicaSets through the API server.

  2. It counts active pods whose labels match the selector and that it owns (pods with ownerReferences pointing to this ReplicaSet, or orphaned pods it adopts).

  3. Count below replicas: create pods from the template. Count above: delete pods, preferring those not yet ready, on crowded nodes, or newest first.

  4. The scheduler places new pods; the kubelet starts them.

Failure scenarios:

  • Container crash: handled by the kubelet on the same node, which restarts the container in place according to restartPolicy. The pod object stays, so the ReplicaSet does nothing.

  • Pod deleted or evicted: the count drops and the ReplicaSet creates a replacement on any suitable node.

  • Node failure: the node controller marks the node NotReady; after the default toleration of about 5 minutes for node.kubernetes.io/unreachable, its pods are evicted and the ReplicaSet recreates them elsewhere.

Example

apiVersion: apps/v1
kind: ReplicaSet
metadata:
  name: web
spec:
  replicas: 3
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
        - name: nginx
          image: nginx:1.27
          readinessProbe:
            httpGet:
              path: /
              port: 80
kubectl apply -f rs.yaml
kubectl delete pod -l app=web --wait=false   # all three deleted
kubectl get pods -l app=web -w               # three new pods appear

What a ReplicaSet Does Not Do

  • No updates to existing pods. Changing the image in the template affects only pods created later. This is why you use a Deployment, which creates a new ReplicaSet for each template change and moves pods across gradually.

  • No spreading by itself. Three replicas can land on one node. Add topologySpreadConstraints or pod anti-affinity across nodes and zones.

  • No traffic decisions. Readiness probes plus a Service keep unready pods out of rotation.

  • No protection during voluntary disruption. A node drain can evict all replicas at once unless a PodDisruptionBudget limits it.

Making Availability Real

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: web-pdb
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app: web
Measure Protects against
replicas ≥ 2 Losing one pod or node
Topology spread across zones Losing a zone
Readiness probes Sending traffic to pods that cannot serve
PodDisruptionBudget Drains and upgrades removing too many pods at once
Resource requests Pods being starved or evicted under node pressure

Interview Tip

Say clearly that you almost never create a ReplicaSet directly; you create a Deployment, which owns ReplicaSets. Interviewers often ask “who restarts a crashed container, the kubelet or the ReplicaSet?” (the kubelet) and “what happens if you hand-create a pod with matching labels?” (the ReplicaSet adopts it and may delete one pod to get back to replicas). A common mistake is overlapping selectors between controllers, which makes them fight over pods.

8

What are Jobs in Kubernetes, and when is it suitable to use them?

A Job runs one or more pods until a set number of them finish successfully, then stops. Unlike a Deployment, which keeps pods running forever and restarts them when they exit, a Job treats exit code 0 as “done” and retries only failures. Use it for work with an end: database migrations, batch processing, report generation, backfills. A CronJob creates Jobs on a schedule.

Read full answer

Applies toKubernetes 1.29+ (`batch/v1`)

Key Fields

Field Meaning Default
completions Successful pods needed for the Job to finish 1
parallelism Pods allowed to run at once 1
backoffLimit Failed retries before the Job is marked Failed 6
activeDeadlineSeconds Hard time limit for the whole Job none
ttlSecondsAfterFinished Delete the Job and its pods after it finishes none (kept)
completionMode: Indexed Each pod gets JOB_COMPLETION_INDEX 0…n−1 NonIndexed
podFailurePolicy Fail fast or ignore specific exit codes or disruptions none
template.spec.restartPolicy OnFailure or Never (Always is not allowed) required

Example: Parallel Batch Job

apiVersion: batch/v1
kind: Job
metadata:
  name: resize-images
spec:
  completions: 10
  parallelism: 3
  completionMode: Indexed
  backoffLimit: 4
  activeDeadlineSeconds: 3600
  ttlSecondsAfterFinished: 86400
  template:
    spec:
      restartPolicy: Never
      containers:
        - name: worker
          image: registry.example.com/image-resizer:1.2.0
          args: ["--shard=$(JOB_COMPLETION_INDEX)", "--shards=10"]
          resources:
            requests:
              cpu: 500m
              memory: 512Mi

Ten shards, three at a time. Each pod processes the shard named by its index, and the Job completes when every index has succeeded once.

Example: CronJob

apiVersion: batch/v1
kind: CronJob
metadata:
  name: nightly-report
spec:
  schedule: "0 2 * * *"
  timeZone: "Asia/Kolkata"
  concurrencyPolicy: Forbid
  startingDeadlineSeconds: 600
  successfulJobsHistoryLimit: 3
  failedJobsHistoryLimit: 3
  jobTemplate:
    spec:
      backoffLimit: 2
      template:
        spec:
          restartPolicy: OnFailure
          containers:
            - name: report
              image: registry.example.com/reports:3.0.1

concurrencyPolicy: Forbid skips a run if the previous one is still going; Replace cancels the old run; Allow (the default) lets them overlap.

kubectl create job manual-report --from=cronjob/nightly-report
kubectl get jobs
kubectl logs job/resize-images
kubectl wait --for=condition=complete job/resize-images --timeout=1h

Patterns

Pattern Setup Use for
Single run completions: 1 Migration, one-off script
Fixed shards completions: N, Indexed Splitting a known dataset
Work queue parallelism: N, no completions; workers pull from a queue and exit when it is empty Unknown amount of work
Scheduled CronJob Nightly reports, cleanup, certificate checks

When to Use a Job, and When Not To

Use a Job when the work has a clear end and can be retried. Avoid it when:

  • The work never ends (a queue consumer that should always run): use a Deployment.

  • It must run on every node: use a DaemonSet.

  • You need multi-step workflows with dependencies and fan-in: a workflow engine such as Argo Workflows or an external scheduler is clearer than chaining Jobs.

  • The schedule must fire exactly once: a CronJob can occasionally create two Jobs or none for one schedule slot (controller restarts, missed deadlines), so the work must be idempotent.

Trade-offs and Failure Handling

  • Retries re-run the whole pod. Make the work idempotent (upserts, checkpoints, “skip if already processed”).

  • restartPolicy: OnFailure restarts the container in the same pod, which is cheaper but hides logs of earlier attempts; Never creates a new pod per attempt, which keeps logs but costs more scheduling.

  • Without ttlSecondsAfterFinished, finished Jobs and their pods pile up in etcd.

  • Migrations run as Jobs before a rollout need a decision about what happens when they fail; Helm hooks or a CI step usually gate the deployment on condition=complete.

Interview Tip

Interviewers check that you know why restartPolicy: Always is rejected for Jobs and that CronJobs are not exactly-once. A common mistake is running a migration inside the app’s container start-up, so every replica races to migrate; a single Job before the rollout avoids that.

9

How do Labels and Selectors work together in Kubernetes?

Labels are key-value pairs attached to objects (app: checkout, env: prod), and selectors are queries over those labels. Together they are how Kubernetes objects find each other without hard-coded names: a Service finds its pods, a Deployment finds the pods it owns, a NetworkPolicy finds the pods it protects, and kubectl -l filters what you see. Changing a pod’s labels changes which objects consider it theirs.

Read full answer

Applies toKubernetes 1.29+

Labels

  • Syntax: optional prefix plus name (app.kubernetes.io/name), value up to 63 characters from letters, digits, -, _ and ..

  • Meant for identifying and grouping, so they are indexed and queryable.

  • Annotations look similar but are for non-identifying metadata (build URL, checksum, tool settings) and cannot be selected on.

Recommended common labels:

Label Example
app.kubernetes.io/name checkout
app.kubernetes.io/instance checkout-eu
app.kubernetes.io/version 2.1.0
app.kubernetes.io/component api
app.kubernetes.io/part-of shop
app.kubernetes.io/managed-by helm

Selector Types

Type Operators Example
Equality-based =, ==, != env=prod,tier!=cache
Set-based in, notin, exists, ! (does not exist) env in (prod,staging), !canary

Services and ReplicationControllers accept only a plain map (equality). Deployments, ReplicaSets, StatefulSets, DaemonSets, Jobs, NetworkPolicies and affinity rules accept matchLabels and matchExpressions, which are ANDed together.

Example

apiVersion: apps/v1
kind: Deployment
metadata:
  name: checkout
  labels:
    app.kubernetes.io/name: checkout
spec:
  replicas: 3
  selector:
    matchLabels:
      app.kubernetes.io/name: checkout
    matchExpressions:
      - key: track
        operator: In
        values: ["stable"]
  template:
    metadata:
      labels:
        app.kubernetes.io/name: checkout
        track: stable
        env: prod
    spec:
      containers:
        - name: app
          image: registry.example.com/checkout:2.1.0
---
apiVersion: v1
kind: Service
metadata:
  name: checkout
spec:
  selector:
    app.kubernetes.io/name: checkout
  ports:
    - port: 80
      targetPort: 8080

The Deployment selects only track=stable pods, while the Service selects every checkout pod. A second Deployment with track: canary pods would receive a share of the Service’s traffic in proportion to its replica count: a simple canary.

kubectl get pods -l app.kubernetes.io/name=checkout
kubectl get pods -l 'env in (prod,staging),track!=canary' --show-labels
kubectl label pod checkout-6f8d-abcde track=debug --overwrite

The last command takes a pod out of the Deployment’s selector: the ReplicaSet creates a replacement, and the relabelled pod stays alive for debugging, still receiving Service traffic unless you also change app.kubernetes.io/name.

Where Selectors Are Used

Object Selects Effect
Service Pods Which pods receive traffic
Deployment / ReplicaSet / StatefulSet Pods Which pods it counts and owns
NetworkPolicy Pods and namespaces Which pods the rules apply to and allow
PodDisruptionBudget Pods Which pods are protected during drains
Node affinity / nodeSelector Node labels Where pods may be scheduled
Pod affinity, anti-affinity, topology spread Pod labels Co-location and spreading

Rules and Pitfalls

  • A Deployment’s selector is immutable after creation, and the template labels must satisfy it, or the API rejects the object.

  • Overlapping selectors between two Deployments make their controllers fight over the same pods. Always include a label unique to the workload.

  • Do not put changing values, such as the version, in a Deployment selector; you could never roll to a new version without recreating the Deployment.

  • A Service with a typo in its selector has no endpoints and returns connection refused or timeouts. Check kubectl get endpointslices -l kubernetes.io/service-name=<svc>.

  • Labels are not security controls: anyone allowed to edit a pod can change them.

Interview Tip

Interviewers want to hear that labels give loose coupling: objects select by intent, not by name, which is what makes rolling updates, canaries and Services possible. A typical follow-up is debugging a Service with no endpoints; the first check is whether its selector matches the pod labels exactly.

10

What would you consider when performing a rolling update in Kubernetes?

A rolling update replaces a Deployment’s pods with new ones a few at a time, so the service keeps serving throughout. Kubernetes handles the mechanics; you are responsible for the parts it cannot see: readiness probes that mean “ready”, graceful shutdown, enough capacity headroom, backward-compatible changes between old and new versions running side by side, and a rollback plan.

Read full answer

Applies toKubernetes 1.29+ (`apps/v1` Deployments)

How It Works

  1. Changing the pod template (image, env, resources) creates a new ReplicaSet.

  2. The controller scales the new ReplicaSet up and the old one down, bounded by maxSurge (extra pods allowed) and maxUnavailable (pods allowed to be missing). Both default to 25%.

  3. A new pod counts as available only after its readiness probe passes and it has stayed ready for minReadySeconds.

  4. If no progress is made within progressDeadlineSeconds (600 by default), the rollout is marked failed. It is not rolled back automatically.

Configuration

apiVersion: apps/v1
kind: Deployment
metadata:
  name: payments-api
spec:
  replicas: 6
  revisionHistoryLimit: 5
  minReadySeconds: 10
  progressDeadlineSeconds: 300
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  selector:
    matchLabels:
      app: payments-api
  template:
    metadata:
      labels:
        app: payments-api
    spec:
      terminationGracePeriodSeconds: 45
      containers:
        - name: api
          image: registry.example.com/payments-api:3.8.0
          readinessProbe:
            httpGet:
              path: /ready
              port: 8080
            periodSeconds: 5
            failureThreshold: 2
          lifecycle:
            preStop:
              sleep:
                seconds: 10
          resources:
            requests:
              cpu: 500m
              memory: 512Mi

maxUnavailable: 0 with maxSurge: 1 never drops below six serving pods but needs room for a seventh. The preStop sleep gives kube-proxy, load balancers and ingress controllers time to remove the pod from rotation before it stops accepting connections. The built-in sleep action needs Kubernetes 1.30+; on older clusters use an exec hook running sleep.

Checklist

Area What to check
Readiness The probe checks the app can serve (dependencies reachable, caches warm), not just that the process is up
Shutdown App handles SIGTERM: stop accepting, finish in-flight requests, then exit before the grace period ends
Capacity Cluster has room for surge pods; quotas allow them; HPA min/max fit
Compatibility Old and new versions run together for minutes: APIs, message formats and DB schema must work for both
Database changes Use expand/contract: add columns first, deploy code, remove old columns in a later release
Disruption A PodDisruptionBudget protects availability if a node drain overlaps the rollout
Image Immutable tag or digest; :latest makes rollbacks meaningless
Sessions and connections Long-lived WebSocket or gRPC connections are cut when old pods stop; clients must reconnect
Observability Watch error rate and latency per version during the rollout

Commands

kubectl set image deployment/payments-api api=registry.example.com/payments-api:3.9.0
kubectl rollout status deployment/payments-api --timeout=5m
kubectl rollout history deployment/payments-api
kubectl rollout undo deployment/payments-api
kubectl rollout pause deployment/payments-api

Choosing maxSurge and maxUnavailable

Setting Effect When
maxSurge: 25%, maxUnavailable: 25% Balanced default Most stateless services
maxSurge: 1, maxUnavailable: 0 Never lose capacity; slower Services at high utilisation
maxSurge: 0, maxUnavailable: 1 No extra resources needed; temporarily one pod short Tight quotas, licence-limited pods
type: Recreate All old pods stop before new ones start; downtime Versions that cannot run together

Beyond Rolling Updates

A rolling update shifts traffic in proportion to pod counts and cannot pause on metrics by itself. For risky releases use a canary (a small second Deployment, or Argo Rollouts/Flagger with automated analysis) or blue-green (switch a Service selector between two full Deployments, at double the resource cost).

Interview Tip

Interviewers look for the mixed-version window: both versions serve traffic at the same time, so schema and API changes must be compatible both ways. A common mistake is a readiness probe that always returns 200, which lets a broken version replace every healthy pod. Also mention that a stuck rollout needs kubectl rollout undo; Kubernetes does not roll back on its own.

11

How do Services route traffic to pods in Kubernetes?

A Service does not proxy traffic through a process of its own. Its label selector feeds the EndpointSlice controller, which keeps a list of the matching ready pod IPs. On every node, kube-proxy (or an eBPF data plane such as Cilium’s) turns that list into packet-forwarding rules, so a connection to the Service’s virtual IP is rewritten by the node’s kernel (DNAT) to one chosen pod IP. CoreDNS maps the Service name to its IP.

Read full answer

Applies toKubernetes 1.29+

The Path of One Request

  1. A client pod calls http://orders.shop:80.

  2. DNS: CoreDNS returns the ClusterIP, for example 10.96.14.20.

  3. The client opens a TCP connection to 10.96.14.20:80.

  4. Node rules: on the client’s node, iptables, IPVS or nftables rules match that IP and port and pick one backend, such as 10.244.3.17:8080, rewriting the destination.

  5. Pod network: the CNI plugin delivers the packet to the pod, possibly on another node.

  6. Conntrack remembers the translation, so replies and later packets of the same connection reach the same pod.

The ClusterIP is not bound to any interface; it exists only in those rules.

How Endpoints Are Maintained

  • The EndpointSlice controller watches pods matching the selector.

  • A pod appears as ready: true only when its readiness probe passes; terminating pods are marked terminating and stop receiving new connections.

  • Slices hold up to 100 endpoints by default, so large Services update incrementally instead of rewriting one huge object.

kubectl get endpointslices -n shop -l kubernetes.io/service-name=orders -o wide
kubectl describe svc orders -n shop

kube-proxy Modes

Mode How it picks a pod Scaling behaviour
iptables Random choice via probability rules Rule updates slow down with tens of thousands of endpoints
IPVS Kernel load balancer: round robin, least connections and others Hash tables, better for very large clusters
nftables Newer kernel API replacing iptables Faster updates than iptables
eBPF replacement (Cilium and others) Socket- or packet-level maps No per-Service chain walking; often better observability

Traffic Policies

apiVersion: v1
kind: Service
metadata:
  name: orders
  namespace: shop
  annotations:
    service.kubernetes.io/topology-mode: Auto
spec:
  selector:
    app: orders
  ports:
    - port: 80
      targetPort: http
  internalTrafficPolicy: Cluster
  sessionAffinity: None
Setting Effect Trade-off
externalTrafficPolicy: Local External traffic only to pods on the receiving node; keeps client IP Uneven load if pods are unevenly spread
internalTrafficPolicy: Local In-cluster traffic only to pods on the same node Fails if the node has no ready pod
sessionAffinity: ClientIP Same client IP goes to the same pod Hot spots behind NAT
topology-mode: Auto Prefer endpoints in the same zone Saves cross-zone latency and cost; needs enough pods per zone
targetPort: http (named) Resolves to the container port named http Lets pods change port numbers without editing the Service

Balancing Is Per Connection

The node rules choose a pod when a connection opens. With HTTP/1.1 keep-alive, HTTP/2 or gRPC, one client connection carries many requests to the same pod, so a few busy clients can overload a few pods. Remedies: a headless Service with client-side load balancing, a service mesh or proxy that balances per request (Envoy), or periodic connection recycling.

When Traffic Does Not Arrive

Symptom Likely cause
Service has no endpoints Selector does not match pod labels, or no pod is ready
Endpoints exist, connection refused targetPort does not match the container port
Works from some nodes only kube-proxy or CNI broken on a node
Name does not resolve Wrong namespace in the name, or CoreDNS issues
Traffic blocked A NetworkPolicy denies it

Interview Tip

Interviewers want the chain selector → EndpointSlice → node rules (DNAT) → CNI, and the point that a ClusterIP is virtual. Expect the follow-up about gRPC sending everything to one pod, and be ready to explain why readiness probes are what remove a pod from rotation during a rollout. A common mistake is saying kube-proxy forwards each request through itself; in iptables, IPVS and nftables modes the kernel does the forwarding.

12

Describe the purpose of Ingress and its key components.

Ingress exposes HTTP and HTTPS routes from outside the cluster to Services inside it, using one entry point for many services. An Ingress object declares rules such as “shop.example.com/api goes to the api Service, / goes to web”, plus TLS settings. It does nothing by itself: an Ingress controller (a reverse proxy such as NGINX, HAProxy, Traefik, Envoy-based controllers or a cloud load balancer integration) watches these objects and configures itself to match. The newer Gateway API is the successor for new designs.

Read full answer

Applies toKubernetes 1.29+ (`networking.k8s.io/v1`)

Why It Exists

Without Ingress, each public service needs its own LoadBalancer Service: one cloud load balancer and one IP per service, and no host or path routing. Ingress puts one L7 proxy in front and routes by host and path, terminates TLS once, and keeps routing rules in version-controlled manifests.

Request flow: Client → DNS shop.example.com → cloud LB → Ingress controller pods → Service api → pod.

Key Components

Component Role
Ingress resource Rules: hosts, paths, backends, TLS
Ingress controller The proxy that implements the rules; you install it (it is not part of core Kubernetes)
IngressClass Names which controller handles an Ingress (ingressClassName)
Backend Service A normal ClusterIP Service the rule forwards to
TLS Secret kubernetes.io/tls Secret with the certificate and key, often managed by cert-manager
Default backend Where requests matching no rule go

Example

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: shop
  namespace: shop
spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - shop.example.com
      secretName: shop-example-com-tls
  rules:
    - host: shop.example.com
      http:
        paths:
          - path: /api
            pathType: Prefix
            backend:
              service:
                name: api
                port:
                  number: 80
          - path: /
            pathType: Prefix
            backend:
              service:
                name: web
                port:
                  number: 80

pathType is required: Prefix matches by path segment (/api matches /api/orders but not /apiary), Exact matches one path, and ImplementationSpecific leaves it to the controller. The most specific match wins.

kubectl get ingress -n shop
kubectl describe ingress shop -n shop
kubectl get ingressclass

What Ingress Does Not Cover

The Ingress spec is deliberately small: hosts, paths, TLS. Rewrites, rate limits, timeouts, header manipulation, canary weights and authentication are controller-specific annotations, so manifests are not portable between controllers. It also handles only HTTP(S); raw TCP or UDP needs a LoadBalancer Service or controller-specific configuration.

Ingress versus Gateway API

Aspect Ingress Gateway API
Status Stable, feature-frozen Stable (v1) and actively developed
Roles One object for everything GatewayClass (infra provider), Gateway (platform team), HTTPRoute (app team)
Features Host/path routing, TLS Header matching, traffic weights, redirects, rewrites, mirroring, gRPC and TCP routes, in the spec
Portability Low; features live in annotations Higher; features are part of the API
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: api
  namespace: shop
spec:
  parentRefs:
    - name: public-gateway
      namespace: infra
  hostnames:
    - shop.example.com
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /api
      backendRefs:
        - name: api
          port: 80

The community ingress-nginx controller announced its retirement in 2025, which is one more reason new clusters often start with Gateway API or another actively maintained controller.

Trade-offs

  • One controller becomes a critical shared component: run several replicas with a PodDisruptionBudget, and watch its CPU and connection counts.

  • TLS terminates at the controller; traffic to pods is plain HTTP unless you add re-encryption or a mesh.

  • Misconfigured host rules or a missing ingressClassName mean the Ingress is silently ignored.

Interview Tip

Make the resource versus controller distinction clearly: creating an Ingress without a controller does nothing. Interviewers often ask how Ingress differs from a LoadBalancer Service (L7 routing for many services versus one L4 entry per service) and what you would use for a new cluster today (Gateway API).

13

Explain Kubernetes DNS resolution for services.

Every Service gets a DNS name of the form <service>.<namespace>.svc.<cluster-domain>, usually orders.shop.svc.cluster.local, served by CoreDNS running in the cluster. Pods are configured to use CoreDNS as their resolver, with search domains that let them use short names: orders from the same namespace, orders.shop from another. A normal Service name resolves to its ClusterIP; a headless Service name resolves to the individual pod IPs.

Read full answer

Applies toKubernetes 1.29+ with CoreDNS

How a Pod Resolves a Name

The kubelet writes each pod’s /etc/resolv.conf (with the default dnsPolicy: ClusterFirst):

nameserver 10.96.0.10
search shop.svc.cluster.local svc.cluster.local cluster.local
options ndots:5
  • nameserver is the ClusterIP of the kube-dns Service in front of the CoreDNS pods.

  • search lists the suffixes tried for short names, starting with the pod’s own namespace.

  • ndots:5: any name with fewer than five dots is tried with each search suffix before being tried as-is.

Lookup of orders from a pod in namespace shop:

  1. orders.shop.svc.cluster.local → found, returns 10.96.14.20.

Lookup of api.github.com (two dots) from the same pod:

  1. api.github.com.shop.svc.cluster.local → NXDOMAIN

  2. api.github.com.svc.cluster.local → NXDOMAIN

  3. api.github.com.cluster.local → NXDOMAIN

  4. api.github.com → forwarded upstream, found

Each step is often sent for both A and AAAA, so external lookups can cost several round trips. A trailing dot (api.github.com.) skips the search list.

Records CoreDNS Serves

Record Name Answer
A/AAAA, normal Service orders.shop.svc.cluster.local ClusterIP
A/AAAA, headless Service db.shop.svc.cluster.local All ready pod IPs
A/AAAA, StatefulSet pod db-0.db.shop.svc.cluster.local That pod’s IP
SRV _http._tcp.orders.shop.svc.cluster.local Port and target for named ports
CNAME, ExternalName Service billing.shop.svc.cluster.local The external hostname

Example: Headless Service for a StatefulSet

apiVersion: v1
kind: Service
metadata:
  name: db
  namespace: shop
spec:
  clusterIP: None
  selector:
    app: db
  ports:
    - name: postgres
      port: 5432

With serviceName: db in the StatefulSet, peers reach each other as db-0.db, db-1.db, which stay valid when the pods move to other nodes.

kubectl run dns-test --rm -it --image=busybox:1.36 -n shop -- nslookup orders
kubectl get svc kube-dns -n kube-system
kubectl logs -n kube-system -l k8s-app=kube-dns

Tuning and Options

Option Use
dnsConfig.options: ndots: "2" Fewer wasted lookups for apps that mostly call external names
Fully qualified names with trailing dot Same effect, per call
NodeLocal DNSCache A cache on every node; cuts latency and conntrack pressure on CoreDNS
dnsPolicy: Default Use the node’s resolver (no cluster names)
dnsPolicy: None with dnsConfig Fully custom resolver settings
CoreDNS forward and cache plugins Upstream servers and cache TTLs, in the coredns ConfigMap

Trade-offs and Failure Modes

  • CoreDNS is on the path of almost every connection. Run at least two replicas, scale it with cluster size, and monitor its latency and error rate.

  • Caching in clients: some runtimes cache DNS answers for a long time. For normal Services this is harmless because the ClusterIP rarely changes; for headless Services, stale pod IPs cause failures after rescheduling.

  • Cross-namespace calls need at least service.namespace; a bare name resolves only within the caller’s own namespace.

  • Intermittent 5-second delays have historically come from conntrack races on parallel A/AAAA UDP queries; NodeLocal DNSCache or the single-request-reopen resolver option are common fixes.

Interview Tip

Interviewers like the ndots:5 detail because it explains real latency problems with external calls. Also explain the headless versus ClusterIP difference and why StatefulSets need a headless Service. A common mistake is hard-coding pod IPs or forgetting the namespace part when calling a Service in another namespace.

14

What is the Container Network Interface (CNI), and how does it integrate with Kubernetes?

The Container Network Interface (CNI) is a specification and a set of plugins for configuring network interfaces in containers. Kubernetes does not implement pod networking itself: when the container runtime creates a pod, it calls a CNI plugin, which creates the pod’s interface, assigns an IP (through an IPAM plugin) and sets up the routes so the pod can reach every other pod. Calico, Cilium, Flannel and cloud plugins such as the AWS VPC CNI are common implementations.

Read full answer

Applies toKubernetes 1.29+ with containerd or CRI-O

The Kubernetes Network Model CNI Must Provide

  • Every pod gets its own IP address.

  • Pods can reach all other pods, on any node, without NAT.

  • Agents on a node (kubelet, system daemons) can reach all pods on that node.

How a plugin achieves this (overlay, routing, cloud network) is its choice.

How It Integrates

  1. The scheduler assigns a pod to a node; the kubelet asks the runtime (via CRI) to create the pod sandbox.

  2. The runtime (containerd or CRI-O) creates a network namespace and reads the config in /etc/cni/net.d/.

  3. It runs the plugin binary from /opt/cni/bin/ with the command ADD, passing the namespace path and pod details as environment variables and JSON on stdin.

  4. The plugin creates a virtual interface pair (or attaches a cloud interface), moves one end into the pod, calls IPAM for an IP, sets routes, and returns the result as JSON.

  5. On pod deletion, the runtime calls DEL to release the IP and clean up. CHECK verifies the setup.

Plugins are usually installed by a DaemonSet that copies the binary and config onto every node and runs a node agent. Until a CNI is installed, nodes report NotReady and pods stay Pending or ContainerCreating.

{
  "cniVersion": "1.0.0",
  "name": "k8s-pod-network",
  "plugins": [
    { "type": "calico", "ipam": { "type": "calico-ipam" } },
    { "type": "portmap", "capabilities": { "portMappings": true } }
  ]
}

Data Plane Approaches

Approach How pods on different nodes talk Pros Cons
Overlay (VXLAN, Geneve) Packets wrapped in UDP between nodes Works on any network Encapsulation overhead, smaller effective MTU
Routed / BGP Node routes advertise pod CIDRs No encapsulation, easy to trace Needs control of the underlying network
Cloud-native Pods get real VPC IPs (AWS VPC CNI, Azure CNI) Native performance, VPC features apply to pods Consumes VPC IP space; per-node pod limits
eBPF (Cilium) Kernel programs handle routing and policy Fast, can replace kube-proxy, rich visibility Needs a recent kernel; more to learn

Choosing a Plugin

Need Consider
Simple lab or small cluster Flannel (no NetworkPolicy enforcement on its own)
NetworkPolicy enforcement, BGP integration Calico
eBPF data plane, L7 policy, kube-proxy replacement Cilium
Pods as first-class VPC citizens on a cloud The provider’s CNI

NetworkPolicy is enforced by the CNI. If the plugin does not support it, NetworkPolicy objects are accepted by the API server and silently do nothing.

ls /etc/cni/net.d/ /opt/cni/bin/
kubectl get pods -n kube-system -o wide | grep -Ei 'calico|cilium|flannel'
kubectl get nodes -o custom-columns=NAME:.metadata.name,PODCIDR:.spec.podCIDR

Trade-offs and Operational Points

  • IP planning: pod CIDRs must not overlap with node, Service or corporate ranges; changing them later usually means rebuilding the cluster.

  • MTU: overlays reduce usable MTU; a mismatch causes hangs on large packets while small requests work.

  • Changing CNI on a live cluster is disruptive; choose early.

  • Multiple interfaces per pod (for telecom or storage networks) need a meta-plugin such as Multus.

  • CNI covers pod networking only; Service virtual IPs are handled by kube-proxy or the plugin’s replacement for it.

Interview Tip

Interviewers check that you know who calls CNI (the container runtime, triggered by the kubelet) and what Kubernetes requires (a flat, NAT-free pod network). A common mistake is assuming NetworkPolicies work on every cluster; they depend on the CNI plugin. Expect a follow-up on overlay versus routed networking and its MTU and performance costs.

15

How do you implement Network Policies, and what are their benefits?

A NetworkPolicy is a namespaced object that controls which traffic may reach (ingress) or leave (egress) a set of pods, selected by labels. By default every pod can talk to every other pod in the cluster; you implement policies by first applying a default deny in a namespace and then adding narrow allow rules for each real communication path. The benefit is a smaller blast radius: a compromised or misbehaving pod can reach only what it was explicitly allowed to.

Read full answer

Applies toKubernetes 1.29+ (`networking.k8s.io/v1`) with a CNI that enforces policies (Calico, Cilium, most cloud CNIs)

How Policies Are Evaluated

  • podSelector picks the pods the policy applies to; {} means all pods in the namespace.

  • Once any policy selects a pod for a direction (policyTypes: Ingress or Egress), that direction becomes deny-by-default for the pod, and only traffic matching some rule is allowed.

  • Policies are additive: the allowed set is the union of all policies selecting the pod. There is no “deny” rule in the core API.

  • Peers are matched by podSelector, namespaceSelector, both combined, or ipBlock (CIDRs), plus ports.

  • Return traffic for an allowed connection is always allowed (the rules are stateful).

Implementation Steps

  1. Confirm enforcement. Without a supporting CNI, policies are accepted and ignored.

  2. Map the traffic: which services call which, on which ports, and which pods need external access.

  3. Apply default deny for ingress and egress in the namespace.

  4. Allow DNS egress, or nothing can resolve names.

  5. Add one allow policy per path, using workload labels.

  6. Test from inside pods, then roll out namespace by namespace, starting with non-production.

Example: Web → API → Database

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny
  namespace: shop
spec:
  podSelector: {}
  policyTypes: ["Ingress", "Egress"]
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-dns
  namespace: shop
spec:
  podSelector: {}
  policyTypes: ["Egress"]
  egress:
    - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: kube-system
          podSelector:
            matchLabels:
              k8s-app: kube-dns
      ports:
        - protocol: UDP
          port: 53
        - protocol: TCP
          port: 53
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: api-from-web
  namespace: shop
spec:
  podSelector:
    matchLabels:
      app: api
  policyTypes: ["Ingress"]
  ingress:
    - from:
        - podSelector:
            matchLabels:
              app: web
      ports:
        - protocol: TCP
          port: 8080
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: web-to-api-egress
  namespace: shop
spec:
  podSelector:
    matchLabels:
      app: web
  policyTypes: ["Egress"]
  egress:
    - to:
        - podSelector:
            matchLabels:
              app: api
      ports:
        - protocol: TCP
          port: 8080

The allow-dns rule puts namespaceSelector and podSelector in one list item, meaning “pods labelled kube-dns in kube-system”. Written as two separate items (- namespaceSelector and - podSelector), it would mean “anything in kube-system or pods labelled kube-dns in this namespace”, a common and dangerous mistake. Each direction needs its own allow: the web pod needs egress to the API, and the API needs ingress from the web pod. Policies for api → db and for the ingress controller reaching web follow the same pattern.

Testing

kubectl get networkpolicy -n shop
kubectl exec -n shop deploy/web -- wget -qO- -T 2 http://api:8080/healthz
kubectl exec -n shop deploy/web -- wget -qO- -T 2 http://db:5432 || echo blocked

Benefits

Benefit Example
Limits lateral movement A compromised web pod cannot reach the database directly
Enforces architecture Only the API layer talks to storage
Multi-tenant separation Team namespaces cannot reach each other
Egress control Pods cannot call arbitrary internet hosts and exfiltrate data
Compliance evidence Declared, reviewable rules for segmented networks (for example PCI DSS scopes)

Limits and Trade-offs

  • L3/L4 only: IPs, ports, protocols. No HTTP paths or methods, and no FQDN rules; Cilium and Calico add those through their own CRDs.

  • Operational cost: every new dependency needs a policy change, and a missing rule looks like a timeout. Keep policies next to the app’s manifests and test them in CI.

  • ipBlock is brittle for pod traffic because pod IPs change; use selectors for in-cluster peers.

  • Node and host-network traffic behaves differently between CNIs; check health checks from load balancers still work.

Interview Tip

Interviewers look for default deny plus explicit allows, the DNS egress rule, and the selector-in-one-item versus two-items difference. A common mistake is writing perfect policies on a cluster whose CNI does not enforce them. Mention that a service mesh with mTLS adds identity-based L7 authorization on top, but does not replace network-level policies.

16

How do PersistentVolumes (PVs) and PersistentVolumeClaims (PVCs) interact in Kubernetes?

A PersistentVolume (PV) is a piece of real storage in the cluster (a cloud disk, an NFS export, a SAN volume), represented as a cluster-scoped object. A PersistentVolumeClaim (PVC) is a namespaced request for storage by an application: “20 GiB, ReadWriteOnce, class fast-ssd”. Kubernetes binds each claim to exactly one matching volume, and pods mount the claim, never the volume directly. This separates what the app needs from how the storage is provided.

Read full answer

Applies toKubernetes 1.29+ with CSI drivers

Who Does What

Role Object Concern
Cluster or storage admin StorageClass, PV Which storage exists, its performance and reclaim rules
Application team PVC How much storage, which access mode, which class
Pod volumes.persistentVolumeClaim Where to mount it

Binding

  1. A PVC is created with a size, access modes and a storageClassName.

  2. The PV controller looks for an Available PV with the same class, the requested access modes and at least the requested capacity. A claim for 20 Gi can bind a 25 Gi volume, and it gets all 25 Gi.

  3. If none fits and the class supports dynamic provisioning, a new PV is created for the claim.

  4. Binding is one-to-one and exclusive: the PV records claimRef, the PVC records volumeName, and both become Bound.

  5. A pod that references the PVC is scheduled where the volume can be attached; the kubelet and CSI driver attach and mount it.

Example: Static PV and a Claim

apiVersion: v1
kind: PersistentVolume
metadata:
  name: reports-nfs
spec:
  capacity:
    storage: 100Gi
  accessModes: ["ReadWriteMany"]
  persistentVolumeReclaimPolicy: Retain
  storageClassName: nfs
  nfs:
    server: 10.0.5.20
    path: /exports/reports
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: reports
  namespace: analytics
spec:
  accessModes: ["ReadWriteMany"]
  storageClassName: nfs
  resources:
    requests:
      storage: 50Gi
---
apiVersion: v1
kind: Pod
metadata:
  name: report-writer
  namespace: analytics
spec:
  containers:
    - name: app
      image: registry.example.com/report-writer:1.0.0
      volumeMounts:
        - name: data
          mountPath: /data
  volumes:
    - name: data
      persistentVolumeClaim:
        claimName: reports
kubectl get pv
kubectl get pvc -n analytics
kubectl describe pvc reports -n analytics

Access Modes

Mode Meaning Typical backends
ReadWriteOnce (RWO) Read-write by pods on one node Cloud block disks (EBS, Persistent Disk)
ReadWriteOncePod (RWOP) Read-write by one pod only CSI block volumes that need strict single-writer
ReadOnlyMany (ROX) Read-only by many nodes Shared datasets
ReadWriteMany (RWX) Read-write by many nodes NFS, CephFS, cloud file services

An access mode is a capability the volume must support, not a lock enforced on every backend; RWO allows several pods on the same node.

Lifecycle and Reclaim

Phase Meaning
Available Not bound yet
Bound Attached to a claim
Released Claim deleted; data still on the volume, not reusable automatically
Failed Automatic reclamation failed

persistentVolumeReclaimPolicy decides what happens when the claim is deleted: Delete removes the underlying disk (the default for dynamically provisioned volumes), Retain keeps it for manual recovery. Protection finalizers stop a PVC from being deleted while a pod uses it, and a PV while it is bound.

Trade-offs

  • Static PVs give admins full control but need manual creation for every claim; dynamic provisioning through a StorageClass is the norm in cloud clusters.

  • Block volumes are zonal: a pod using an RWO disk in zone A cannot run in zone B. Use WaitForFirstConsumer binding so the disk is created where the pod is scheduled.

  • RWX file storage is convenient but usually slower and more expensive than block storage, and concurrent writers need application-level coordination.

  • A PVC is not a backup. Use volume snapshots or an external backup tool.

Interview Tip

Use the analogy interviewers like: a PV is like a node (supply), a PVC is like a pod’s resource request (demand), and binding matches them. Then show depth on access modes (RWO is per node, not per pod), reclaim policy (Delete can destroy data when someone deletes a namespace) and zonal scheduling. A common mistake is giving a multi-replica Deployment one RWO claim and wondering why only one replica starts.

17

Explain dynamic provisioning in relation to StorageClasses.

Dynamic provisioning creates a PersistentVolume automatically when a PersistentVolumeClaim asks for one, instead of an administrator creating volumes in advance. The claim names a StorageClass, which says which provisioner (usually a CSI driver), which parameters (disk type, IOPS, encryption, filesystem) and which policies (reclaim, expansion, binding timing) to use. The provisioner calls the storage backend’s API, creates the disk, and registers it as a PV bound to the claim.

Read full answer

Applies toKubernetes 1.29+ (`storage.k8s.io/v1`) with CSI drivers

The Flow

  1. An app creates a PVC with storageClassName: fast-ssd (or no class, which uses the default class).

  2. With volumeBindingMode: WaitForFirstConsumer, nothing happens until a pod using the claim is scheduled; the scheduler picks a node, which decides the zone.

  3. The CSI external-provisioner sidecar sees the claim and calls the driver’s CreateVolume.

  4. The driver creates the disk through the cloud or storage API.

  5. A PV is created with the class’s reclaim policy and bound to the claim.

  6. The kubelet and CSI node plugin attach and mount the disk into the pod.

Example

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: fast-ssd
  annotations:
    storageclass.kubernetes.io/is-default-class: "true"
provisioner: ebs.csi.aws.com
parameters:
  type: gp3
  iops: "6000"
  throughput: "250"
  encrypted: "true"
  csi.storage.k8s.io/fstype: ext4
reclaimPolicy: Delete
allowVolumeExpansion: true
volumeBindingMode: WaitForFirstConsumer
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: orders-db
  namespace: shop
spec:
  accessModes: ["ReadWriteOnce"]
  storageClassName: fast-ssd
  resources:
    requests:
      storage: 50Gi

parameters are opaque to Kubernetes and specific to the driver; the AWS EBS keys above would mean nothing to a Ceph or Azure driver.

kubectl get storageclass
kubectl get pvc orders-db -n shop -w
kubectl get pv

StorageClass Fields

Field Effect Choice
provisioner Which driver creates volumes CSI driver name
parameters Backend options Disk type, IOPS, encryption, replication
reclaimPolicy What happens to the disk when the claim is deleted Delete (default) or Retain
volumeBindingMode When to provision WaitForFirstConsumer for zonal block storage; Immediate for storage reachable from every node
allowVolumeExpansion Whether claims can grow by editing spec.resources.requests.storage Usually true
mountOptions Options passed at mount time For example NFS versions
allowedTopologies Restrict zones or regions Data residency, cost

Designing Classes

A platform team usually offers a small menu:

Class Backend For
standard (default) General-purpose SSD, Delete Most apps
fast-ssd Provisioned IOPS Databases
retain-ssd Same disk, Retain Data that must survive accidental deletion
shared-files NFS or cloud file service, RWX Content shared by many pods

Only one class should carry the default annotation. A claim with storageClassName: "" explicitly asks for no dynamic provisioning and binds only to a pre-created PV.

Trade-offs

Benefit Cost or risk
Self-service storage for teams, no tickets Easy to create many expensive volumes; set ResourceQuotas on requests.storage and per-class counts
Consistent settings (encryption, type) Parameters are fixed per class; changing a class does not change existing volumes
Zone-aware placement with WaitForFirstConsumer A pod is then tied to the zone of its disk
Delete keeps the cloud bill clean Deleting a namespace or claim deletes the data

Expansion is online for most CSI drivers, but shrinking is not supported; size growth is one-way.

Interview Tip

Interviewers expect the chain PVC → StorageClass → CSI provisioner → PV and the reason for WaitForFirstConsumer: with Immediate binding a disk can be created in zone A while the pod is scheduled in zone B and never starts. A common mistake is leaving production databases on a Delete class with no snapshots or backups.

18

What strategies exist for handling persistent data when a pod’s lifecycle ends?

Anything written to a container’s own filesystem, or to an emptyDir volume, is lost when the pod is deleted. To keep data beyond a pod’s life you either store it on a PersistentVolumeClaim, which outlives pods and is re-attached to their replacements, or keep it outside the cluster in a managed database or object storage. On top of that you add snapshots and backups, because persistent volumes protect against pod loss, not against deletion, corruption or zone failure.

Read full answer

Applies toKubernetes 1.29+ with CSI drivers

What Survives What

Storage Container restart Pod deleted or rescheduled Node lost Namespace deleted
Container filesystem No No No No
emptyDir Yes No No No
hostPath Yes Only if rescheduled to the same node No Yes, but tied to one node
PVC (network block or file storage) Yes Yes Yes (re-attached elsewhere, same zone) Depends on reclaim policy
Managed database or object storage Yes Yes Yes Yes

Strategy 1: PVC with a Deployment

For one writer, such as a small internal tool, a single-replica Deployment with an RWO claim works. Set strategy: Recreate, because a rolling update would start the new pod while the old one still holds the disk.

Strategy 2: StatefulSet with Per-Replica Volumes

Each replica gets its own claim from volumeClaimTemplates (data-db-0, data-db-1). When db-1 is rescheduled it re-attaches data-db-1. You also decide what happens to claims when the set is deleted or scaled down:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: db
spec:
  serviceName: db
  replicas: 3
  persistentVolumeClaimRetentionPolicy:
    whenDeleted: Retain
    whenScaled: Delete
  selector:
    matchLabels:
      app: db
  template:
    metadata:
      labels:
        app: db
    spec:
      terminationGracePeriodSeconds: 60
      containers:
        - name: postgres
          image: postgres:16
          volumeMounts:
            - name: data
              mountPath: /var/lib/postgresql/data
  volumeClaimTemplates:
    - metadata:
        name: data
      spec:
        accessModes: ["ReadWriteOnce"]
        storageClassName: retain-ssd
        resources:
          requests:
            storage: 50Gi

whenDeleted: Retain keeps the data if someone deletes the StatefulSet; whenScaled: Delete cleans up claims of removed replicas. The default is Retain for both.

Strategy 3: Keep State Out of the Cluster

Use a managed database (RDS, Cloud SQL), object storage (S3, GCS) or a managed cache, and keep pods stateless. This is often the simplest answer for production: backups, failover and patching become the provider’s job, at the cost of money and some lock-in.

Strategy 4: Snapshots and Backups

  • CSI VolumeSnapshots take point-in-time copies of a claim; a new claim can be restored from one with dataSource.

  • Application-level backups (pg_dump, WAL archiving to object storage) give consistent, portable copies.

  • Cluster backup tools such as Velero capture manifests plus volume snapshots for disaster recovery.

A crash-consistent disk snapshot of a running database may need recovery on restore; quiesce the app or use the database’s own backup method for guaranteed consistency.

Protecting Against Accidental Loss

Measure Protects against
reclaimPolicy: Retain class for important data Disk deleted with the claim
StatefulSet whenDeleted: Retain Claims deleted with the set
RBAC limiting delete on PVCs and namespaces Human error
Scheduled snapshots copied to another region Zone or region failure
Graceful shutdown (SIGTERM handling, grace period) Corruption from abrupt termination

Trade-offs

  • PVCs on block storage are zonal: a pod can only move within the zone of its disk. Cross-zone durability needs application replication or regional disks.

  • RWX file storage lets pods share data but is slower and needs write coordination.

  • Running databases on Kubernetes is reasonable with an operator (CloudNativePG, Strimzi) that handles replication and failover; doing it by hand with a bare StatefulSet is where data loss usually happens.

Interview Tip

Interviewers want you to separate pod lifecycle from data lifecycle, and to say that a PVC is not a backup. Mention reclaim policies, StatefulSet retention policies and zonal limits. A common mistake is storing uploads or session files in the container filesystem, which works until the first rollout deletes them.

19

Describe the process for initializing a Kubernetes cluster.

Initializing a self-managed cluster with kubeadm follows a fixed order: prepare every machine (container runtime, kernel settings, kubelet), run kubeadm init on the first control plane node to generate certificates, start etcd and the control plane, and create a join token; install a CNI network plugin; then run kubeadm join on the other control plane and worker nodes. On a managed service (EKS, GKE, AKS) the provider does the control plane steps, and you create the cluster and node pools with its CLI or Terraform.

Read full answer

Applies toKubernetes 1.31+ with kubeadm and containerd on Linux

1. Prepare Every Node

  • Supported Linux, unique hostname, MAC and product_uuid; nodes reach each other on the required ports (6443 for the API server, 2379–2380 for etcd, 10250 for the kubelet).

  • Disable swap, or configure the kubelet’s swap support explicitly.

  • Load kernel modules overlay and br_netfilter; enable net.ipv4.ip_forward.

  • Install containerd (or CRI-O) with the systemd cgroup driver, matching the kubelet.

  • Install kubeadm, kubelet and kubectl of the same minor version and hold them against automatic upgrades.

sudo modprobe overlay && sudo modprobe br_netfilter
echo 'net.ipv4.ip_forward = 1' | sudo tee /etc/sysctl.d/k8s.conf && sudo sysctl --system
sudo systemctl enable --now containerd kubelet

2. Initialize the First Control Plane Node

apiVersion: kubeadm.k8s.io/v1beta4
kind: ClusterConfiguration
kubernetesVersion: v1.31.2
controlPlaneEndpoint: k8s-api.internal.example.com:6443
networking:
  podSubnet: 10.244.0.0/16
  serviceSubnet: 10.96.0.0/12
sudo kubeadm init --config kubeadm.yaml --upload-certs
mkdir -p $HOME/.kube && sudo cp /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

controlPlaneEndpoint should be a DNS name or load balancer address, not one node’s IP, so more control plane nodes can be added later. kubeadm init runs these phases:

Phase What it does
preflight Checks ports, swap, runtime, versions
certs Creates the cluster CA and component certificates in /etc/kubernetes/pki
kubeconfig Writes kubeconfig files for admin, kubelet, controller manager, scheduler
kubelet-start Configures and starts the kubelet
control-plane, etcd Writes static pod manifests in /etc/kubernetes/manifests, which the kubelet runs
upload-config, upload-certs Stores the config and (encrypted) certificates in the cluster for joining nodes
mark-control-plane Adds the control-plane label and NoSchedule taint
bootstrap-token Creates a token that new nodes use to join
addons Installs CoreDNS and kube-proxy

3. Install a CNI Plugin

Until a network plugin is installed, nodes stay NotReady and CoreDNS stays Pending. Apply the manifests or Helm chart of Calico, Cilium or another plugin, configured with the same pod CIDR.

4. Join the Other Nodes

# Additional control plane nodes (three in total for HA)
sudo kubeadm join k8s-api.internal.example.com:6443 --token <token> \
  --discovery-token-ca-cert-hash sha256:<hash> --control-plane --certificate-key <key>

# Worker nodes
sudo kubeadm join k8s-api.internal.example.com:6443 --token <token> \
  --discovery-token-ca-cert-hash sha256:<hash>

kubeadm token create --print-join-command   # tokens expire after 24 hours by default

The CA hash lets the new node verify it is talking to the right cluster before trusting it.

5. Verify and Finish

kubectl get nodes
kubectl get pods -n kube-system

Then add what a real cluster needs: a metrics server, an ingress or Gateway controller, a CSI storage driver and StorageClass, monitoring and logging, RBAC for users, and etcd backups (etcdctl snapshot save on a schedule).

Managed versus kubeadm

Aspect kubeadm Managed service
Control plane, etcd, certificates You Provider
Upgrades kubeadm upgrade node by node Provider-driven, you trigger and upgrade node pools
Cost Machines plus operations time Cluster fee plus nodes
When On-premises, edge, learning, full control Most cloud production workloads

Interview Tip

Interviewers look for the order (runtime and kubelet → init → CNI → join) and for HA thinking: a stable controlPlaneEndpoint, three control plane nodes for etcd quorum, and etcd backups. A common mistake is a cgroup driver mismatch between containerd and the kubelet, which makes the kubelet crash-loop; another is choosing a pod CIDR that overlaps the corporate network. Also mention that kubeadm certificates expire after one year unless renewed, which upgrades do automatically.

20

What factors need to be considered when adding new nodes to a Kubernetes cluster?

Adding nodes is more than adding capacity. Check that the new nodes are compatible (Kubernetes version skew, OS, runtime, CNI), correctly sized and placed (instance type, zones, IP space), labelled and tainted so the right workloads land there, secured and joined with proper credentials, and that the cluster’s autoscaling, quotas and cost account for them. Treat nodes as identical, replaceable cattle created from a template, not hand-built machines.

Read full answer

Applies toKubernetes 1.29+ (managed node groups, Karpenter, Cluster Autoscaler or kubeadm)

Checklist

Area What to check
Version skew kubelet must not be newer than the API server, and at most three minor versions older (1.28+); keep nodes on the control plane’s version where possible
OS and runtime Same OS image, kernel and containerd version as the rest of the pool; cgroup v2; systemd cgroup driver
CNI and IP space Enough pod IPs: a node’s pod CIDR or, with VPC CNIs, the instance’s IP limit decides max pods; subnet must have free addresses
Max pods Default kubelet limit is 110 pods per node; cloud CNIs may set lower limits by instance type
Sizing Fewer large nodes pack better and waste less per-node overhead; more small nodes reduce the blast radius of one failure
Reserved resources kube-reserved, system-reserved and eviction thresholds so system daemons are not starved
DaemonSet overhead Every DaemonSet (logging, monitoring, CNI, CSI) runs on the new node; subtract it from usable capacity
Zones Spread nodes evenly across availability zones; zonal volumes need nodes in their zone
Labels and taints Pool labels (pool=gpu), zone labels, and taints for special hardware so only matching workloads use it
Storage CSI node plugin present; disk attach limits per instance type
Security Hardened image, no SSH keys shared, minimal IAM role for the node, kubelet authentication and authorization enabled, node restriction admission
Joining Short-lived bootstrap tokens or cloud identity; TLS bootstrapping for kubelet certificates
Observability Node exporter, log agent and alerts working before workloads arrive
Cost and quotas Cloud quotas (vCPU, IPs), budget, reserved or spot capacity mix

Example: Dedicated GPU Pool

Label and taint the nodes, then let only GPU workloads tolerate the taint:

kubectl label node gpu-node-1 pool=gpu
kubectl taint node gpu-node-1 nvidia.com/gpu=present:NoSchedule
apiVersion: apps/v1
kind: Deployment
metadata:
  name: inference
spec:
  replicas: 2
  selector:
    matchLabels:
      app: inference
  template:
    metadata:
      labels:
        app: inference
    spec:
      nodeSelector:
        pool: gpu
      tolerations:
        - key: nvidia.com/gpu
          operator: Equal
          value: present
          effect: NoSchedule
      containers:
        - name: model
          image: registry.example.com/inference:0.9.0
          resources:
            limits:
              nvidia.com/gpu: 1

The GPU itself also needs the vendor’s device plugin DaemonSet, which advertises nvidia.com/gpu as a schedulable resource. In practice, set labels and taints in the node group or kubelet configuration so every new node gets them automatically.

Manual versus Automatic Scaling

Approach How Trade-off
Manual (kubeadm join, resize a node group) Operator adds nodes Predictable; slow to react
Cluster Autoscaler Adds nodes when pods are Pending for lack of resources, removes underused ones Works with node groups; scale-up takes minutes
Karpenter (AWS, others) Picks instance types per pending pod Better bin-packing; more moving parts

Autoscalers act on resource requests, not actual usage, so pods without requests never trigger scale-up correctly.

After Adding Nodes

  • Existing pods do not move to new nodes automatically. The scheduler only places new pods. Rebalancing needs a rollout, the descheduler, or topology spread constraints on the next rollout.

  • Verify with kubectl get nodes -L topology.kubernetes.io/zone,pool and kubectl describe node (allocatable resources, taints, conditions).

  • Watch that DaemonSets became ready on the new nodes.

Interview Tip

Interviewers want to hear about version skew, IP capacity, labels and taints, zones and autoscaling, not just “run kubeadm join”. A frequent follow-up is “you added nodes but pods are still Pending; why?”: usually taints without tolerations, node selectors, exhausted IPs, or zonal volumes in another zone. A common mistake is assuming new nodes relieve hot existing nodes; without rescheduling, they stay empty.

21

How does draining a node affect running applications, and when should you drain?

Draining a node marks it unschedulable (cordon) and then evicts its pods so they are recreated on other nodes. For applications, each evicted pod receives SIGTERM, has its grace period to shut down, and is replaced by its controller elsewhere, so capacity drops temporarily and pods get new names and IPs. Evictions go through the Eviction API, which respects PodDisruptionBudgets, so a well-configured service stays available. Drain before planned maintenance: OS or kernel patching, Kubernetes upgrades, hardware work, or removing the node.

Read full answer

Applies toKubernetes 1.29+ (`kubectl drain`, `policy/v1` PodDisruptionBudget)

What kubectl drain Does

  1. Cordon: sets spec.unschedulable: true; running pods stay, but no new pods are placed there.

  2. Lists the pods on the node and evicts them, except DaemonSet pods (with --ignore-daemonsets) and static mirror pods.

  3. Each eviction is checked against PodDisruptionBudgets; if it would violate one, the API returns 429 Too Many Requests and drain retries.

  4. The kubelet terminates each pod: preStop hook, SIGTERM, then SIGKILL after terminationGracePeriodSeconds.

  5. Controllers (Deployment, StatefulSet, Job) create replacements on other nodes.

kubectl cordon node-7
kubectl drain node-7 --ignore-daemonsets --delete-emptydir-data --timeout=10m
# ... maintenance ...
kubectl uncordon node-7

Effects on Applications

Workload What happens Watch out for
Deployment replicas Recreated elsewhere; Service stops sending traffic once they terminate Enough spare capacity on other nodes
StatefulSet pods Same name recreated elsewhere, re-attaches its volume Zonal disk must have a node in the same zone; startup and data recovery time
Pods with emptyDir Data lost; drain refuses without --delete-emptydir-data Caches and scratch files
Bare pods (no controller) Deleted, not recreated; drain refuses without --force Find and fix these first
DaemonSet pods Left in place (they would be recreated on the same node) Stop by the maintenance itself
Jobs Running pods evicted and retried Non-idempotent work; backoffLimit
Long-lived connections Cut when the pod terminates Clients must reconnect and retry

Keep Services Available with a PDB

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: checkout-pdb
spec:
  maxUnavailable: 1
  selector:
    matchLabels:
      app: checkout

With this budget, a drain evicts only one checkout pod at a time and waits until its replacement is ready before evicting the next. Pitfall: a budget that can never be satisfied, such as minAvailable: 3 on a three-replica Deployment or any budget on a single replica, blocks the drain forever. Use maxUnavailable: 1, or run at least two replicas.

When to Drain

  • Kernel, OS or runtime patching that needs a reboot.

  • Kubernetes upgrades: drain, upgrade kubelet, uncordon, node by node (managed node groups do this for you).

  • Hardware or VM maintenance, or moving to a new instance type.

  • Decommissioning a node (drain, then delete the node object and the machine).

  • Unhealthy node that is still running pods badly (disk pressure, flaky network).

Do not drain for a simple app restart (use kubectl rollout restart), and avoid draining many nodes at once without checking capacity; draining nodes in a single zone concentrates load in the others.

Preparing for Drains

  • At least two replicas, spread across nodes with topology spread constraints.

  • PDBs for important workloads, with values that allow at least one disruption.

  • Correct readiness probes and graceful shutdown so in-flight requests complete.

  • Spare capacity, or an autoscaler that adds nodes when evicted pods become Pending.

  • Cloud spot interruptions behave like a short-notice drain; a node termination handler can trigger the drain automatically.

Interview Tip

Interviewers look for cordon versus drain, the Eviction API and PDBs, and the flags that protect you (--ignore-daemonsets, why --force and --delete-emptydir-data exist). A common mistake is a PDB that blocks every eviction, leaving a node upgrade stuck for hours. Another is assuming bare pods come back after a drain; they do not.

22

When would you scale a Kubernetes deployment, and which tools would you use?

Scale a Deployment when demand or risk changes: traffic grows or follows a daily pattern, a queue backlog builds up, latency breaks its target, you need more replicas for resilience during maintenance, or you want to save money at quiet times. The tools form layers: kubectl scale for one-off changes, the Horizontal Pod Autoscaler (HPA) for replica count driven by metrics, KEDA for event-driven scaling (including to zero), the Vertical Pod Autoscaler (VPA) for right-sizing requests, and Cluster Autoscaler or Karpenter to add nodes when pods no longer fit.

Read full answer

Applies toKubernetes 1.29+ (`autoscaling/v2`)

Scenarios and Tools

Scenario Signal Tool
Traffic spike on a web API CPU, requests per second, latency HPA on CPU or a custom metric
Queue workers falling behind Queue length, consumer lag KEDA (Kafka, RabbitMQ, SQS scalers) or HPA on an external metric
Predictable daily or weekly peaks Clock KEDA cron scaler, or a scheduled job that raises minReplicas
Rare workloads idle most of the day Events KEDA scale to zero
Pods sized wrongly (OOM kills, wasted CPU) Historical usage VPA in recommendation mode
Pods Pending for lack of capacity Unschedulable pods Cluster Autoscaler or Karpenter
Maintenance or incident Human decision kubectl scale or a higher minReplicas

HPA Example

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: checkout
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: checkout
  minReplicas: 3
  maxReplicas: 30
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 65
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 0
      policies:
        - type: Percent
          value: 100
          periodSeconds: 60
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
        - type: Pods
          value: 2
          periodSeconds: 60

The HPA computes desired = ceil(current × currentMetric / target). With 4 pods at 90% CPU against a 65% target, it asks for ceil(4 × 90 / 65) = 6. Utilization is measured against the container’s CPU request, so pods without requests cannot be scaled on CPU. The behavior block scales up quickly and down slowly to avoid flapping.

kubectl scale deployment/checkout --replicas=8
kubectl get hpa checkout -w
kubectl describe hpa checkout

Once an HPA manages a Deployment, remove replicas from the Deployment manifest; otherwise every kubectl apply resets the count.

Choosing a Metric

Metric Good for Caveat
CPU CPU-bound services Lags for I/O-bound apps
Memory Rarely; memory often does not drop after load Can scale up and never down
Requests per second per pod Web APIs Needs Prometheus Adapter or similar
Queue length or lag Workers Needs KEDA or an external metrics adapter
Latency User-facing SLOs Noisy; combine with a throughput metric

Things to Check Before Relying on Autoscaling

  • Start-up time: if a pod takes 90 s to become ready, scaling reacts too late for sharp spikes. Keep headroom with minReplicas, or use scheduled scaling before known peaks.

  • Node capacity: HPA adds pods; if nodes are full they stay Pending until the cluster autoscaler adds a node, which takes minutes.

  • Downstream limits: 30 replicas with 20 database connections each may exhaust the database. Scale the bottleneck, not just the front.

  • HPA and VPA conflict if both act on CPU or memory for the same pods. Use VPA in recommendation mode, or HPA on custom metrics only.

  • PodDisruptionBudgets and topology spread still apply to the new replicas.

  • Cost: set maxReplicas deliberately; an attack or a bug can otherwise scale you into a large bill.

Vertical versus Horizontal

Approach Changes Good for Limit
Horizontal (HPA, KEDA) Number of pods Stateless services Shared bottlenecks such as the database
Vertical (VPA) CPU and memory per pod Right-sizing, singletons Usually restarts pods to apply new values
Cluster (Cluster Autoscaler, Karpenter) Number of nodes Making room for pods Minutes of delay; driven by requests

Interview Tip

Interviewers want to see that you pick the signal first and the tool second, and that you remember HPA depends on resource requests. Mention the cluster autoscaler as the layer that makes HPA scale-ups actually schedulable. A common mistake is autoscaling on memory, or setting maxReplicas without checking what the database can handle.

23

Explain Role-Based Access Control (RBAC) in Kubernetes.

RBAC decides what an authenticated identity may do through the Kubernetes API. A Role (namespaced) or ClusterRole (cluster-wide) lists allowed verbs (get, list, create, delete…) on resources (pods, deployments, secrets…). A RoleBinding or ClusterRoleBinding grants that role to subjects: users, groups or ServiceAccounts. Permissions are purely additive; there are no deny rules, so anything not granted is forbidden.

Read full answer

Applies toKubernetes 1.29+ (`rbac.authorization.k8s.io/v1`)

The Four Objects

Object Scope Purpose
Role One namespace Permissions inside that namespace
ClusterRole Cluster Permissions on cluster-scoped resources (nodes, PVs), on all namespaces, or a reusable template
RoleBinding One namespace Grants a Role or a ClusterRole within that namespace
ClusterRoleBinding Cluster Grants a ClusterRole everywhere

Binding a ClusterRole with a RoleBinding is the common pattern: define deployer once, grant it per namespace.

Authentication Comes First

Kubernetes has no user objects. Users come from client certificates, OIDC tokens (from your identity provider) or cloud IAM integration; their groups come from the same source. Workloads use ServiceAccounts, which are real API objects with short-lived tokens projected into pods. RBAC only decides authorization after authentication succeeds.

Example: A Deploy Pipeline and a Read-Only Team

apiVersion: v1
kind: ServiceAccount
metadata:
  name: ci-deployer
  namespace: shop
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: deployer
  namespace: shop
rules:
  - apiGroups: ["apps"]
    resources: ["deployments"]
    verbs: ["get", "list", "watch", "update", "patch"]
  - apiGroups: [""]
    resources: ["services", "configmaps"]
    verbs: ["get", "list", "create", "update", "patch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: ci-deployer
  namespace: shop
subjects:
  - kind: ServiceAccount
    name: ci-deployer
    namespace: shop
roleRef:
  kind: Role
  name: deployer
  apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: support-view
  namespace: shop
subjects:
  - kind: Group
    name: support-team
    apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: view
  apiGroup: rbac.authorization.k8s.io

apiGroups: [""] is the core group (pods, services, secrets). The built-in view role deliberately excludes Secrets.

kubectl auth can-i update deployments -n shop --as=system:serviceaccount:shop:ci-deployer
kubectl auth can-i delete secrets -n shop --as=jane --as-group=support-team
kubectl auth can-i --list -n shop --as=system:serviceaccount:shop:ci-deployer

Built-in ClusterRoles

Role Grants
view Read most objects, not Secrets
edit Create and change workloads and Secrets; no RBAC changes
admin Everything in a namespace, including Roles and RoleBindings
cluster-admin Everything, everywhere

Permissions That Are More Powerful Than They Look

Permission Why it is risky
get/list on secrets Reads every credential in the namespace
create on pods (or Deployments) Can run a pod using any ServiceAccount in the namespace and mount its Secrets
create on pods/exec Shell into running containers
bind, escalate on roles Grant yourself more than you have
impersonate Act as another user or group
Wildcards * Include resources added later, such as CRDs
nodes/proxy Direct access to kubelet APIs

Best Practices

  • Least privilege: namespaced Roles, explicit verbs and resources, no wildcards.

  • Bind to groups from your identity provider, not to individual users.

  • Give each workload its own ServiceAccount; set automountServiceAccountToken: false for pods that do not call the API.

  • Reserve cluster-admin for break-glass access, with audit logging.

  • Review bindings regularly; tools such as kubectl-who-can or rbac-lookup help.

  • Keep RBAC manifests in Git and review them like code.

Interview Tip

Interviewers want the Role/ClusterRole versus binding split, the fact that RBAC is additive with no deny, and awareness of escalation paths like create pods or list secrets. A common mistake is granting edit or cluster-admin to a CI pipeline “to make it work”; a scoped Role for the resources it deploys is the expected answer.

24

Describe how to securely manage Secrets in Kubernetes.

A Kubernetes Secret is only base64-encoded by default, not encrypted: anyone who can read it from the API or from etcd sees the value. Managing Secrets securely means encrypting them at rest in etcd (ideally with a cloud KMS), restricting RBAC read access to the few identities that need them, keeping them out of Git in plain form, preferring an external secret store (Vault, AWS Secrets Manager, Azure Key Vault, Google Secret Manager) as the source of truth, and rotating them.

Read full answer

Applies toKubernetes 1.29+

Threats and Controls

Threat Control
etcd backup or disk is stolen Encryption at rest with a KMS provider
Developer or pipeline reads Secrets it does not need RBAC: no get/list/watch on secrets except for owners
Someone who can create pods mounts any Secret in the namespace Separate namespaces per trust level; admission policies
Secret committed to Git Sealed Secrets, SOPS, or an External Secrets operator; secret scanning in CI
Value leaks through logs or crash dumps Mount as files, not env vars; never log config
Long-lived credential stolen Rotation, short-lived credentials, workload identity

1. Encrypt at Rest

The API server encrypts Secrets before writing them to etcd, configured by an EncryptionConfiguration file:

apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
  - resources: ["secrets"]
    providers:
      - kms:
          apiVersion: v2
          name: cloud-kms
          endpoint: unix:///var/run/kms-plugin/socket.sock
          timeout: 3s
      - identity: {}

The KMS v2 provider (stable since 1.29) uses envelope encryption: data keys are encrypted by a key that never leaves the KMS. Local aescbc or secretbox keys work too, but the key then sits on the control plane disk. After enabling it, rewrite existing Secrets (kubectl get secrets -A -o json | kubectl replace -f -) so old ones are encrypted. Managed services offer this as a setting.

2. Restrict Access

  • Grant get on specific Secrets with resourceNames where possible; avoid list and watch, which return every value.

  • Remember that create pods in a namespace effectively grants reading its Secrets.

  • Enable audit logging for Secret reads.

3. Use an External Store as the Source of Truth

The External Secrets Operator syncs values from a store into Kubernetes Secrets:

apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
  name: payments-db
  namespace: payments
spec:
  refreshInterval: 1h
  secretStoreRef:
    kind: ClusterSecretStore
    name: aws-secrets-manager
  target:
    name: payments-db
  data:
    - secretKey: password
      remoteRef:
        key: prod/payments/db
        property: password

Alternatives: the Secrets Store CSI Driver mounts values straight from the store as files, so the value need not exist as a Kubernetes Secret; Sealed Secrets or SOPS encrypt Secrets so the encrypted form can live in Git.

4. Consume Secrets Carefully

apiVersion: v1
kind: Pod
metadata:
  name: payments-api
  namespace: payments
spec:
  serviceAccountName: payments-api
  automountServiceAccountToken: false
  containers:
    - name: api
      image: registry.example.com/payments-api:3.8.0
      volumeMounts:
        - name: db
          mountPath: /etc/secrets/db
          readOnly: true
  volumes:
    - name: db
      secret:
        secretName: payments-db
        defaultMode: 0400

Mounted files update when the Secret changes (after a short delay); environment variables are fixed at container start and are easier to leak through /proc, debug output or child processes. Mark rarely changing Secrets immutable: true to prevent accidental edits and reduce API load.

5. Rotate and Shorten Lifetimes

  • Prefer workload identity (IRSA on EKS, Workload Identity on GKE and AKS) so pods get short-lived cloud credentials instead of stored access keys.

  • Use dynamic credentials where the store supports them (for example, Vault database secrets that expire).

  • Plan rotation so the app reloads files or restarts with a rolling update.

Trade-offs

Option Benefit Cost
Native Secrets plus KMS encryption Simple, built in Values still readable through the API by anyone with RBAC access
External Secrets Operator Central store, audit, rotation Another controller; values still copied into etcd
Secrets Store CSI Driver Values not stored in etcd Pod start depends on the store being reachable
Sealed Secrets or SOPS GitOps-friendly Key management for the decryption key

Interview Tip

Lead with “base64 is not encryption”, then describe the layers: encryption at rest with KMS, RBAC, external store, and rotation. Interviewers often ask how a pod gets a cloud credential without a stored key (workload identity). A common mistake is committing a Secret manifest to Git because the values “look encoded”.

25

What is a security context, and how does it apply to pod security?

A security context is a set of Linux security settings in a pod spec that controls who a container runs as and what it may do on the host kernel: user and group IDs, whether it may run as root or gain privileges, Linux capabilities, a read-only root filesystem, and seccomp, AppArmor or SELinux profiles. It applies at two levels: spec.securityContext for the whole pod and containers[].securityContext per container, which overrides pod-level values. Hardened security contexts limit what an attacker can do after exploiting a container.

Read full answer

Applies toKubernetes 1.30+ (Pod Security Admission, `appArmorProfile` field)

Why It Matters

Containers share the node’s kernel. A process running as root with default capabilities, a writable filesystem and no syscall filter has far more room to escalate or escape after a vulnerability is exploited than a non-root process with nothing extra. The security context is how you remove that room.

Key Settings

Setting Level Effect Recommended
runAsNonRoot: true Pod or container Kubelet refuses to start a container whose user is UID 0 Yes
runAsUser, runAsGroup Pod or container Fixed UID and GID A non-zero ID, such as 10001
fsGroup Pod Group ownership applied to mounted volumes When the app writes to volumes
allowPrivilegeEscalation: false Container Blocks setuid binaries from gaining privileges Yes
capabilities.drop: ["ALL"] Container Removes Linux capabilities such as NET_RAW and CHOWN Yes; add back only what is needed
readOnlyRootFilesystem: true Container Image filesystem becomes read-only Yes; mount emptyDir for temp files
privileged: true Container Near-full host access Never for applications
seccompProfile.type: RuntimeDefault Pod or container Filters dangerous syscalls Yes
appArmorProfile, seLinuxOptions Pod or container Mandatory access control profiles Where the node OS supports them

Example: A Hardened Pod

apiVersion: v1
kind: Pod
metadata:
  name: api
spec:
  securityContext:
    runAsNonRoot: true
    runAsUser: 10001
    runAsGroup: 10001
    fsGroup: 10001
    seccompProfile:
      type: RuntimeDefault
  containers:
    - name: api
      image: registry.example.com/api:4.2.0
      securityContext:
        allowPrivilegeEscalation: false
        readOnlyRootFilesystem: true
        capabilities:
          drop: ["ALL"]
      ports:
        - containerPort: 8080
      volumeMounts:
        - name: tmp
          mountPath: /tmp
  volumes:
    - name: tmp
      emptyDir: {}

Port 8080 matters: binding to ports below 1024 needs NET_BIND_SERVICE, so a non-root app should listen on a high port and let the Service map port 80 to it.

kubectl exec api -- id
kubectl exec api -- touch /test     # fails: read-only file system

Enforcing It Cluster-Wide

A security context only helps if every pod has one. Pod Security Admission enforces the Pod Security Standards per namespace:

Level Allows
privileged Anything (system namespaces only)
baseline Blocks known escalations: privileged containers, host namespaces, hostPath
restricted Requires non-root, allowPrivilegeEscalation: false, dropping all capabilities, and a seccomp profile
kubectl label namespace shop pod-security.kubernetes.io/enforce=restricted
kubectl label namespace shop pod-security.kubernetes.io/warn=restricted

Start with warn or audit mode to find violations, then switch to enforce. Policy engines such as Kyverno or OPA Gatekeeper add custom rules (allowed registries, required labels) on top.

Trade-offs

  • Images must cooperate: many images assume root or write into their own filesystem. Fix the Dockerfile (USER 10001, write to /tmp or a volume) rather than weakening the context.

  • fsGroup can slow pod start on large volumes because ownership is changed recursively; fsGroupChangePolicy: OnRootMismatch avoids repeating it.

  • Some agents (CNI, CSI, monitoring) legitimately need privileges; isolate them in their own namespaces with a looser policy.

  • A security context reduces impact; it does not replace patching images, NetworkPolicies or RBAC.

Interview Tip

Interviewers look for the concrete restricted baseline: non-root, no privilege escalation, drop all capabilities, read-only root filesystem, RuntimeDefault seccomp, and for Pod Security Admission as the enforcement mechanism (PodSecurityPolicy was removed in 1.25). A common mistake is setting runAsNonRoot: true with an image whose default user is root and no runAsUser, so the pod fails to start with CreateContainerConfigError.

26

How do NetworkPolicies contribute to pod security, and how would you implement them?

NetworkPolicies are the cluster’s firewall between pods. Kubernetes allows all pod-to-pod and pod-to-internet traffic by default, so a single compromised pod can probe the database, the cloud metadata endpoint and every other service. NetworkPolicies turn that flat network into segments with explicit allowed paths, which limits lateral movement and data exfiltration. Implement them as default deny per namespace, then allow rules per real dependency, with special attention to egress.

Read full answer

Applies toKubernetes 1.29+ (`networking.k8s.io/v1`) with a CNI that enforces policies

What They Defend Against

Attack step Without policies With policies
Recon after a web pod is compromised Scan every pod IP in the cluster Only the API’s port is reachable
Steal cloud credentials from the metadata endpoint 169.254.169.254 reachable Egress to it blocked
Exfiltrate data Any internet host Only approved destinations
Move into another team’s namespace Open Blocked by namespace isolation
Reach the control plane or kubelet ports Often open Restricted to pods that need it

NetworkPolicies are one layer. They work with RBAC, Pod Security (non-root, no privileges) and, where used, service mesh mTLS for identity-based authorization.

Implementation Plan

  1. Verify enforcement with a test deny policy; some CNIs accept policies but do not enforce them.

  2. Observe real traffic with flow logs (Cilium Hubble, Calico flow logs, cloud VPC flow logs) to build the allow list from facts.

  3. Namespace isolation first: allow traffic only from the same namespace and from the ingress controller.

  4. Default deny egress, then allow DNS, in-cluster dependencies and specific external ranges.

  5. Block the metadata endpoint for application pods; use workload identity for cloud access.

  6. Tighten to per-workload rules for sensitive tiers such as databases and payment services.

  7. Keep policies with the app’s manifests, review them in pull requests and test connectivity in CI.

Example: Isolated Namespace with Controlled Egress

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: namespace-isolation
  namespace: payments
spec:
  podSelector: {}
  policyTypes: ["Ingress"]
  ingress:
    - from:
        - podSelector: {}
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: ingress-system
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: restricted-egress
  namespace: payments
spec:
  podSelector: {}
  policyTypes: ["Egress"]
  egress:
    - to:
        - podSelector: {}
    - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: kube-system
          podSelector:
            matchLabels:
              k8s-app: kube-dns
      ports:
        - protocol: UDP
          port: 53
        - protocol: TCP
          port: 53
    - to:
        - ipBlock:
            cidr: 0.0.0.0/0
            except:
              - 10.0.0.0/8
              - 169.254.169.254/32
      ports:
        - protocol: TCP
          port: 443

The first policy lets pods in payments accept traffic only from each other and from the ingress controller’s namespace. The second lets them reach each other, DNS, and HTTPS on the internet, but not the private 10.0.0.0/8 range or the metadata endpoint. Narrow the last rule to the payment provider’s published ranges if they are stable.

Testing

kubectl exec -n payments deploy/api -- curl -s -m 3 https://example.com -o /dev/null -w '%{http_code}\n'
kubectl exec -n payments deploy/api -- curl -s -m 3 http://169.254.169.254/ || echo blocked
kubectl exec -n shop deploy/web -- curl -s -m 3 http://api.payments:8080/ || echo blocked

Limits of the Core API

Limit Workaround
No deny rules, only allows Default deny plus allows; CNI-specific policies with explicit deny and priorities
No cluster-wide defaults Apply per namespace via templates or policy engines; Calico and Cilium offer cluster-wide policies
L3/L4 only, no DNS names or HTTP paths Cilium or Calico FQDN and L7 policies, or a mesh’s authorization policies
Host-network pods and node traffic CNI-specific host policies

Trade-offs

Policies add friction: every new dependency needs a change, and a forgotten rule looks like a timeout in production. Reduce the cost with flow-log-driven rules, consistent labels, and connectivity tests in CI. Start with namespace isolation and egress control, which give most of the security value, before per-pod rules everywhere.

Interview Tip

Interviewers want to hear the threat model (lateral movement, metadata credential theft, exfiltration) and default deny with explicit egress, not just ingress rules. A common mistake is forgetting DNS after applying default-deny egress, which breaks every service at once; another is assuming policies work without checking the CNI.

27

Describe the role and capabilities of the Metrics Server in a Kubernetes cluster.

The Metrics Server is a lightweight cluster add-on that collects current CPU and memory usage for every node and pod from the kubelets and serves it through the Resource Metrics API (metrics.k8s.io). It exists to feed autoscaling and kubectl top: the Horizontal Pod Autoscaler reads it to scale on CPU and memory, and the Vertical Pod Autoscaler uses it for recommendations. It keeps only the latest values in memory, so it is not a monitoring system; history, dashboards and alerts need Prometheus or a similar tool.

Read full answer

Applies toKubernetes 1.29+ with metrics-server 0.7+

How It Works

  1. Every kubelet exposes resource usage on its /metrics/resource endpoint, measured from the container runtime and cgroups.

  2. Metrics Server scrapes all kubelets on an interval (--metric-resolution, 15 seconds by default).

  3. It stores the latest sample per node and pod in memory.

  4. It registers an APIService so the API server proxies requests for metrics.k8s.io to it (API aggregation).

  5. Clients call the normal API: kubectl top, the HPA controller, the VPA recommender.

Flow: kubelet → Metrics Server → API server (aggregation layer) → HPA / kubectl top.

kubectl top nodes
kubectl top pods -n shop --sort-by=cpu
kubectl top pod checkout-7d9f-x2k4p --containers
kubectl get apiservice v1beta1.metrics.k8s.io
kubectl get --raw /apis/metrics.k8s.io/v1beta1/namespaces/shop/pods

What It Provides and What It Does Not

Provides Does not provide
Current CPU (cores) and memory (working set) per node, pod and container History or trends
The data source for HPA on Resource metrics Custom or application metrics (requests per second, queue length)
kubectl top Network, disk or GPU metrics
A stable API (metrics.k8s.io) Alerting or dashboards

The Three Metrics APIs

API Served by Used for
metrics.k8s.io (resource metrics) Metrics Server HPA on CPU and memory, kubectl top
custom.metrics.k8s.io Prometheus Adapter, KEDA, vendor adapters HPA on per-object metrics such as requests per second per pod
external.metrics.k8s.io KEDA, cloud adapters HPA on metrics outside the cluster, such as an SQS queue length

HPA Dependency

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: api
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api
  minReplicas: 2
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70

If Metrics Server is down or not installed, this HPA shows <unknown> targets and stops scaling. Utilization is computed against the pods’ CPU requests.

Installation and Operations

  • Many managed clusters install it by default; otherwise apply the project’s release manifest or Helm chart.

  • It needs to reach every kubelet on port 10250 and trust the kubelet serving certificates. The --kubelet-insecure-tls flag seen in tutorials skips that verification and is for labs only; fix kubelet certificate signing instead.

  • Its resource needs grow with the number of pods; the project documents scaling guidance, and running two replicas with a PodDisruptionBudget avoids autoscaling gaps during node maintenance.

  • Values are recent averages over a short window, so short CPU spikes can be invisible to kubectl top.

Trade-offs

Choice Benefit Cost
Metrics Server only Tiny footprint, enough for CPU/memory HPA No history, no custom metrics
Prometheus + Prometheus Adapter History, alerting, custom-metric autoscaling More components to run and tune
KEDA Event-driven scaling, scale to zero Another controller; still uses Metrics Server for CPU and memory triggers

Interview Tip

The distinction interviewers look for: Metrics Server is the autoscaling pipeline, Prometheus is the monitoring system. Be ready to explain why kubectl top fails on a fresh kubeadm cluster (not installed, or kubelet TLS problems) and why an HPA shows <unknown> (no metrics or no CPU requests). A common mistake is pointing dashboards or capacity planning at Metrics Server data, which has no history.

28

How can Prometheus be configured to monitor Kubernetes components?

The usual setup is the Prometheus Operator, installed with the kube-prometheus-stack Helm chart. It deploys Prometheus, Alertmanager, Grafana, node-exporter (node metrics) and kube-state-metrics (object state), and configures scraping of the kubelet and cAdvisor, API server, CoreDNS and, where reachable, etcd, scheduler and controller manager. Prometheus discovers targets through the Kubernetes API (kubernetes_sd_configs), and you describe what to scrape with ServiceMonitor and PodMonitor resources instead of editing prometheus.yml by hand.

Read full answer

Applies toKubernetes 1.29+ with Prometheus Operator (kube-prometheus-stack)

What to Scrape

Component Endpoint What you learn
kubelet :10250/metrics Pod start latency, volume operations, kubelet health
cAdvisor (inside kubelet) :10250/metrics/cadvisor Container CPU, memory, network, throttling
node-exporter (DaemonSet) :9100/metrics Node CPU, memory, disk, filesystem, network
kube-state-metrics :8080/metrics Desired vs actual: replicas, pod phase, restarts, Job status, PVC status
kube-apiserver :6443/metrics Request rate, latency, errors per verb and resource
etcd :2381/metrics (kubeadm) Leader changes, fsync and commit latency, DB size
kube-scheduler, controller-manager :10259, :10257 Scheduling latency, work-queue depth
CoreDNS :9153/metrics Query rate, errors, latency

On kubeadm clusters, the scheduler, controller manager and etcd metrics listen on 127.0.0.1 by default, so they show as down until you change their bind addresses or scrape them from the node. Managed services usually hide these components; scrape what the provider exposes.

Install

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm install monitoring prometheus-community/kube-prometheus-stack \
  --namespace monitoring --create-namespace \
  --set prometheus.prometheusSpec.retention=15d
kubectl get servicemonitors -n monitoring
kubectl port-forward -n monitoring svc/monitoring-kube-prometheus-prometheus 9090

Scraping Your Own Services

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: checkout
  namespace: shop
  labels:
    release: monitoring
spec:
  selector:
    matchLabels:
      app: checkout
  endpoints:
    - port: metrics
      interval: 30s
      path: /metrics

The Operator turns this into scrape configuration for every endpoint of Services labelled app: checkout with a port named metrics. The release: monitoring label must match the Prometheus resource’s serviceMonitorSelector, which is the most common reason a ServiceMonitor is silently ignored.

Alerts

apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: workload-alerts
  namespace: monitoring
  labels:
    release: monitoring
spec:
  groups:
    - name: workloads
      rules:
        - alert: PodCrashLooping
          expr: increase(kube_pod_container_status_restarts_total[15m]) > 3
          for: 5m
          labels:
            severity: warning
          annotations:
            summary: "{{ $labels.namespace }}/{{ $labels.pod }} is restarting repeatedly"
        - alert: CPUThrottlingHigh
          expr: |
            sum by (namespace, pod) (rate(container_cpu_cfs_throttled_periods_total[5m]))
              / sum by (namespace, pod) (rate(container_cpu_cfs_periods_total[5m])) > 0.25
          for: 15m
          labels:
            severity: info

The chart also ships a tested set of rules and Grafana dashboards (the kubernetes-mixin) for the API server, nodes, workloads and etcd; start from those and remove what you do not need.

Production Considerations

Concern Approach
Storage Persistent volume for Prometheus; retention by time and size
High availability Two Prometheus replicas scraping the same targets; Alertmanager cluster of three
Long-term or multi-cluster storage Remote write to Thanos, Grafana Mimir, Cortex or a managed service
Cardinality Avoid labels with unbounded values (user IDs, request paths); drop unused series with relabelling
Resource usage Memory grows with active series; size it from prometheus_tsdb_head_series
Security Scrape over TLS where supported; RBAC for Prometheus’s ServiceAccount to list endpoints and pods

Trade-offs

  • The Operator adds CRDs and a controller, but removes hand-written scrape configs and makes monitoring part of each app’s manifests.

  • Scraping every 15–30 seconds is a good default; shorter intervals multiply storage and CPU.

  • Prometheus is pull-based and single-node per replica; very large estates need sharding or a horizontally scalable backend.

Interview Tip

Interviewers look for the split between node-exporter (machine), cAdvisor (containers), kube-state-metrics (Kubernetes object state) and control plane metrics, plus ServiceMonitor as the configuration mechanism. A common mistake is confusing kube-state-metrics with Metrics Server: one exposes object state for monitoring, the other serves current CPU and memory to autoscalers. Expect a follow-up on high cardinality.

29

Discuss the integration of Grafana for Kubernetes observability.

Grafana is the visualisation and alerting layer on top of the data sources that actually collect Kubernetes telemetry: Prometheus (or Mimir/Thanos) for metrics, Loki (or Elasticsearch) for logs and Tempo (or Jaeger) for traces. In a cluster it usually runs as part of the kube-prometheus-stack Helm chart, with data sources and dashboards provisioned as code from ConfigMaps, so dashboards for nodes, workloads, the API server and your own services appear automatically and live in Git.

Read full answer

Applies toKubernetes 1.29+ with Grafana 10+ and Prometheus (kube-prometheus-stack)

Where Grafana Fits

Flow: kubelet, cAdvisor, node-exporter, kube-state-metrics and app /metrics → Prometheus (scrape and store) → Grafana (query with PromQL, show, alert). Logs: container stdout → agent (Promtail/Alloy, Fluent Bit) → Loki → Grafana. Traces: app with OpenTelemetry → collector → Tempo → Grafana.

Grafana stores no metrics itself; it queries the sources at view time. Its value is correlation: jump from a latency spike on a dashboard to the matching logs and traces for the same pod and time range.

Setting It Up

helm install monitoring prometheus-community/kube-prometheus-stack \
  --namespace monitoring --create-namespace \
  --set grafana.adminPassword=change-me \
  --set grafana.sidecar.dashboards.enabled=true
kubectl port-forward -n monitoring svc/monitoring-grafana 3000:80

The chart preconfigures the Prometheus data source and ships dashboards for cluster capacity, nodes, namespaces, pods, the API server, etcd and CoreDNS. Expose Grafana through an Ingress or Gateway with SSO rather than a shared admin password.

Dashboards as Code

The Grafana sidecar watches ConfigMaps with a label and loads their JSON as dashboards:

apiVersion: v1
kind: ConfigMap
metadata:
  name: checkout-dashboard
  namespace: monitoring
  labels:
    grafana_dashboard: "1"
data:
  checkout.json: |
    {
      "title": "Checkout service",
      "templating": {"list": [{"name": "namespace", "type": "query",
        "query": "label_values(kube_pod_info, namespace)"}]},
      "panels": [
        {"type": "timeseries", "title": "Requests per second",
         "targets": [{"expr": "sum(rate(http_requests_total{namespace=\"$namespace\", app=\"checkout\"}[5m]))"}]},
        {"type": "timeseries", "title": "p95 latency",
         "targets": [{"expr": "histogram_quantile(0.95, sum by (le) (rate(http_request_duration_seconds_bucket{app=\"checkout\"}[5m])))"}]}
      ]
    }

Keep dashboard JSON in Git next to the service and deploy it with the app’s Helm chart, so dashboards are reviewed and never lost when Grafana is rebuilt.

Useful Kubernetes Panels

Question PromQL
CPU used vs requested per namespace sum by (namespace) (rate(container_cpu_usage_seconds_total{container!=""}[5m])) vs sum by (namespace) (kube_pod_container_resource_requests{resource="cpu"})
Pods not ready sum by (namespace) (kube_pod_status_ready{condition="false"})
Container restarts increase(kube_pod_container_status_restarts_total[1h])
Memory close to limit container_memory_working_set_bytes / on (namespace, pod, container) kube_pod_container_resource_limits{resource="memory"}
API server errors sum(rate(apiserver_request_total{code=~"5.."}[5m]))

Template variables (cluster, namespace, workload, pod) let one dashboard serve every team instead of copying it per service.

Alerting

Grafana can evaluate alert rules and route notifications (Slack, PagerDuty, email). Many teams keep alert rules in Prometheus/Alertmanager (PrometheusRule CRDs, versioned with the app) and use Grafana for dashboards only, which avoids two alerting systems. Pick one place for alert rules and stick to it.

Design Practices

  • Start with RED (rate, errors, duration) for services and USE (utilisation, saturation, errors) for nodes.

  • One overview dashboard per service with links to detailed ones; avoid 40-panel walls.

  • Use folders and permissions per team; connect Grafana to your identity provider.

  • Watch query cost: wide time ranges over high-cardinality series slow Prometheus for everyone. Use recording rules for expensive queries.

Trade-offs

Option Benefit Cost
Self-hosted Grafana in the cluster Free, full control You run upgrades, storage for its database, SSO and backups
Managed Grafana (cloud or vendor) No operations; scales across clusters Ongoing cost; data egress
Dashboards edited in the UI only Fast to start Drift and loss; nobody reviews changes

Interview Tip

Make clear that Grafana visualises, while Prometheus, Loki and Tempo collect and store. Interviewers like hearing about dashboards as code, template variables, and correlating metrics with logs and traces during an incident. A common mistake is building dashboards without alerts, or with alerts on causes (CPU 80%) instead of symptoms users feel (error rate, latency).

30

How can you use Node Selectors and Affinity rules to influence pod placement?

nodeSelector is the simplest placement control: the pod runs only on nodes that have all the listed labels. Node affinity is the expressive version: required rules (hard constraints, with operators such as In, NotIn, Exists, Gt) and preferred rules (weighted hints). Pod affinity and anti-affinity place pods relative to other pods: together (same node or zone) or apart. All of these act only at scheduling time; running pods are not moved if labels change later.

Read full answer

Applies toKubernetes 1.29+

The Tools

Mechanism Attracts or repels Hard or soft Typical use
nodeSelector Attracts to nodes Hard “Run on pool=gpu nodes”
Node affinity, required Attracts to nodes Hard Zone, architecture, instance family, with set logic
Node affinity, preferred Attracts to nodes Soft (weight 1–100) “Prefer spot nodes, fall back to on-demand”
Pod affinity Attracts to other pods Hard or soft Co-locate a cache with its app
Pod anti-affinity Repels from other pods Hard or soft Spread replicas across nodes or zones
Taints and tolerations Nodes repel pods Hard (or PreferNoSchedule) Reserve nodes for special workloads
Topology spread constraints Balances pods Hard or soft Even spread across zones and nodes

Affinity attracts pods to nodes; taints repel pods that do not tolerate them. For dedicated nodes you usually need both: a taint to keep others off, plus a selector or affinity to pull the right pods on.

nodeSelector

apiVersion: v1
kind: Pod
metadata:
  name: trainer
spec:
  nodeSelector:
    pool: gpu
    kubernetes.io/arch: amd64
  containers:
    - name: trainer
      image: registry.example.com/trainer:1.0.0

Node Affinity and Pod Anti-Affinity Together

apiVersion: apps/v1
kind: Deployment
metadata:
  name: api
spec:
  replicas: 3
  selector:
    matchLabels:
      app: api
  template:
    metadata:
      labels:
        app: api
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                  - key: topology.kubernetes.io/zone
                    operator: In
                    values: ["eu-west-1a", "eu-west-1b", "eu-west-1c"]
          preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 80
              preference:
                matchExpressions:
                  - key: node.kubernetes.io/lifecycle
                    operator: In
                    values: ["spot"]
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchLabels:
                  app: api
              topologyKey: kubernetes.io/hostname
      containers:
        - name: api
          image: registry.example.com/api:4.2.0
  • The required node rule limits pods to three zones.

  • The preferred rule favours spot nodes but allows others. The spot label key differs by provider and tool; use whatever your node groups actually set.

  • The pod anti-affinity rule keeps replicas on different nodes (topologyKey: kubernetes.io/hostname). With topology.kubernetes.io/zone it would mean one replica per zone.

Several nodeSelectorTerms are ORed; expressions inside one term are ANDed.

Pod Affinity for Co-location

affinity:
  podAffinity:
    preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 100
        podAffinityTerm:
          labelSelector:
            matchLabels:
              app: redis-cache
          topologyKey: topology.kubernetes.io/zone

This prefers the zone where the cache runs, cutting cross-zone latency and transfer cost.

Choosing Hard or Soft

Situation Choose
The pod cannot work elsewhere (GPU, ARM image, licence) Required
Cost or latency optimisation Preferred
Spreading for availability with more replicas than nodes Preferred anti-affinity or topology spread
Spreading where losing two replicas on one node is unacceptable Required anti-affinity, with enough nodes

Hard rules that cannot be satisfied leave pods Pending. kubectl describe pod shows messages like 0/6 nodes are available: 3 node(s) didn't match pod anti-affinity rules.

Trade-offs

  • Required pod anti-affinity with hostname caps replicas at the node count; scaling to 10 on 6 nodes leaves 4 pending.

  • Inter-pod affinity is expensive for the scheduler in large clusters; topology spread constraints are the lighter, more predictable way to spread.

  • IgnoredDuringExecution means changing node labels does not evict pods; rebalancing needs a rollout or the descheduler.

  • Over-constrained placement fights the cluster autoscaler, which may add nodes that still do not match.

Interview Tip

Interviewers want the attract versus repel distinction (affinity versus taints) and the required versus preferred choice with its failure mode (Pending pods). A common mistake is using required anti-affinity for spreading in a small cluster, so a scale-up silently gets stuck; topology spread constraints with ScheduleAnyway are the safer default.

31

Explain Horizontal Pod Autoscaling (HPA) and its triggers within Kubernetes.

The Horizontal Pod Autoscaler (HPA) changes the number of replicas of a Deployment, StatefulSet or other scalable resource so that a chosen metric stays near a target. Every 15 seconds the HPA controller reads the metrics, computes desired = ceil(currentReplicas × currentValue / targetValue), clamps it between minReplicas and maxReplicas, applies the scaling behaviour rules, and updates the target’s scale. Triggers can be resource metrics (CPU, memory), per-pod custom metrics, object metrics, or external metrics from outside the cluster.

Read full answer

Applies toKubernetes 1.29+ (`autoscaling/v2`)

Metric Types (Triggers)

Type Source API Example Target types
Resource metrics.k8s.io (Metrics Server) CPU at 70% of requests Utilization, AverageValue
ContainerResource metrics.k8s.io CPU of the app container only, ignoring sidecars Utilization, AverageValue
Pods custom.metrics.k8s.io http_requests_per_second averaged per pod AverageValue
Object custom.metrics.k8s.io Requests per second on an Ingress Value, AverageValue
External external.metrics.k8s.io Message queue depth in a cloud service Value, AverageValue

Custom and external metrics need an adapter, such as Prometheus Adapter, KEDA or a cloud provider adapter.

Example: CPU Plus Requests per Second

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: search-api
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: search-api
  minReplicas: 3
  maxReplicas: 40
  metrics:
    - type: ContainerResource
      containerResource:
        name: cpu
        container: api
        target:
          type: Utilization
          averageUtilization: 60
    - type: Pods
      pods:
        metric:
          name: http_requests_per_second
        target:
          type: AverageValue
          averageValue: "100"
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
        - type: Percent
          value: 20
          periodSeconds: 60

With several metrics, the HPA computes a replica count for each and uses the largest, so whichever metric is under most pressure wins.

The Algorithm in Detail

  1. Fetch the metric for every pod of the target.

  2. Ignore pods that are being deleted or have failed. Pods with missing metrics are treated conservatively: as 0% of the target when scaling up and 100% when scaling down. For CPU, pods that are not ready or have only just started are set aside (--horizontal-pod-autoscaler-cpu-initialization-period, 5 minutes by default), so start-up spikes do not cause runaway scaling.

  3. Compute the ratio currentValue / targetValue. If it is within the tolerance (10% by default), do nothing.

  4. Otherwise compute desired. Example: 5 pods averaging 150 requests per second against 100 gives ceil(5 × 150 / 100) = 8.

  5. Apply behavior: scale-up is immediate by default (stabilisation window 0); scale-down uses the highest recommendation of the last 5 minutes, which prevents flapping.

  6. Update spec.replicas through the target’s scale subresource.

kubectl get hpa search-api
kubectl describe hpa search-api      # conditions: AbleToScale, ScalingActive, ScalingLimited
kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta1/namespaces/default/pods/*/http_requests_per_second"

Prerequisites

  • Resource requests on the containers for Utilization targets; utilization is a percentage of the request.

  • Metrics Server for CPU and memory; an adapter for custom or external metrics.

  • Remove replicas from the Deployment manifest once the HPA owns it, or every apply fights the HPA.

  • Node capacity: a cluster autoscaler or Karpenter, or new pods wait as Pending.

Choosing Triggers

Workload Good trigger Why
CPU-bound API CPU utilisation Direct and built in
I/O-bound API Requests per second per pod, or concurrency CPU stays low while latency rises
Queue worker Queue depth or lag (External) Measures backlog, not effort
Memory-heavy cache Usually not HPA Memory rarely drops after scaling, so it never scales down

Limits and Trade-offs

  • HPA cannot scale to zero by default (minReplicas ≥ 1); KEDA adds scale-to-zero for event-driven workloads.

  • Reaction time is the sum of metric delay, the sync period, pod start-up and, if nodes are needed, node provisioning. For sharp, predictable peaks, raise minReplicas beforehand.

  • HPA and VPA on the same CPU or memory metric fight each other.

  • Scaling a stateless tier can overload a shared database; set maxReplicas from downstream capacity, not just budget.

Interview Tip

Interviewers expect the formula, the dependence on requests, the multiple metrics → maximum rule and the scale-down stabilisation window. A common mistake is scaling an I/O-bound service on CPU and wondering why it never scales while latency climbs; per-pod request rate or queue depth is the better trigger.

32

What steps would you take to troubleshoot a pod that is not starting?

Start from the pod’s status and events, because they tell you which stage failed: scheduling (Pending), image pull (ImagePullBackOff), container configuration (CreateContainerConfigError), volume or network setup (stuck in ContainerCreating), or the application itself (CrashLoopBackOff, OOMKilled). kubectl describe pod and kubectl logs --previous answer most cases within a few minutes; only then go deeper with kubectl debug or node-level checks.

Read full answer

Applies toKubernetes 1.29+

Step 1: Read the Status and Events

kubectl get pod checkout-7d9f-x2k4p -n shop -o wide
kubectl describe pod checkout-7d9f-x2k4p -n shop        # Events at the bottom
kubectl get events -n shop --sort-by=.lastTimestamp | tail -20

Step 2: Match the Symptom

Status Stage Common causes Check
Pending, no node Scheduling Not enough CPU or memory requested capacity, taints without tolerations, node selector or affinity mismatch, unbound PVC, quota exceeded Events: 0/5 nodes are available: ...; kubectl describe node; kubectl get pvc
ErrImagePull, ImagePullBackOff Image Wrong name or tag, private registry without imagePullSecrets, registry rate limit, architecture mismatch Events show the registry error; try the exact image reference
CreateContainerConfigError Config Referenced ConfigMap, Secret or key missing; runAsNonRoot with a root image Events name the missing object
ContainerCreating for minutes Sandbox, volumes, network Volume attach or mount failure (zonal disk, RWO already attached elsewhere), CNI failure, Secret volume missing Events: FailedMount, FailedAttachVolume, FailedCreatePodSandBox
Init:CrashLoopBackOff, Init:0/2 Init containers Init container failing or waiting forever (dependency down) kubectl logs <pod> -c <init-container>
CrashLoopBackOff Application App exits: bad config, missing env var, cannot reach a dependency, wrong command kubectl logs <pod> --previous
OOMKilled (exit code 137) Application Memory limit too low or a leak kubectl describe pod → Last State; raise the limit or fix usage
Running but not Ready Readiness Probe path or port wrong, app slow to start Probe failures in events; add a startup probe

Exit codes help: 1 is a generic app error, 127 command not found, 126 not executable, 137 killed by SIGKILL (often OOM), 143 SIGTERM.

Step 3: Look Inside

kubectl logs checkout-7d9f-x2k4p -n shop --previous
kubectl logs checkout-7d9f-x2k4p -n shop -c init-db
kubectl get pod checkout-7d9f-x2k4p -n shop -o yaml | grep -A10 containerStatuses
kubectl debug -it checkout-7d9f-x2k4p -n shop --image=busybox:1.36 --target=app
kubectl debug checkout-7d9f-x2k4p -n shop -it --copy-to=checkout-debug --container=app -- sh
  • --previous shows logs of the crashed attempt; the current attempt may have no output yet.

  • An ephemeral container (kubectl debug --target) shares the process namespace, so you can inspect a running but broken container whose image has no shell.

  • --copy-to creates a copy of the pod with a different command, useful when the app exits too fast to inspect.

Step 4: Check the Workload and Its Dependencies

kubectl rollout status deployment/checkout -n shop
kubectl get rs -n shop -l app=checkout
kubectl get resourcequota -n shop
kubectl describe pvc data-checkout -n shop
  • A Deployment with no pods at all often means a ResourceQuota or admission policy rejected them; the reason is in the ReplicaSet’s events, not on a pod.

  • Compare the failing version with the last working one: kubectl rollout history and the diff of the pod template.

Step 5: Node-Level Checks (If Several Pods on One Node Fail)

  • kubectl describe node for DiskPressure, MemoryPressure, PIDPressure or NotReady.

  • On the node: journalctl -u kubelet, crictl ps -a, crictl logs <id> for runtime and CNI errors.

Example Walkthrough

A new release goes into CrashLoopBackOff. describe shows exit code 1 and restarts. logs --previous prints missing required env DATABASE_URL. The Deployment references a Secret key renamed in the last change. Fix the key, kubectl rollout restart, and add a check in CI that rendered manifests reference existing keys.

Habits That Prevent It

  • Set requests and limits from measured usage.

  • Use immutable image tags and test pulls from the cluster.

  • Startup probes for slow starters, so liveness does not kill them.

  • Fail fast with clear log lines when configuration is missing.

Interview Tip

Interviewers want a systematic order: status → events → logs (--previous) → describe the dependencies → node. Naming the meaning of Pending versus CrashLoopBackOff versus ImagePullBackOff shows you know which component failed (scheduler, kubelet and registry, or the app). A common mistake is reading only current logs of a crash-looping pod and finding them empty.

33

How would you diagnose network issues in Kubernetes?

Diagnose network problems layer by layer along the request path, testing from inside the cluster: DNS (does the name resolve?), Service (does it have ready endpoints, and does the port match?), pod-to-pod connectivity (does the pod IP answer directly?), NetworkPolicy (is the traffic being denied?), node and CNI (is the problem limited to one node?), and finally ingress or egress at the cluster edge. Each step either finds the fault or rules out a layer.

Read full answer

Applies toKubernetes 1.29+

Get a Debugging Shell in the Right Place

kubectl run netdebug -n shop --rm -it --image=nicolaka/netshoot -- bash
kubectl debug -it -n shop web-5f7c9-abcde --image=nicolaka/netshoot --target=web

The second form runs the tools in the same network namespace as the failing pod, so you see exactly what it sees, including NetworkPolicy effects.

Layer by Layer

Step Test Healthy result If it fails
1. DNS nslookup orders.shop.svc.cluster.local ClusterIP returned Check CoreDNS pods, logs, kube-dns Service; egress policy allowing port 53
2. Service endpoints kubectl get endpointslices -n shop -l kubernetes.io/service-name=orders Ready pod IPs listed Selector does not match labels, or pods are not ready
3. Service port curl -v http://orders.shop:80/healthz HTTP response targetPort does not match the container port; app listens on 127.0.0.1 instead of 0.0.0.0
4. Pod IP directly curl -v http://10.244.3.17:8080/healthz HTTP response App problem if local; CNI problem if only cross-node fails
5. NetworkPolicy kubectl get networkpolicy -n shop and policy flow logs Allowed path exists Missing allow rule; DNS egress forgotten
6. Node scope Repeat from a pod on another node Same result everywhere One node’s CNI agent or kube-proxy is broken
7. Edge Ingress controller logs, cloud LB health checks, egress NAT Traffic flows Ingress rules, TLS, security groups, NAT gateway

Reading the Error

Symptom Usual meaning
Name or service not known, NXDOMAIN Wrong name or namespace, or DNS down
Connection refused Reached the IP, nothing listening on that port (wrong targetPort, app not up)
Timeout Packets dropped: NetworkPolicy, security group, CNI routing, or no endpoints with some data planes
Works with small requests, hangs on large MTU mismatch, common with overlays and VPNs
Intermittent failures, about 1 in N One bad endpoint (a pod passing readiness but broken) or one bad node
Slow DNS (seconds) ndots:5 lookups for external names, conntrack races, overloaded CoreDNS

Useful Commands

kubectl get pods -n shop -o wide                     # pod IPs and nodes
kubectl describe svc orders -n shop
kubectl logs -n kube-system -l k8s-app=kube-dns --tail=50
kubectl get pods -n kube-system -o wide | grep -Ei 'cni|calico|cilium|kube-proxy'
# Inside a netshoot shell:
dig +search orders
curl -sv --connect-timeout 3 http://orders.shop/healthz
tcpdump -i any -nn port 8080
mtr -n 10.244.3.17

With Cilium, hubble observe --namespace shop --verdict DROPPED shows exactly which policy dropped which flow; Calico offers flow logs with policy verdicts.

Example Walkthrough

web reports timeouts calling orders.

  1. nslookup orders from the web pod works, so DNS is fine.

  2. EndpointSlice lists three ready pods, so the selector is fine.

  3. curl orders:80 times out; curl to a pod IP on the same node works, on another node times out.

  4. Every cross-node failure involves node worker-4. Its CNI agent pod is in CrashLoopBackOff after a kernel update.

  5. Cordon and drain the node, fix the agent, uncordon.

Prevent and Detect

  • Alert on CoreDNS errors and latency, CNI agent readiness, and kube-proxy sync errors.

  • Run a synthetic in-cluster check that calls key Services from every node.

  • Keep NetworkPolicies with the app manifests and test them in CI.

  • Document the pod, Service and node CIDRs and the MTU.

Interview Tip

Interviewers look for a method, not a list of tools: follow the path DNS → Service → endpoints → pod → policy → node → edge, and use the error type (refused versus timeout) to narrow it down. A common mistake is debugging from your laptop through kubectl port-forward, which bypasses Services, kube-proxy and NetworkPolicies, so it can “work” while real traffic fails.

34

Define Custom Resource Definitions (CRDs) and their utility in Kubernetes.

A Custom Resource Definition (CRD) adds a new object type to the Kubernetes API, such as Certificate, KafkaTopic or Database. Once registered, you create, list, watch and RBAC-control those objects with kubectl exactly like built-in ones, and they are stored in etcd. A CRD only defines data; the behaviour comes from a controller (an operator when it encodes operational knowledge) that watches the custom objects and reconciles real systems to match them.

Read full answer

Applies toKubernetes 1.29+ (`apiextensions.k8s.io/v1`)

Why They Are Useful

  • Extend the declarative model to anything: certificates, DNS records, cloud databases, Kafka topics, backup schedules.

  • One API and toolchain: the same RBAC, audit logs, kubectl, GitOps and policy engines work for custom objects.

  • Encode operations: an operator can automate failover, upgrades and backups of a database the way an experienced operator would.

  • Platform abstractions: a platform team can offer a simple WebApp object that expands into a Deployment, Service, HPA and Ingress.

Many common tools are built this way: cert-manager (Certificate), Prometheus Operator (ServiceMonitor), Argo CD (Application), the Gateway API (HTTPRoute), External Secrets (ExternalSecret).

Example: Defining a CRD

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: databases.platform.example.com
spec:
  group: platform.example.com
  scope: Namespaced
  names:
    kind: Database
    plural: databases
    singular: database
    shortNames: ["db"]
  versions:
    - name: v1
      served: true
      storage: true
      subresources:
        status: {}
      additionalPrinterColumns:
        - name: Engine
          type: string
          jsonPath: .spec.engine
        - name: Phase
          type: string
          jsonPath: .status.phase
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              required: ["engine", "storageGi"]
              properties:
                engine:
                  type: string
                  enum: ["postgres", "mysql"]
                storageGi:
                  type: integer
                  minimum: 10
                  maximum: 1000
                replicas:
                  type: integer
                  default: 1
              x-kubernetes-validations:
                - rule: "self.engine != 'mysql' || self.replicas <= 3"
                  message: "mysql supports at most 3 replicas here"
            status:
              type: object
              properties:
                phase:
                  type: string

The name must be <plural>.<group>. The OpenAPI schema validates input at the API server, default fills missing fields, and CEL rules (x-kubernetes-validations) express cross-field checks without a webhook.

Using It

apiVersion: platform.example.com/v1
kind: Database
metadata:
  name: orders
  namespace: shop
spec:
  engine: postgres
  storageGi: 50
  replicas: 2
kubectl apply -f crd.yaml -f orders-db.yaml
kubectl get db -n shop
kubectl explain database.spec

The Controller Side

A controller, usually written with controller-runtime (Kubebuilder or Operator SDK), runs a reconcile loop:

  1. Watch Database objects and the resources it creates.

  2. On change, read the desired spec, observe the actual state (a StatefulSet, a cloud database), and act to converge.

  3. Write progress to .status through the status subresource, so users and tools see phase: Ready.

  4. Use finalizers to clean up external resources before the object is deleted.

  5. Set ownerReferences so child objects are garbage-collected with their parent.

Without a controller, a custom object is just stored data that nothing acts on.

Versioning and Evolution

  • Several versions can be served; exactly one is the storage version.

  • Changing schemas between versions needs a conversion webhook unless the versions are structurally identical.

  • Follow Kubernetes conventions: spec for desired state, status for observed state, never change meaning of a field within a version.

Trade-offs

Benefit Cost
Uniform API, RBAC and GitOps for custom concepts You own a controller: code, tests, upgrades, on-call
Validation and defaults at the API server Schema mistakes are hard to fix once objects exist
Operators automate complex operations A buggy operator can break many workloads at once
Discoverable with kubectl explain Too many CRDs add load on the API server and etcd

Do not create a CRD for plain configuration that a ConfigMap or Helm values handle, and prefer a mature existing operator over writing your own for common systems.

Interview Tip

Interviewers want you to separate the CRD (schema and storage) from the controller (behaviour) and to connect it to the reconcile-loop model. Mention the status subresource, finalizers and CEL validation to show depth. A common mistake is saying a CRD “runs” something; only a controller does.

35

Discuss the functionalities provided by a Service Mesh like Istio in Kubernetes.

A service mesh moves networking concerns out of application code into a dedicated infrastructure layer of proxies. In Istio, an Envoy proxy handles traffic for each workload (as a sidecar per pod, or with node-level and waypoint proxies in ambient mode), and the istiod control plane programs those proxies. That gives every service mutual TLS and identity, fine-grained traffic routing (canaries, retries, timeouts, circuit breaking) and uniform telemetry without changing the services themselves. The cost is extra latency, resource use and operational complexity.

Read full answer

Applies toKubernetes 1.29+ with Istio 1.22+ (`networking.istio.io/v1`, `security.istio.io/v1`)

Functional Areas

Area What the mesh provides Istio resources
Security Automatic mTLS between workloads, certificate rotation, workload identity (SPIFFE), L7 authorization PeerAuthentication, AuthorizationPolicy
Traffic management Weighted routing, header-based routing, mirroring, fault injection VirtualService, DestinationRule, Gateway API routes
Resilience Timeouts, retries with limits, circuit breaking via outlier detection, connection pools VirtualService, DestinationRule
Observability Request metrics (rate, errors, latency), access logs, distributed trace spans for every hop Telemetry API; Prometheus, Grafana, Kiali, Jaeger/Tempo
Ingress and egress Managed entry and exit points with policy Gateways, ServiceEntry

Request Flow (Sidecar Mode)

Service A container → local Envoy (mTLS, retries, metrics) → network → Service B’s Envoy (verifies identity, applies authorization) → Service B container.

Example: Strict mTLS, a Canary and Resilience

apiVersion: security.istio.io/v1
kind: PeerAuthentication
metadata:
  name: default
  namespace: shop
spec:
  mtls:
    mode: STRICT
---
apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: checkout
  namespace: shop
spec:
  host: checkout
  subsets:
    - name: stable
      labels:
        version: v1
    - name: canary
      labels:
        version: v2
  trafficPolicy:
    outlierDetection:
      consecutive5xxErrors: 5
      interval: 10s
      baseEjectionTime: 30s
---
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: checkout
  namespace: shop
spec:
  hosts: ["checkout"]
  http:
    - timeout: 2s
      retries:
        attempts: 2
        perTryTimeout: 800ms
        retryOn: 5xx,connect-failure
      route:
        - destination:
            host: checkout
            subset: stable
          weight: 90
        - destination:
            host: checkout
            subset: canary
          weight: 10
---
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: checkout-callers
  namespace: shop
spec:
  selector:
    matchLabels:
      app: checkout
  action: ALLOW
  rules:
    - from:
        - source:
            principals: ["cluster.local/ns/shop/sa/web"]
      to:
        - operation:
            methods: ["POST"]
            paths: ["/api/checkout*"]

Ten percent of requests go to v2 regardless of replica counts, which a plain Kubernetes rolling update cannot do. Only the web ServiceAccount may call POST /api/checkout*. Retries must be limited and used only for idempotent operations, or a struggling service receives up to three times the load.

istioctl install --set profile=default
kubectl label namespace shop istio-injection=enabled
kubectl rollout restart deployment -n shop
istioctl analyze -n shop
istioctl proxy-config routes deploy/web -n shop

Sidecar versus Ambient Mode

Aspect Sidecar Ambient
Proxy placement Envoy in every pod Per-node ztunnel for L4 and mTLS; optional waypoint proxies for L7
Resource use Proxy CPU and memory in every replica Lower; L7 cost only where needed
Rollout Pods restart to get the sidecar Label the namespace, no pod restart
Maturity Long-established Stable since Istio 1.24, newer in production

Costs and When Not to Use a Mesh

Cost Detail
Latency An extra proxy hop on each side of every call; the added time depends on payload, policies and hardware, so measure p50 and p99 before and after
Resources Proxies add CPU and memory across all replicas
Complexity New CRDs, upgrades, certificate authority, debugging through proxies
Failure modes A misconfigured policy can block traffic mesh-wide

Skip a mesh for a few services where TLS at the ingress, NetworkPolicies and a client library for retries are enough. Consider it when you have many services, need zero-trust mTLS for compliance, or want consistent canaries and telemetry across teams. Linkerd is a lighter alternative; Cilium offers some mesh features in its eBPF data plane.

Interview Tip

Interviewers want the data plane versus control plane split, the three pillars (security, traffic, observability) and an honest view of the cost. A frequent follow-up is how mesh retries interact with application retries (they multiply; set one layer’s retries deliberately). A common mistake is adopting a mesh for a small system that does not need it.

36

What are the use cases for Kubernetes Federation?

Kubernetes Federation means managing several clusters as one fleet: deploying the same workloads and policies to many clusters from one place, and routing or failing traffic over between them. The original projects (Federation v1 and then KubeFed) are retired, since KubeFed was archived in 2023. The same goals are now met by multi-cluster tools: GitOps fleet management (Argo CD ApplicationSets, Flux), propagation engines (Karmada, Open Cluster Management), cloud fleet services, the Multi-Cluster Services API, and multi-cluster service meshes. The use cases are what matter in an interview.

Read full answer

Applies toKubernetes 1.29+ fleets across regions, clouds or on-premises

Use Cases

Use case Why one cluster is not enough Typical approach
Disaster recovery and regional failover A cluster or region can fail as a unit Same apps in two or more regions; global DNS or load balancer health checks shift traffic
Low latency for global users Cross-region round trips are roughly 50–150 ms Clusters near users; geo-routing DNS or anycast
Data residency and compliance Data must stay in a country or region Per-region clusters with placement rules for which apps and data go where
Blast-radius reduction One bad upgrade, CRD or admission webhook can break a whole cluster Several smaller clusters; staged rollouts cluster by cluster
Scale limits Very large clusters strain etcd, the API server and networking Shard workloads across clusters
Hybrid and multi-cloud Workloads on-premises and in several clouds, or avoiding one provider One control layer that places workloads on any member cluster
Burst capacity On-premises capacity is fixed Overflow to cloud clusters at peak
Edge and retail sites Hundreds of small clusters in stores or factories Fleet management pushing the same config to every site
Environment and tenant separation Hard isolation between prod and dev, or between customers Cluster per environment or tenant, managed centrally

Building Blocks Today

Need Tools
Deploy the same manifests to many clusters Argo CD ApplicationSets, Flux, Rancher Fleet
Policy-driven placement and propagation Karmada, Open Cluster Management, cloud fleet managers
Service discovery across clusters Multi-Cluster Services API (ServiceExport/ServiceImport), Cilium Cluster Mesh, Istio multi-cluster
Global traffic routing and failover GeoDNS, global load balancers, anycast
Consistent policy Policy engines (Kyverno, Gatekeeper) deployed to every cluster from Git

Example: GitOps Fan-Out with an ApplicationSet

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: checkout
  namespace: argocd
spec:
  generators:
    - clusters:
        selector:
          matchLabels:
            env: prod
  template:
    metadata:
      name: "checkout-{{name}}"
    spec:
      project: default
      source:
        repoURL: https://git.example.com/platform/apps.git
        path: checkout/overlays/prod
        targetRevision: main
      destination:
        server: "{{server}}"
        namespace: shop
      syncPolicy:
        automated:
          prune: true
          selfHeal: true

Every cluster registered with the label env: prod gets the checkout app; adding a new region means registering one more cluster.

Request Flow for Regional Failover

User → global DNS or load balancer (health-checked) → nearest healthy region’s ingress → services in that cluster → regional database replica. When a region fails its health checks, new requests go to the next region.

Hard Parts and Trade-offs

  • State is the real problem. Stateless services copy easily; databases need cross-region replication with a chosen consistency model. Synchronous replication adds cross-region latency to writes; asynchronous replication can lose recent writes on failover (a non-zero RPO).

  • Capacity for failover: each surviving region must absorb the failed region’s traffic, so you pay for headroom.

  • Operational cost: more clusters to upgrade, secure and monitor; tooling becomes mandatory.

  • Configuration drift: without GitOps, clusters diverge quickly.

  • Cross-cluster calls add latency and new failure modes; keep request paths within one cluster where possible.

When Not to Do It

A single multi-zone cluster already survives node and zone failures. If the business does not need regional DR, residency or very large scale, multi-cluster adds cost and complexity without benefit. Start with one well-run regional cluster per environment.

Interview Tip

Mention that KubeFed is retired and describe the modern approach (GitOps fan-out plus multi-cluster networking), then focus on why: DR, latency, residency, blast radius and scale. Interviewers usually follow up on data: how do you replicate the database and what happens to writes during failover? A common mistake is designing active-active clusters without a plan for consistent state.

37

How does Helm enhance application management within Kubernetes?

Helm is the package manager for Kubernetes. A chart bundles the templated manifests for an application (Deployment, Service, Ingress, HPA, ConfigMap…) with a values.yaml of defaults. Installing a chart with environment-specific values creates a release, which Helm tracks with numbered revisions, so you can upgrade, diff, roll back and uninstall the whole application as one unit. Charts are shared through repositories and OCI registries, so teams reuse well-tested packages instead of copying YAML.

Read full answer

Applies toKubernetes 1.29+ with Helm 3 or later

What Helm Adds Over Plain Manifests

Problem with raw YAML Helm feature
Same app in dev, staging and prod with small differences One chart, a values file per environment
Dozens of related objects to apply and delete together A release groups them; helm uninstall removes all
No history of what was deployed Release revisions stored in the cluster
Rollback means finding old YAML helm rollback checkout 3
Installing third-party software (ingress controllers, databases, monitoring) Versioned public charts
Ordering tasks such as migrations Hooks (pre-upgrade, post-install)
Shared conventions across services Library charts and dependencies

Chart Structure

checkout/
  Chart.yaml          # name, version, appVersion, dependencies
  values.yaml         # defaults
  values-prod.yaml    # environment overrides (often kept in a separate repo)
  templates/
    deployment.yaml
    service.yaml
    hpa.yaml
    _helpers.tpl      # reusable template snippets
# values.yaml
replicaCount: 2
image:
  repository: registry.example.com/checkout
  tag: "2.1.0"
resources:
  requests:
    cpu: 200m
    memory: 256Mi
autoscaling:
  enabled: true
  maxReplicas: 20
# templates/deployment.yaml (excerpt)
apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ include "checkout.fullname" . }}
  labels:
    {{- include "checkout.labels" . | nindent 4 }}
spec:
  {{- if not .Values.autoscaling.enabled }}
  replicas: {{ .Values.replicaCount }}
  {{- end }}
  template:
    metadata:
      annotations:
        checksum/config: {{ include (print $.Template.BasePath "/configmap.yaml") . | sha256sum }}
    spec:
      containers:
        - name: app
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
          resources:
            {{- toYaml .Values.resources | nindent 12 }}

Two common patterns are shown: replicas is omitted when an HPA owns scaling, and a checksum annotation restarts pods when the ConfigMap changes.

Lifecycle Commands

helm lint ./checkout
helm template checkout ./checkout -f values-prod.yaml | kubectl apply --dry-run=server -f -
helm upgrade --install checkout ./checkout -n shop -f values-prod.yaml --atomic --timeout 5m
helm history checkout -n shop
helm rollback checkout 3 -n shop
helm push checkout-1.4.0.tgz oci://registry.example.com/charts

--atomic rolls back automatically if the upgrade fails or its resources do not become ready within the timeout.

How It Works

Helm runs on the client: it renders templates with the values, sends the resulting manifests to the API server, and stores the release record (rendered manifest plus values) as a Secret in the release’s namespace. Helm 3 removed the old in-cluster Tiller server, so permissions are simply the user’s RBAC.

Trade-offs

Benefit Cost
Packaging, versioning and rollback Go templates over YAML are hard to read and debug (indentation, nindent)
Huge ecosystem of charts Third-party charts can expose hundreds of values and hide insecure defaults
Hooks for ordered steps Hooks are not tracked like normal resources and can fail in confusing ways
One unit per app Rollback restores manifests, not data; database migrations do not roll back
CRD installation Helm installs CRDs from crds/ on first install but does not upgrade them

Helm and Kustomize

Helm suits distributable packages and apps with many configurable options; Kustomize suits patching plain YAML per environment without templates. Many teams combine them: use Helm for third-party software and Kustomize (or Helm with small values files) for their own services. GitOps tools such as Argo CD and Flux render Helm charts themselves, keeping the desired state in Git.

Interview Tip

Interviewers look for charts, values, releases and revisions, and for the limits: rollback does not undo schema migrations, and CRD upgrades need separate handling. A common mistake is putting secrets in plain values.yaml committed to Git; use an external secret store or encrypted values.

38

Explain the role of Kustomize in Kubernetes configuration management.

Kustomize customises plain Kubernetes YAML without templates. You keep a base of ordinary manifests and create overlays for each environment that patch them: change replicas, images, resources, add labels or a name prefix, and generate ConfigMaps and Secrets. Because it is built into kubectl (kubectl apply -k), and every input and output is valid Kubernetes YAML, it fits GitOps workflows where each environment’s directory is the desired state.

Read full answer

Applies toKubernetes 1.29+ (`kubectl apply -k`, standalone `kustomize`)

The Base and Overlay Model

checkout/
  base/
    kustomization.yaml
    deployment.yaml
    service.yaml
  overlays/
    staging/
      kustomization.yaml
    prod/
      kustomization.yaml
      replicas-patch.yaml
# base/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - deployment.yaml
  - service.yaml
labels:
  - pairs:
      app.kubernetes.io/name: checkout
    includeSelectors: true
# overlays/prod/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: shop-prod
resources:
  - ../../base
images:
  - name: registry.example.com/checkout
    newTag: "2.1.0"
patches:
  - path: replicas-patch.yaml
  - target:
      kind: Deployment
      name: checkout
    patch: |-
      - op: add
        path: /spec/template/spec/containers/0/env/-
        value:
          name: LOG_LEVEL
          value: warn
configMapGenerator:
  - name: checkout-config
    literals:
      - FEATURE_NEW_CART=true
      - PAYMENT_TIMEOUT_MS=2000
# overlays/prod/replicas-patch.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: checkout
spec:
  replicas: 6
kubectl kustomize overlays/prod          # print the rendered YAML
kubectl diff -k overlays/prod
kubectl apply -k overlays/prod

The JSON patch’s env/- path appends to an existing list, so the base container must already define env; otherwise use a strategic-merge patch.

Main Features

Feature What it does
resources Includes manifests, directories or remote bases
patches Strategic-merge patches (partial YAML) or JSON 6902 patches (precise operations)
images Changes image names, tags or digests without editing manifests
namespace, namePrefix, nameSuffix Applies to all resources and updates references
labels, commonAnnotations Adds metadata everywhere
configMapGenerator, secretGenerator Builds ConfigMaps and Secrets with a content hash suffix
replicas Sets replica counts by name
components Reusable optional features (for example “enable tracing”) mixed into several overlays

The hash suffix is a key benefit: when the config content changes, the ConfigMap name changes (checkout-config-7k2f9m), the Deployment reference is updated, and pods roll automatically. Old ConfigMaps stay until pruned.

Kustomize versus Helm

Aspect Kustomize Helm
Approach Patch plain YAML Render templates with values
Readability Manifests stay valid YAML Templates mix Go syntax and YAML
Packaging and versioning None; directories in Git Charts with versions, repositories, OCI
Release history and rollback None; Git history and GitOps instead Built-in revisions and helm rollback
Conditional logic Limited by design Full template logic
Best for Your own services across environments Distributable, highly configurable packages

They combine well: Kustomize can render a Helm chart (helmCharts) and patch the output, and Argo CD and Flux support both.

Trade-offs and Pitfalls

  • Deep overlay chains (base → region → environment → cluster) become hard to follow; keep at most two levels and use components for optional features.

  • Patches fail silently or loudly when the base changes names or list positions; JSON patches by index are fragile.

  • secretGenerator output is still a plain Secret; keep secret values out of Git and use an external secret store or encrypted files.

  • No release concept means no built-in uninstall; use labels with kubectl delete -k or GitOps pruning.

Interview Tip

Interviewers want base plus overlays, template-free patching, and the generator hash suffix that triggers rollouts on config changes. Be ready to compare it with Helm and say when you would use each. A common mistake is copying whole manifests into each environment folder, which defeats the point; overlays should contain only differences.

39

Describe the importance of a Container Registry in a Kubernetes environment.

A container registry stores and serves the images that every pod runs. Kubernetes never builds images: each time a pod starts on a node that does not have the image cached, the kubelet asks the container runtime to pull it from a registry. That makes the registry a runtime dependency for scaling, rollouts and recovery, and a key supply-chain security control: what is in the registry, and who can push to it, decides what can run in the cluster.

Read full answer

Applies toKubernetes 1.29+ with OCI registries (Harbor, Amazon ECR, Google Artifact Registry, Azure Container Registry, GitHub Container Registry, Docker Hub)

Why It Matters

Concern Role of the registry
Deployment Source of every image version; rollouts and rollbacks pull from it
Availability New pods (scale-up, node failure, drain) cannot start if the registry is unreachable and the image is not cached
Performance Pull time adds to pod start-up; large images and distant registries slow scaling
Security Access control, vulnerability scanning, signatures, immutable tags
Compliance and audit Record of what was pushed, by whom, and what is running
Cost Storage and cross-region data transfer; public registry rate limits
Artifacts OCI registries also store Helm charts, SBOMs and signatures

Pulling from a Private Registry

kubectl create secret docker-registry regcred -n shop \
  --docker-server=registry.example.com --docker-username=ci-bot --docker-password="$REGISTRY_TOKEN"
apiVersion: apps/v1
kind: Deployment
metadata:
  name: checkout
  namespace: shop
spec:
  replicas: 3
  selector:
    matchLabels:
      app: checkout
  template:
    metadata:
      labels:
        app: checkout
    spec:
      imagePullSecrets:
        - name: regcred
      containers:
        - name: app
          image: registry.example.com/shop/checkout@sha256:4f6a2c0d9e1b7a35c8f2d6e4b9a1c3d5e7f9a2b4c6d8e0f1a3b5c7d9e1f2a4b6
          imagePullPolicy: IfNotPresent

On managed clouds, prefer node or workload identity (for example IAM roles for nodes pulling from ECR) over long-lived pull secrets. Attach pull secrets to a ServiceAccount to avoid repeating them in every manifest.

Tags and Digests

  • A tag (:2.1.0) is a movable pointer; someone can push a different image under the same tag.

  • A digest (@sha256:…) identifies exact content and cannot change.

  • Enable tag immutability in the registry, never deploy :latest to production, and let CI write the digest into manifests. Then every node runs the same bytes and rollbacks are exact.

  • imagePullPolicy: Always re-checks the registry on every start; with immutable tags or digests, IfNotPresent avoids unnecessary dependency on the registry.

Supply-Chain Security

  1. Scan images on push and continuously (Trivy, Grype, or the registry’s built-in scanner); block critical vulnerabilities in CI.

  2. Sign images in CI (Sigstore cosign) and attach an SBOM.

  3. Verify at admission: Kyverno, Sigstore policy-controller or Gatekeeper reject unsigned images or images from unapproved registries.

  4. Restrict push access to CI identities; developers pull only.

  5. Use minimal base images (distroless, slim) to reduce vulnerabilities and pull time.

Reliability and Performance

Practice Benefit
Registry in the same region as the cluster, or geo-replicated Faster pulls, no cross-region transfer cost
Pull-through cache or mirror for public images Avoids Docker Hub rate limits and outages
Small images, shared base layers Faster scale-up; layers are cached per node
Pre-pulling critical images (DaemonSet or node image) Fast start for latency-sensitive scaling
Retention and garbage-collection policies Controls storage cost; keep images still referenced by running releases
Monitoring pull errors (ErrImagePull, ImagePullBackOff) Early warning of registry or credential problems

Trade-offs

  • Managed cloud registries integrate with IAM and need no operations, but tie you to a provider.

  • Self-hosted Harbor gives control, replication and scanning on-premises, at the cost of running a critical service with its own HA and backups.

  • Public Docker Hub images are convenient, but rate-limited and outside your control; mirror the ones you depend on.

Interview Tip

Interviewers want to hear that the registry is a runtime dependency (an outage blocks scaling and recovery) and a security boundary (signing, scanning, admission control). Mention digests versus tags. A common mistake is deploying :latest with imagePullPolicy: Always, which makes rollbacks meaningless and every pod start depend on the registry.

40

What best practices would you follow for high availability in Kubernetes deployments?

High availability means no single failure takes the service down: not a pod, a node, a zone, an upgrade or a bad release. In Kubernetes that comes from multiple replicas spread across nodes and zones, health probes and graceful shutdown so traffic only reaches pods that can serve, PodDisruptionBudgets for planned disruption, safe rollouts, enough spare capacity, an HA control plane, and removing single points of failure in dependencies, since an app is only as available as its database, DNS and ingress.

Read full answer

Applies toKubernetes 1.29+ on multi-zone clusters

As rough targets, 99.9% availability allows about 8.8 hours of downtime a year, and 99.99% about 53 minutes, which leaves no room for manual recovery.

Workload-Level Practices

Practice Protects against How
At least 2–3 replicas Pod or node loss replicas or HPA minReplicas
Spread across nodes and zones Node and zone failure topologySpreadConstraints
Readiness, liveness and startup probes Traffic to broken or starting pods Probes that test real readiness
Graceful shutdown Dropped requests during rollouts and drains SIGTERM handling, preStop delay, grace period
PodDisruptionBudget Drains and upgrades removing too many pods maxUnavailable: 1
Resource requests and limits Eviction and noisy neighbours Requests from measured usage; memory limits
PriorityClass for critical services Being preempted under pressure Higher priority for core workloads
Safe rollout strategy Bad releases maxUnavailable: 0, canaries, quick rollback

Example

apiVersion: apps/v1
kind: Deployment
metadata:
  name: checkout
spec:
  replicas: 3
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  selector:
    matchLabels:
      app: checkout
  template:
    metadata:
      labels:
        app: checkout
    spec:
      priorityClassName: business-critical
      terminationGracePeriodSeconds: 40
      topologySpreadConstraints:
        - maxSkew: 1
          topologyKey: topology.kubernetes.io/zone
          whenUnsatisfiable: DoNotSchedule
          labelSelector:
            matchLabels:
              app: checkout
        - maxSkew: 1
          topologyKey: kubernetes.io/hostname
          whenUnsatisfiable: ScheduleAnyway
          labelSelector:
            matchLabels:
              app: checkout
      containers:
        - name: app
          image: registry.example.com/checkout:2.1.0
          resources:
            requests:
              cpu: 300m
              memory: 384Mi
            limits:
              memory: 768Mi
          startupProbe:
            httpGet:
              path: /healthz
              port: 8080
            failureThreshold: 30
            periodSeconds: 2
          readinessProbe:
            httpGet:
              path: /ready
              port: 8080
          livenessProbe:
            httpGet:
              path: /healthz
              port: 8080
          lifecycle:
            preStop:
              sleep:
                seconds: 10
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: checkout
spec:
  maxUnavailable: 1
  selector:
    matchLabels:
      app: checkout

The business-critical PriorityClass must exist in the cluster. Keep liveness probes simple (is the process stuck?) and never make them depend on a database; otherwise a database blip restarts every pod at once.

Cluster-Level Practices

  • HA control plane: three control plane nodes for etcd quorum, or a managed control plane with an SLA. Back up etcd if you run it.

  • Nodes in at least three zones, with node pools sized so losing one zone leaves enough capacity: with three zones, run at about two-thirds utilisation or rely on fast autoscaling.

  • Cluster autoscaler or Karpenter, plus a little headroom (low-priority placeholder pods that get preempted) so new pods schedule immediately.

  • Critical add-ons (CoreDNS, ingress or Gateway controllers, CNI, Metrics Server) with multiple replicas, PDBs and spread; they are shared single points of failure.

  • Regular, tested upgrades: one node pool at a time, with surge nodes.

Dependencies and Data

  • Use managed or replicated databases across zones; a StatefulSet with one replica is a single point of failure.

  • Use timeouts, retries with exponential backoff and jitter, circuit breakers and bulkheads for calls between services.

  • Design graceful degradation: serve cached or partial results when a non-critical dependency fails.

  • Use idempotency keys for retried writes such as payments.

Beyond One Cluster

For regional failures, run clusters in two regions behind health-checked global DNS or load balancing, with a replication strategy for data and capacity to absorb the failover.

Verify It

  • Run game days: drain a node, cordon a zone, kill pods, and fail a dependency in staging.

  • Alert on symptoms (error rate, latency, SLO burn) rather than only on CPU.

  • Review incidents for single points of failure you missed.

Interview Tip

Interviewers look for layers: replicas and spread, probes and graceful shutdown, PDBs, capacity, control plane, dependencies. Quote the downtime numbers to show you know what each “nine” costs. A common mistake is three replicas that all land in one zone, or liveness probes that check the database and cause cascading restarts.

41

How do ConfigMaps and Secrets improve configuration management?

ConfigMaps hold non-sensitive configuration (feature flags, URLs, config files) and Secrets hold sensitive values (passwords, tokens, TLS keys) as Kubernetes objects separate from the container image. They improve configuration management because the same image runs in every environment with different config, changes are declarative and versioned alongside the manifests, access can be controlled with RBAC, and values are injected as environment variables or files without rebuilding. Secrets are only base64-encoded by default, so they need encryption at rest and tight access control.

Read full answer

Applies toKubernetes 1.29+

What They Solve

Without them With them
Config baked into images, one build per environment One immutable image promoted from staging to production
Credentials in Git or in Dockerfiles Secrets with restricted RBAC and encryption at rest
Editing files on servers Declarative objects reviewed in pull requests
Config scattered across scripts One place per namespace, visible with kubectl

Ways to Consume Them

apiVersion: v1
kind: ConfigMap
metadata:
  name: checkout-config
data:
  PAYMENT_TIMEOUT_MS: "2000"
  FEATURE_NEW_CART: "true"
  app.yaml: |
    cache:
      ttlSeconds: 300
---
apiVersion: v1
kind: Secret
metadata:
  name: checkout-db
type: Opaque
stringData:
  DATABASE_PASSWORD: example-only-change-me
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: checkout
spec:
  replicas: 2
  selector:
    matchLabels:
      app: checkout
  template:
    metadata:
      labels:
        app: checkout
    spec:
      containers:
        - name: app
          image: registry.example.com/checkout:2.1.0
          env:
            - name: PAYMENT_TIMEOUT_MS
              valueFrom:
                configMapKeyRef:
                  name: checkout-config
                  key: PAYMENT_TIMEOUT_MS
          envFrom:
            - secretRef:
                name: checkout-db
          volumeMounts:
            - name: config
              mountPath: /etc/checkout
              readOnly: true
      volumes:
        - name: config
          configMap:
            name: checkout-config
            items:
              - key: app.yaml
                path: app.yaml

stringData lets you write plain values that the API server stores base64-encoded; in practice the Secret would come from an external secret store rather than a committed file.

Method Updates when the object changes? Notes
env / envFrom No, only at container start Simple; values visible in process environment
Volume mount Yes, eventually (kubelet sync, typically within a minute or two) The app must re-read the file
Volume with subPath No Common surprise
Read through the API Immediately Needs RBAC and client code; rarely worth it

Rolling Out Config Changes

Changing a ConfigMap does not restart pods. Options:

  • Hash-suffixed names (Kustomize configMapGenerator) or a checksum annotation (Helm) so the pod template changes and a rolling update happens.

  • kubectl rollout restart deployment/checkout after the change.

  • Apps that watch mounted files and reload on change.

Rolling out config through a new pod template has a further benefit: the change goes through the same rolling update, readiness checks and rollback path as code.

Good Practices

  • Mark config that should not change after creation as immutable: true; it protects against accidental edits and reduces kubelet watch load. Change it by creating a new object.

  • Keep objects under the 1 MiB size limit; large files belong in a volume or object storage.

  • Encrypt Secrets at rest with a KMS provider, restrict get/list on Secrets, and prefer an external store (Vault, cloud secret managers) synced with External Secrets or mounted with the Secrets Store CSI driver.

  • Prefer files over environment variables for secrets; env vars leak more easily into logs and child processes.

  • Validate configuration at start-up and fail fast with a clear message.

Trade-offs

Choice Benefit Cost
Env vars Works with any app Restart needed for every change
Mounted files Live updates possible App must support reload; eventual, not instant
Immutable objects Safety, lower API load Every change is a new object name
External config service Dynamic flags, audit, per-user targeting Another dependency on the request path

Interview Tip

Interviewers want separation of config from images (build once, deploy everywhere), the env versus volume update behaviour, and the reminder that Secrets are base64, not encrypted. A common follow-up is “you changed a ConfigMap and nothing happened, why?”: pods read env vars only at start, or the file is mounted with subPath.

42

Describe multitenancy considerations for resource utilization in Kubernetes.

When several teams or customers share a cluster, resource utilization must be fair, bounded and accountable: no tenant should starve others (the noisy neighbour problem), every tenant’s usage should be capped and visible, and the cluster should still be packed efficiently. The main tools are requests and limits, ResourceQuota and LimitRange per tenant namespace, QoS classes and PriorityClasses for what gets evicted or preempted first, dedicated node pools where isolation matters more than efficiency, and cost allocation by labels.

Read full answer

Applies toKubernetes 1.29+ with namespace-per-tenant (soft) multitenancy

Layers of Control

Layer Tool Controls
Per container Requests and limits Scheduling guarantees (requests) and hard caps (limits)
Per namespace defaults LimitRange Default and maximum requests and limits for containers
Per namespace totals ResourceQuota Total CPU, memory, storage, GPUs, object counts
Who is evicted first QoS class Guaranteed, Burstable, BestEffort
Who is scheduled or preempted first PriorityClass Critical tenants or system workloads first
Physical isolation Node pools with taints, tolerations and affinity Tenants on separate nodes
API fairness API Priority and Fairness One tenant’s controllers cannot flood the API server

Quotas and Defaults per Tenant

apiVersion: v1
kind: ResourceQuota
metadata:
  name: tenant-quota
  namespace: tenant-acme
spec:
  hard:
    requests.cpu: "16"
    requests.memory: 32Gi
    limits.memory: 64Gi
    requests.storage: 500Gi
    persistentvolumeclaims: "20"
    requests.nvidia.com/gpu: "2"
    services.loadbalancers: "2"
    pods: "150"
---
apiVersion: v1
kind: LimitRange
metadata:
  name: tenant-defaults
  namespace: tenant-acme
spec:
  limits:
    - type: Container
      defaultRequest:
        cpu: 100m
        memory: 128Mi
      default:
        memory: 512Mi
      max:
        cpu: "4"
        memory: 8Gi

When a quota covers requests.cpu, every new pod must declare a CPU request or it is rejected; the LimitRange supplies defaults so tenants are not blocked by that rule. Quotas on services.loadbalancers and storage prevent one tenant running up cloud costs.

QoS Classes and Eviction

QoS class Condition Under node memory pressure
Guaranteed Requests equal limits for CPU and memory in every container Evicted last
Burstable Some requests set, not all equal to limits Evicted based on usage above requests
BestEffort No requests or limits Evicted first

Overcommit and Its Risks

Requests decide scheduling; usage above requests is opportunistic. Setting limits higher than requests lets tenants burst into idle capacity, which raises utilization, but:

  • Memory cannot be throttled; when a node runs out, the kernel OOM-kills containers or the kubelet evicts pods. Keep memory limits close to requests for critical tenants.

  • CPU limits cause throttling even when the node is idle. Many teams set CPU requests but no CPU limits, relying on requests for fair CPU shares, while keeping limits for memory.

  • Low requests with high real usage make the scheduler overpack nodes. Compare requests with actual usage regularly and right-size (VPA in recommendation mode helps).

Noisy Neighbours Beyond CPU and Memory

Resource Risk Mitigation
Disk I/O and ephemeral storage One tenant’s logs or temp files fill the node disk ephemeral-storage requests and limits, log rotation
Network bandwidth Bulk transfers hurt latency for others Separate node pools, CNI bandwidth limits
PIDs Fork bombs Kubelet podPidsLimit
API server Controllers or CI hammering the API API Priority and Fairness, object-count quotas
Shared caches and CPU caches Hard to isolate Dedicated nodes for latency-sensitive tenants

Dedicated Nodes versus Shared Nodes

Approach Utilization Isolation Cost
Shared nodes, quotas only High Lowest Cheapest
Tenant node pools (taints plus affinity) Medium; idle capacity per pool Better: no shared kernel or node resources More nodes
Virtual clusters (vCluster) or cluster per tenant Lower Strongest Highest

Hard multitenancy for untrusted customers usually needs separate node pools or clusters, plus sandboxed runtimes such as gVisor or Kata Containers, not just quotas.

Visibility and Chargeback

  • Label namespaces and workloads with tenant and cost-center, and enforce the labels with a policy engine.

  • Use OpenCost or Kubecost to attribute CPU, memory, storage and network cost per tenant, based on requests (what was reserved) or usage, and agree which.

  • Dashboards per tenant: quota used versus allowed, requests versus actual usage, throttling and OOM kills.

Interview Tip

Interviewers want to hear requests versus limits (scheduling versus caps), ResourceQuota plus LimitRange per namespace, QoS and priority for what is sacrificed under pressure, and the utilization versus isolation trade-off of dedicated nodes. A common mistake is setting quotas without LimitRange defaults, so tenants’ pods are rejected, or relying on namespaces alone for isolation between untrusted tenants.

Sign in to keep reading

You've opened 2 full answers. Sign in to read all 42 Kubernetes answers, mark what you know and pick up where you left off.

Report a mistake

What's wrong?

All System Design subjects