You run kubectl delete pod, and a few seconds later a pod with a different name is running in its place. It looks as if Kubernetes ignored you. In fact it did what you asked for earlier, when you wrote replicas: 3.
Declared state, not commands
Kubernetes is a reconciliation loop. You declare the state you want, and controllers keep comparing it with the state they observe, acting to close the gap. "Three replicas" is not an instruction to start three pods. It is an invariant that something works permanently to maintain.
Delete one of those pods and the ReplicaSet controller now observes two where three are declared, so it creates another. Nothing replayed your command. The replacement even arrives with a new name, because the declaration counted pods and never named them.
One idea, four surprises
| What you see | What is actually happening |
|---|---|
| A pod you deleted comes back | The ReplicaSet controller observed 2 where 3 are declared and closed the gap |
A kubectl edit is reverted minutes later | A GitOps controller observed a difference from the repository, and the repository won |
| A deleted node's pods reappear elsewhere | The same loop, run by a different controller: the declaration said three running pods, not three pods on those nodes |
| Scaling to zero and back restores everything | The declaration only ever said how many; identity was never part of the contract, which is exactly what a StatefulSet changes |
The operating habit follows from the same idea. You do not fix a Kubernetes system by issuing commands. You change what is declared, and let the loop close.
Engineering Insight
A cluster run through GitOps has two sources of truth, the cluster and the repository, and the repository wins. A fix applied with kubectl edit and never committed will be reverted, so make the change where it is declared.
When the loop is worth running
The loop has a price. A cluster is a distributed system you now operate: control-plane upgrades on a support window you do not set, node image rotation, and API deprecations that break manifests between minor versions. The course puts the choice as five situations:
- Three services, one team, one cloud: ECS or Fargate does the same job with a fraction of the API surface, and there is no cluster to upgrade.
- Many services and many teams: Kubernetes, because its API becomes the contract between a platform team and its users.
- A genuine portability requirement: Kubernetes. "We might move clouds" is not one; a signed commitment is.
- A platform team whose product this is: Kubernetes. The complexity is the product, and someone is paid to own it.
- "We might need to scale": neither. That names no workload, no constraint and no date. An autoscaling group scales.
Kubernetes earns its complexity in three of those five. The skill is being able to say which one you are in.
