kubelet Explained: The Node Agent

By Khimananda Oli | Last reviewed: August 2026

When a pod fails to start or a node shows NotReady, the root cause usually lies with the node agent itself. Kubelet Explained: The Node Agent is your reference for understanding how this critical component bridges the Kubernetes control plane and the actual containers running on your hardware. It is the primary actor responsible for translating API server declarations into running processes, making its correct configuration essential for any production cluster.

What does kubelet actually do as the Kubernetes node agent?

The kubelet acts as the local supervisor for every node in your cluster. Unlike the API server or scheduler which run centrally, the kubelet runs locally on each machine and maintains a persistent gRPC connection to both the container runtime and the Kubernetes API. Its responsibilities are distinct from other components like kube-proxy or the CNI plugin.

In practice, the kubelet performs four non-negotiable functions that define its role as the node agent:

• Node Registration: On startup, it creates or updates the Node object in the API server, reporting capacity (CPU, memory, ephemeral storage), allocatable resources, and conditions like disk pressure or memory pressure.
• Pod Lifecycle Management: It watches for PodSpecs assigned to its node, then calls the Container Runtime Interface (CRI) to pull images, create sandboxes, and start containers in the correct order.
• Status Reporting: Every few seconds, it sends heartbeats and detailed status updates including container states, restart counts, and resource usage metrics back to the control plane.
• Resource Enforcement: It works with cgroups to ensure containers stay within their requested CPU and memory limits, killing or evicting pods when node-level thresholds are breached.

Understanding these boundaries prevents confusion with Kubernetes resource limits and requests, which the kubelet enforces but does not calculate. The scheduler decides placement; the kubelet executes it.

How does kubelet interact with CRI and the container runtime?

Since Kubernetes v1.24 removed dockershim, the kubelet no longer speaks Docker directly. Instead, it uses the Container Runtime Interface (CRI), a standardized gRPC protocol that decouples the node agent from any specific runtime. This abstraction is why you can swap containerd for CRI-O without changing kubelet code.

CRI Communication Flow

1. Socket Connection: Kubelet connects to the runtime via a Unix domain socket, typically /run/containerd/containerd.sock or /var/run/crio/crio.sock.
2. Sandbox Creation: Before starting application containers, kubelet requests a pause sandbox (infra container) that holds network namespaces and cgroup configurations.
3. Image Pull: Kubelet issues PullImage RPCs. If image pull secrets are needed, they are passed through the CRI auth mechanism.
4. Container Start: After the sandbox is ready, kubelet sends CreateContainer and StartContainer calls with the full OCI spec derived from the PodSpec.
5. Streaming & Exec: For kubectl logs or kubectl exec, kubelet proxies streaming requests through CRI endpoints rather than handling I/O directly.

<!-- Example: Checking CRI socket status -->
$ crictl info
{
  "config": {
    "containerdEndpoint": "/run/containerd/containerd.sock",
    "runtimeEndpoint": "/run/containerd/containerd.sock"
  },
  "runtimeVersion": "v1.30.2"
}

A common mistake in 2026 is assuming the kubelet manages networking. It does not. The kubelet invokes the CNI plugin via the runtime’s sandbox setup, but IP allocation and route programming happen entirely outside the kubelet process. For deeper networking context, see Kubernetes network policies explained.

How do you configure kubelet for production stability?

Default kubelet settings work for development but often fail under production load or compliance requirements. Configuration lives in two places: command-line flags (legacy) and the structured KubeletConfiguration file (recommended since v1.28+).

Critical Production Parameters

# /etc/kubernetes/kubelet-config.yaml (KubeletConfiguration v1beta1)
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
maxPods: 150
serializeImagePulls: false
imageGCHighThresholdPercent: 85
imageGCLowThresholdPercent: 70
evictionHard:
  memory.available: "500Mi"
  nodefs.available: "10%"
rotateCertificates: true
serverTLSBootstrap: true
authentication:
  anonymous:
    enabled: false
  webhook:
    enabled: true

In Nepal-based deployments where bandwidth to global registries is limited, setting serializeImagePulls: false combined with a local registry mirror dramatically improves pod startup times during scaling events. Always validate config changes with kubelet --config /path/to/config.yaml --dry-run before restarting the service.

How do you debug kubelet when pods fail to start?

When kubectl describe pod shows ContainerCreating indefinitely or ErrImagePull without clear cause, the kubelet logs are your primary diagnostic tool. Unlike application logs, kubelet output reveals infrastructure-layer failures that never reach the pod’s stdout.

Essential Diagnostic Commands

# View live kubelet logs with context
journalctl -u kubelet -f --no-pager -n 200

# Check node conditions for pressure signals
kubectl get node <node-name> -o jsonpath='{.status.conditions}' | jq

# Validate CRI connectivity
crictl ps -a
crictl pods

# Inspect specific pod sandbox failure
crictl inspectp <sandbox-id>

# Verify kubelet config is loaded correctly
kubelet --config /etc/kubernetes/kubelet-config.yaml --dry-run 2>&1 | head -20

For CrashLoopBackOff issues specifically tied to resource constraints, cross-reference kubelet eviction logs with your manifest limits. My guide on debugging CrashLoopBackOff in Kubernetes covers the application-side correlation.

Kubelet vs kube-proxy: what’s the difference in responsibilities?

Engineers new to Kubernetes often conflate these two node-level daemons because both run on every worker. Their functions are completely orthogonal, and confusing them leads to wasted debugging time.

The kubelet owns everything related to running containers: image pulls, cgroup enforcement, probe execution, volume mounting, and status reporting. Kube-proxy owns everything related to reaching containers: translating Service abstractions into kernel-level routing rules so traffic arrives at the correct pod IP. You can delete kube-proxy entirely in eBPF-based clusters (e.g., Cilium) and the kubelet continues functioning perfectly. Conversely, if kubelet dies, pods stop being managed regardless of whether networking is healthy.

This separation matters for incident response. If pods are running but unreachable, investigate kube-proxy, CNI, or network policies. If pods aren’t starting or keep restarting, investigate kubelet, CRI, or resource constraints. Mixing these domains wastes hours during outages.

Maintaining Kubelet Health in Production Clusters

Treating the kubelet as a set-and-forget daemon guarantees eventual failure. In production, especially for teams pursuing SOC 2 or ISO 27001 compliance, you must actively monitor and maintain the node agent itself. Enable serverTLSBootstrap to ensure all kubelet-to-API communication uses rotated certificates. Set up Prometheus scraping of the kubelet’s /metrics endpoint to track pod start latency, image pull duration, and eviction rates. These metrics form the foundation of meaningful SLIs and SLOs for node reliability.

On Ubuntu systems, pin your containerd and kubelet versions together during upgrades to avoid CRI incompatibilities. Use apt-mark hold kubelet during maintenance windows to prevent unattended upgrades from breaking node agents mid-deployment. For teams managing multiple clusters across regions, consider GitOps-driven kubelet configuration via tools like Kubespray or Cluster API to ensure consistency and auditability.

If your cluster shows intermittent NotReady flaps or slow pod scheduling despite healthy control plane metrics, the issue almost always traces back to kubelet misconfiguration or resource starvation at the node level. Audit your current settings against the parameters outlined here, validate with dry-run, and monitor the results. When you need hands-on assistance hardening node agents for compliance or performance, reach out to discuss your infrastructure.