Graceful Shutdown and Health Checks in Rust

By Khimananda Oli | Last reviewed: August 2026

Dropping active connections during deploys is a common failure mode for backend services, but implementing graceful shutdown and health checks in Rust prevents this entirely. When running in Kubernetes or behind a load balancer, your application must explicitly handle termination signals and expose standardized probe endpoints to avoid 502 errors and failed audits. This guide walks through the exact Tokio patterns and Axum configurations needed to make your Rust service production-ready.

How do you handle OS signals for graceful shutdown in Rust?

In production environments like AWS EKS or on-premise Kubernetes clusters, the orchestrator sends a SIGTERM signal when terminating a pod. If your Rust application ignores this, the process is eventually killed with SIGKILL after the grace period expires, severing active TCP connections and potentially corrupting in-flight transactions. Proper signal handling is non-negotiable for zero-downtime deployments.

Tokio provides a cross-platform signal abstraction that integrates directly into your async runtime. The pattern involves selecting between your server's future and a signal future. When the signal resolves, you trigger the shutdown mechanism on your HTTP framework.

use tokio::signal;
use std::time::Duration;

#[tokio::main]
async fn main() {
    let app = axum::Router::new()
        .route("/", axum::routing::get(|| async { "Hello" }));

    let listener = tokio::net::TcpListener::bind("0.0.0.0:8080")
        .await
        .unwrap();

    // Create the server instance so we can call .into_make_service()
    // and retain control over shutdown
    axum::serve(listener, app)
        .with_graceful_shutdown(shutdown_signal())
        .await
        .unwrap();
}

async fn shutdown_signal() {
    let ctrl_c = async {
        signal::ctrl_c()
            .await
            .expect("failed to install Ctrl+C handler");
    };

    #[cfg(unix)]
    let terminate = async {
        signal::unix::signal(signal::unix::SignalKind::terminate())
            .expect("failed to install signal handler")
            .recv()
            .await;
    };

    #[cfg(not(unix))]
    let terminate = std::future::pending::<()>();

    tokio::select! {
        _ = ctrl_c => {},
        _ = terminate => {},
    }

    eprintln!("Shutdown signal received, starting graceful drain...");
}

A common mistake is placing heavy cleanup logic inside the signal handler itself. Keep the handler lean: its only job is to notify the server to stop accepting new connections. Resource cleanup should happen via Drop implementations or explicit shutdown hooks registered with your dependency pool.

Setting a hard timeout for stuck requests

Even with graceful draining, some clients may hold connections open indefinitely. Always pair your signal handler with a maximum drain timeout. If active requests haven't completed within this window, force-exit to prevent the orchestrator from killing the process uncleanly.

// Wrap your graceful shutdown with a timeout
let shutdown_with_timeout = async {
    tokio::select! {
        _ = shutdown_signal() => {},
        _ = tokio::time::sleep(Duration::from_secs(30)) => {
            eprintln!("Graceful shutdown timed out after 30s, forcing exit");
        }
    }
};

What are the correct health check endpoints for Kubernetes?

Kubernetes uses three distinct probes, and conflating them causes cascading failures. Your Rust service must expose separate endpoints for each probe type, returning appropriate HTTP status codes and response bodies.

• Liveness Probe: Answers "Is the process alive?" A simple 200 OK indicates the binary hasn't deadlocked. Do NOT check database connectivity here; if the DB is down but your app is healthy, failing liveness restarts every pod simultaneously.
• Readiness Probe: Answers "Can this pod serve traffic?" Check critical dependencies like PostgreSQL or Redis. Return 503 if any dependency is unreachable, causing the load balancer to remove this pod from rotation without restarting it.
• Startup Probe: Answers "Has initialization completed?" Essential for Rust services with heavy warm-up phases (loading ML models, pre-computing caches). Use a longer failure threshold to accommodate slow starts.

Implementing dependency-aware readiness checks

Your readiness endpoint must actually verify connectivity, not just return a static response. Use connection pool ping methods with short timeouts to avoid blocking the probe under load. This approach aligns with principles covered in monitoring golden signals, where saturation and availability drive operational decisions.

use axum::{Json, http::StatusCode};
use serde::Serialize;
use sqlx::PgPool;

#[derive(Serialize)]
struct HealthResponse {
    status: &'static str,
    database: &'static str,
    cache: &'static str,
}

async fn readiness_handler(
    State(pool): State<PgPool>,
) -> Result<Json<HealthResponse>, StatusCode> {
    // Check DB with a 2-second timeout
    let db_ok = tokio::time::timeout(
        Duration::from_secs(2),
        sqlx::query("SELECT 1").execute(&pool)
    ).await.is_ok_and(|r| r.is_ok());

    // Check Redis/cache similarly
    let cache_ok = check_redis().await;

    if db_ok && cache_ok {
        Ok(Json(HealthResponse {
            status: "healthy",
            database: "connected",
            cache: "connected",
        }))
    } else {
        Err(StatusCode::SERVICE_UNAVAILABLE)
    }
}

How does Axum integrate with Tokio for zero-downtime deploys?

Axum 0.7+ delegates graceful shutdown entirely to Tokio's signal infrastructure rather than implementing its own. This design choice means your shutdown behavior is consistent whether you're running bare-metal, in Docker, or on managed Kubernetes. The key integration point is axum::serve().with_graceful_shutdown(), which accepts any future that completes when shutdown should begin.

During the drain phase, Axum stops accepting new TCP connections on the bound socket but continues processing in-flight requests. Each active request is tracked internally, and the server future only resolves once all requests complete or the outer timeout fires. This is fundamentally different from older frameworks that required manual connection tracking.

Coordinating background tasks during shutdown

Most production Rust services run background workers alongside the HTTP server: queue consumers, metric exporters, cache warmers. These must also respect the shutdown signal. Use tokio::select! in each worker loop, or share a CancellationToken from the tokio-util crate for cleaner coordination across multiple tasks.

use tokio_util::sync::CancellationToken;

async fn queue_worker(token: CancellationToken) {
    loop {
        tokio::select! {
            _ = token.cancelled() => {
                eprintln!("Queue worker shutting down gracefully");
                break;
            }
            msg = dequeue_next_job() => {
                if let Some(job) = msg {
                    process_job(job).await;
                }
            }
        }
    }
}

Why do readiness probes fail after implementing graceful shutdown?

The most frequent issue engineers encounter is a race condition between the shutdown signal and probe responses. When Kubernetes sends SIGTERM, it simultaneously removes the pod from the service endpoints list. However, there's a propagation delay (typically 1-3 seconds) before all load balancers update their routing tables. During this window, your readiness endpoint may still receive requests even though shutdown has begun.

To handle this correctly, your readiness handler should check whether a shutdown has been initiated and return 503 immediately if so. This accelerates endpoint removal and prevents new traffic from being routed to a draining pod. Store shutdown state in an Arc<AtomicBool> shared between your signal handler and route handlers.

Testing shutdown behavior locally

Don't wait for staging to validate your implementation. Write integration tests that spawn your server, send a request with an artificial delay, issue SIGTERM, and assert the delayed request completes successfully while new connections are refused. The tokio-test and axum::test crates provide utilities for this, but real signal testing requires spawning an actual process.

// Pseudocode for shutdown integration test
// 1. Spawn server as child process
// 2. Send GET /slow-endpoint (sleeps 3s)
// 3. After 500ms, send SIGTERM to child
// 4. Assert /slow-endpoint returns 200
// 5. Assert new GET / returns connection refused
// 6. Assert child exits with code 0

Production Checklist for Graceful Shutdown and Health Checks in Rust

Getting graceful shutdown and health checks in Rust right requires attention to details that only surface under production load. Before shipping, verify these items against your deployment target. For teams managing observability alongside shutdown logic, the patterns in structured logging best practices complement this work by ensuring shutdown events are captured with proper context.

1. Signal coverage: Handle both SIGTERM (Kubernetes) and SIGINT (local/Docker Ctrl+C). Test both paths.
2. Drain timeout: Set shorter than your orchestrator's terminationGracePeriodSeconds (e.g., 25s drain for 30s grace).
3. Probe separation: Never combine liveness and readiness into a single endpoint.
4. Dependency timeouts: Cap all health check queries at 2-3 seconds to prevent probe pileup.
5. Shutdown-aware readiness: Return 503 immediately after receiving SIGTERM.
6. Background task cancellation: Use CancellationToken or equivalent for all non-HTTP workers.
7. Connection pool cleanup: Verify Drop impls close DB/cache connections; don't rely on OS cleanup.
8. Metrics flush: Ensure Prometheus exporters push final metrics before process exit.

Next Steps

Implementing graceful shutdown and health checks in Rust transforms your service from fragile to resilient, eliminating the most common source of deploy-time errors in cloud-native environments. Start with the signal handler pattern above, add separated probe endpoints, and test locally before deploying to Kubernetes. If you need help auditing your Rust service's production readiness or designing compliant infrastructure for regulated workloads, reach out to discuss your architecture.