Production Logging for Go Applications

Khimananda Oli 9 min read Programming and Languages
Production Logging for Go Applications

By Khimananda Oli | Last reviewed: August 2026

Implementing effective production logging for Go applications is the difference between resolving an incident in minutes or spending hours guessing why a service failed. While Go’s standard library provides basic output, production environments demand structured, machine-parseable logs that integrate with observability platforms without introducing latency. This guide covers the architectural patterns, library choices, and configuration strategies necessary to build reliable, high-performance logging subsystems in modern Go services.

How do you implement structured production logging for Go applications?

Structured logging is non-negotiable for structured logging best practices in any serious backend system. Unstructured text logs are human-readable but machine-hostile; they require expensive regex parsing and break whenever a developer changes a message format. Structured logs, typically JSON, treat log entries as data records with consistent fields, enabling instant filtering and aggregation in tools like Grafana Loki or Elasticsearch.

Go Applicationslog / zerologJSON EncoderAsync BufferNon-blocking WriteBatch FlushLog AggregatorLoki / ELKIndex & QueryFigure 1: Structured logging pipeline preventing I/O blocking in production Go services
Structured production logging for Go applications flows through async buffers to prevent request latency

Since Go 1.21, the standard library includes log/slog, which provides a solid foundation for structured logging without external dependencies. For many teams, this is now the default choice. However, if your service handles tens of thousands of requests per second and profiling shows logging as a bottleneck, libraries like Zerolog or Zap offer zero-allocation JSON encoding that significantly reduces GC pressure.

Configuring slog for production

The key to production readiness is configuring the handler correctly at startup. Never use the default text handler in production; always opt for JSON. Set the level based on environment variables to allow dynamic tuning without redeployment.

package main

import (
    "log/slog"
    "os"
)

func initLogger() {
    opts := &slog.HandlerOptions{
        Level: slog.LevelInfo, // Configure via env var in real apps
        AddSource: false,      // Disable in prod for performance
    }
    
    handler := slog.NewJSONHandler(os.Stdout, opts)
    logger := slog.New(handler).With(
        "service", "payment-api",
        "version", "v2.4.1",
    )
    
    slog.SetDefault(logger)
}

This configuration ensures every log line contains consistent metadata. The With method creates a child logger with pre-bound fields, eliminating repetitive key-value pairs throughout your codebase. In practice, I recommend creating a logger factory function that reads configuration from environment variables and returns a configured instance, rather than relying on global state.

Which Go logging library performs best under high load?

Library selection depends entirely on your throughput requirements and operational constraints. While slog is sufficient for 90% of applications, benchmark-critical paths demand specialized tools. Understanding these trade-offs prevents premature optimization while ensuring you don't hit a wall during peak traffic.

LibraryAllocationThroughputAPI StyleBest For
log/slogModerateGoodStandard, extensibleMost services, simplicity
ZerologZeroExcellentChained, fluentHigh-throughput APIs
ZapNear-zeroExcellentSugared/TypedEnterprise systems
LogrusHighFairTraditionalLegacy maintenance only

In my experience managing high-traffic fintech services, Zerolog consistently wins raw benchmarks because it writes directly to an io.Writer without intermediate map allocations. However, slog's integration with the broader ecosystem—including native support in database drivers and HTTP middleware—often outweighs marginal performance gains for typical workloads. Always profile before switching; intuition about logging performance is frequently wrong.

Benchmarking your specific workload

Don't trust generic benchmarks. Your log structure, field count, and output destination dramatically affect real-world performance. Create a representative benchmark using your actual log schema:

func BenchmarkLogging(b *testing.B) {
    logger := slog.New(slog.NewJSONHandler(io.Discard, nil))
    b.ResetTimer()
    
    for i := 0; i < b.N; i++ {
        logger.Info("request completed",
            "method", "POST",
            "path", "/api/v1/payments",
            "status", 200,
            "duration_ms", 45,
            "user_id", "usr_abc123",
        )
    }
}

Run this with -benchmem to see allocation counts. If you're seeing hundreds of allocations per operation and handling massive scale, consider Zerolog. Otherwise, stick with the standard library for better long-term maintainability.

How do you correlate logs with distributed traces in Go?

Logs without context are noise. In microservices architectures, a single user request touches multiple services, and you need to reconstruct the entire journey during debugging. This requires propagating trace and span IDs through context and injecting them into every log entry automatically. This correlation is fundamental to OpenTelemetry observability standards.

HTTP HandlerExtract TraceIDService Layerctx.With(traceID)Database CallAuto-inject IDResulting Log Entry{"level":"info","msg":"query executed","trace_id":"abc123","span_id":"def456","duration_ms":12,"rows":1}Figure 2: Trace context propagation enables end-to-end request reconstruction
Correlating production logging for Go applications with distributed traces via context injection

Automatic context extraction middleware

Manually extracting trace IDs in every handler is error-prone. Use middleware to inject a pre-configured logger into the context, ensuring all downstream code automatically includes correlation fields. With OpenTelemetry and slog, this becomes straightforward:

func LoggingMiddleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        ctx := r.Context()
        
        // Extract trace/span from OTel context
        spanCtx := trace.SpanContextFromContext(ctx)
        attrs := []any{
            "http.method", r.Method,
            "http.path", r.URL.Path,
        }
        
        if spanCtx.HasTraceID() {
            attrs = append(attrs, 
                "trace_id", spanCtx.TraceID().String(),
                "span_id", spanCtx.SpanID().String(),
            )
        }
        
        // Create request-scoped logger
        reqLogger := slog.Default().With(attrs...)
        ctx = ContextWithLogger(ctx, reqLogger)
        
        next.ServeHTTP(w, r.WithContext(ctx))
    })
}

// Helper to retrieve logger from context
func Logger(ctx context.Context) *slog.Logger {
    if l, ok := ctx.Value(loggerKey{}).(*slog.Logger); ok {
        return l
    }
    return slog.Default()
}

This pattern ensures that even deeply nested function calls produce correlated logs without explicit parameter passing. When investigating incidents, you can filter by trace_id and see the complete request flow across all services. This is especially critical when integrating with OpenTelemetry instrumentation for comprehensive observability.

What security and compliance considerations apply to Go logging?

Logging is a frequent source of data breaches and compliance failures. I've audited systems where credit card numbers, JWT tokens, and PII were written to plaintext log files stored indefinitely. For teams pursuing SOC 2 or ISO 27001 certification, logging controls are explicitly evaluated. Treat log data with the same rigor as database records.

  • Never log secrets: Implement field-level redaction for sensitive keys. Use allowlists rather than denylists—it's safer to accidentally omit a field than to leak credentials.
  • Sanitize PII: Hash or mask email addresses, phone numbers, and national IDs unless business justification exists and encryption-at-rest is guaranteed.
  • Control verbosity dynamically: Debug logging in production should be toggleable without restarts. Excessive logging creates both security exposure and cost overruns.
  • Retention policies: Define and enforce retention periods aligned with regulatory requirements. Automate deletion; manual cleanup never happens reliably.
  • Access controls: Restrict who can view production logs. Log aggregation platforms must support RBAC and audit trails for log access itself.

Implementing safe logging wrappers

Create wrapper functions that enforce sanitization rules at the API boundary. This centralizes security logic and prevents individual developers from accidentally leaking data:

func SafeLogPayment(ctx context.Context, payment Payment) {
    Logger(ctx).Info("payment processed",
        "payment_id", payment.ID,
        "amount", payment.Amount,
        "currency", payment.Currency,
        // NEVER log: card_number, cvv, full_name
        "card_last4", lastFour(payment.CardNumber),
        "user_id", hashIfPII(payment.UserID),
    )
}

func lastFour(s string) string {
    if len(s) < 4 {
        return "**"
    }
    return "**" + s[len(s)-4:]
}

This approach makes unsafe logging structurally difficult. Code reviews should flag direct logger calls in sensitive domains and require wrapper usage instead. For deeper guidance on securing data pipelines, see resources on protecting PII and secrets which apply equally to traditional logging.

How do you optimize logging performance without losing visibility?

Performance and observability often conflict. Synchronous logging adds latency to every request; aggressive sampling risks missing rare errors. The solution is intelligent buffering and tiered verbosity that adapts to system health.

Log Event GeneratedCheck Level & SamplingError/Critical?Always Write SyncYESImmediate FlushNOAsync BufferBatch WriterFlush on Size/TimeFigure 3: Tiered logging strategy preserves errors while optimizing happy-path throughput
Optimizing production logging for Go applications with conditional sync/async write paths

Asynchronous buffering strategies

For info and debug logs, synchronous writes add unnecessary latency. Use a buffered writer that batches entries and flushes periodically or when capacity is reached. Libraries like Zap provide this natively; with slog, you can wrap the handler:

// Conceptual async handler wrapper
type AsyncHandler struct {
    inner   slog.Handler
    buffer  chan []byte
    done    chan struct{}
}

func (h *AsyncHandler) Handle(ctx context.Context, r slog.Record) error {
    // Serialize to buffer channel non-blockingly
    // Background goroutine batches and writes to inner handler
    // On shutdown, drain buffer to prevent data loss
    select {
    case h.buffer <- serialize(r):
        return nil
    default:
        // Buffer full: drop or fallback to sync
        metrics.Increment("logs.dropped")
        return nil
    }
}

Critical caveat: async logging can lose entries during crashes or graceful shutdowns. Always implement signal handling that drains the buffer before exit. For error-level logs, bypass the buffer entirely—data integrity matters more than latency for failures.

Dynamic sampling for high-cardinality events

Some endpoints generate enormous log volume during normal operation. Rather than disabling logging entirely, implement probabilistic sampling that increases to 100% when errors occur. This preserves debugging capability while controlling costs. Many aggregators support server-side sampling, but client-side reduction saves network bandwidth and serialization CPU.

Production Logging for Go Applications: Next Steps

Effective production logging for Go applications combines structured output, contextual correlation, security-aware field selection, and performance-conscious architecture. Start with slog and JSON formatting, add trace propagation early, and only optimize when profiling proves necessity. Remember that logs are a liability as much as an asset—every field you emit carries storage cost, security risk, and cognitive overhead.

If your team needs help designing observable, compliant Go services or auditing existing logging infrastructure for SOC 2 readiness, reach out to discuss your specific challenges. Getting logging right from the start prevents costly rework and ensures your team can actually debug production issues when they matter most.

Frequently Asked Questions

slog is the standard choice since Go 1.21. It provides structured logging natively without external dependencies, supports JSON output, and integrates with context propagation for tracing in production environments.

Use slog.NewJSONHandler with os.Stdout and set level to Info or Warn. Wrap it in a handler that adds service metadata like version and hostname for consistent log parsing.

Always log to stdout in Kubernetes. Container orchestrators capture standard streams automatically, enabling centralized collection via Fluent Bit or Vector without managing file rotation or permissions inside pods.

Store request IDs in context using middleware, then retrieve them via slog.LogAttrs or custom handlers. This ensures every log entry within a request lifecycle includes the trace identifier.

Default to Info level. Reserve Debug for local development only. Use Warn for recoverable errors and Error for failures requiring alerts. Avoid logging sensitive data at any level.

Never log raw user input, tokens, or PII. Use redaction middleware or custom slog.Attr transformers to mask fields before serialization. Audit log schemas regularly against compliance requirements like GDPR or SOC2.

Yes. Extract trace and span IDs from OpenTelemetry context and inject them into slog records via a custom handler. This links logs to specific spans in Jaeger or Tempo dashboards.

Use buffered writers or async log exporters. Libraries like zapcore.BufferedWriteSyncer batch writes to reduce syscall overhead while maintaining ordering guarantees within each goroutine.

Minimal when using slog with precomputed attributes. Avoid string formatting in hot paths. Benchmarks show under 500ns per log call with JSON handlers on modern hardware in 2026.

Use lumberjack.v2 as a slog writer backend. It handles size-based rotation, compression, and retention policies transparently without stopping the application or losing log entries during transitions.

Ensure your slog handler emits RFC3339Nano timestamps. Some cloud parsers fail on Unix epochs or malformed formats. Verify the time field name matches your ingestion pipeline schema exactly.

Replace the default logger with a test handler that captures records. Assert on message content, level, and attributes using slogtest package utilities introduced in recent Go versions.

Usually multiple handlers attached to the same logger instance or misconfigured sidecar collectors. Audit handler registration code and verify only one output destination exists per process boundary.

Add middleware that suppresses logs for /healthz or /readyz endpoints. Alternatively, use slog.HandlerOptions.Level to dynamically adjust verbosity based on URL path or request metadata.

Only if you need zero-allocation performance below slog’s baseline. For most teams, slog’s stdlib integration, ecosystem support, and maintainability outweigh marginal speed gains from third-party libraries.