Production Logging for Ruby Applications

Khimananda Oli 8 min read Programming and Languages
Production Logging for Ruby Applications

By Khimananda Oli | Last reviewed: August 2026

Debugging a live incident without structured data is like searching for a needle in a haystack while blindfolded. Effective production logging for Ruby applications transforms chaotic text streams into queryable, actionable intelligence that reduces mean time to resolution (MTTR). This guide covers the exact configuration patterns I use to ship secure, compliant, and observable logs from Rails, Puma, and Sidekiq environments.

Ruby ApplicationRails / Puma / SidekiqJSON FormatterSensitive FilterLog ShipperFluent Bit / VectorBuffer & CompressAsync ShippingCentralized StoreLoki / ElasticsearchIndex & RetentionAlerting HooksGrafanaQuery &Visualize
End-to-end architecture for production logging for Ruby applications showing safe async shipping and centralized querying.

How do you configure structured production logging for Ruby applications?

The default Rails logger outputs human-readable text that is nearly impossible to parse reliably at scale. For production logging for Ruby applications, you must switch to a structured JSON formatter. This allows log aggregators to index fields like request_id, user_id, and duration_ms without expensive regex parsing. As outlined in our structured logging best practices, consistency in field naming is critical for building reliable dashboards and alerts.

Configure the Oj or JSON formatter

Add the oj gem to your Gemfile for high-performance JSON serialization. Then configure the logger in config/environments/production.rb:

# config/environments/production.rb
require 'active_support/logger'
require 'oj'

Oj.default_options = { mode: :compat }

config.log_formatter = proc do |severity, datetime, progname, msg|
  payload = {
    timestamp: datetime.iso8601(3),
    level: severity,
    message: msg,
    pid: Process.pid,
    hostname: Socket.gethostname,
    service: ENV.fetch('SERVICE_NAME', 'ruby-app')
  }

  # Merge thread-local context (request_id, user_id)
  payload.merge!(Thread.current[:log_context] || {})

  Oj.dump(payload) + "\n"
end

# Ensure STDOUT is used for containerized environments
config.logger = ActiveSupport::Logger.new(STDOUT)
config.logger.level = Logger::INFO

Inject request context automatically

Logs without correlation IDs are useless during incidents. Use middleware to inject trace and request identifiers into every log line emitted during that request cycle:

# app/middleware/log_context_middleware.rb
class LogContextMiddleware
  def initialize(app)
    @app = app
  end

  def call(env)
    Thread.current[:log_context] = {
      request_id: env['action_dispatch.request_id'],
      trace_id: env['HTTP_X_TRACE_ID'],
      method: env['REQUEST_METHOD'],
      path: env['PATH_INFO']
    }

    status, headers, body = @app.call(env)
    [status, headers, body]
  ensure
    Thread.current[:log_context] = nil
  end
end

# config/application.rb
config.middleware.use LogContextMiddleware

How do you filter sensitive data in Ruby production logs?

A common mistake in production logging for Ruby applications is accidentally persisting PII, tokens, or passwords. Under ISO 27001 and SOC 2 frameworks, this constitutes a control failure. You must implement filtering at the formatter level, not just rely on Rails parameter filtering which only applies to controller params.

Implement a recursive sanitizer

Create a dedicated sanitizer module that scrubs sensitive keys before serialization. This catches nested hashes and arrays that simple parameter filters miss:

# lib/log_sanitizer.rb
module LogSanitizer
  SENSITIVE_KEYS = %w[password token secret api_key authorization credit_card ssn].freeze
  REDACTED = '[REDACTED]'

  def self.sanitize(data)
    case data
    when Hash
      data.each_with_object({}) do |(key, value), result|
        if SENSITIVE_KEYS.any? { |k| key.to_s.downcase.include?(k) }
          result[key] = REDACTED
        else
          result[key] = sanitize(value)
        end
      end
    when Array
      data.map { |item| sanitize(item) }
    when String
      # Redact Bearer tokens and basic auth headers
      data.gsub(/Bearer\s+[A-Za-z0-9\-._~+\/]+=*/, "Bearer #{REDACTED}")
          .gsub(/Basic\s+[A-Za-z0-9+\/=]+/, "Basic #{REDACTED}")
    else
      data
    end
  end
end

Integrate this into your formatter's payload construction step. Never log raw exception backtraces without sanitization; they often contain environment variables or query strings with embedded secrets. For deeper guidance on protecting credentials across your stack, see our article on Kubernetes secrets management done right.

What are the correct log levels for Ruby production environments?

Setting the wrong log level either floods your storage with noise or hides critical failures. In practice, most teams over-log at INFO and under-log at WARN. Understanding the semantic meaning of each level prevents alert fatigue and ensures your SLOs remain meaningful, as discussed in defining meaningful SLIs and SLOs.

LevelSemantic MeaningProduction Use CaseRetention
DEBUGVerbose diagnostic informationNever enabled in prod; dev/staging onlyN/A
INFONormal operational milestonesRequest completion, job success, deploy events30 days
WARNRecoverable issues requiring attentionRetry attempts, deprecated API usage, rate limits90 days
ERRORFailures affecting individual requests/jobsExceptions, validation failures, external API errors1 year
FATALProcess-crashing or system-wide failuresDB connection loss, OOM, unrecoverable stateIndefinite

Avoid logging inside hot paths

Even with asynchronous shipping, excessive INFO logging in tight loops or high-RPS endpoints adds measurable latency. Profile your logging overhead. If an endpoint handles 5,000 RPS, five INFO logs per request means 25,000 log writes per second. Sample verbose logs or move them to DEBUG behind a feature flag.

How do you handle logging in Puma and Sidekiq workers?

Rails configuration alone does not cover your entire runtime. Puma and Sidekiq have their own logging pipelines that bypass config.log_formatter unless explicitly configured. Missing these creates blind spots exactly where failures occur most frequently.

Puma structured logging

In config/puma.rb, redirect Puma's internal logs through your Rails logger so they share the same JSON format and context:

# config/puma.rb
stdout_redirect '/dev/stdout', '/dev/stderr', true

on_worker_boot do
  ActiveRecord::Base.establish_connection
  # Re-initialize logger after fork to avoid shared file descriptors
  Rails.logger.reopen(STDOUT)
end

# Custom log formatter for Puma-specific events
log_formatter do |str|
  Oj.dump({
    timestamp: Time.now.iso8601(3),
    level: 'INFO',
    source: 'puma',
    message: str.strip,
    pid: Process.pid
  }) + "\n"
end

Sidekiq job context propagation

Sidekiq jobs run outside the HTTP request cycle, so they lack request_id. Inject job metadata via server middleware:

# config/initializers/sidekiq.rb
Sidekiq.configure_server do |config|
  config.server_middleware do |chain|
    chain.add SidekiqLogContextMiddleware
  end
end

class SidekiqLogContextMiddleware
  def call(worker, job, queue)
    Thread.current[:log_context] = {
      job_id: job['jid'],
      job_class: worker.class.name,
      queue: queue,
      args_hash: LogSanitizer.sanitize(job['args'])
    }
    yield
  ensure
    Thread.current[:log_context] = nil
  end
end
HTTP RequestMiddlewareControllerService LayerSet Thread ContextCall ActionExecute Business LogicLogger.info(JSON)Includes request_id + trace_idClear Thread Context (ensure block)
Log context propagation sequence ensuring every Ruby application log line carries request and trace correlation IDs.

How do you ship Ruby logs without blocking application threads?

Writing logs synchronously to disk or network introduces tail latency. In high-throughput Ruby applications, this can add 5–20ms per request. The solution is decoupling: write to stdout or a local buffer, then let a dedicated shipper handle transport. This pattern also provides resilience against network partitions and backend outages.

Choose the right shipper

  • Fluent Bit: Lightweight C-based agent, ideal for Kubernetes sidecars and resource-constrained VPS deployments. Low memory footprint (~10MB).
  • Vector: Rust-based, excellent for complex transformations and multi-sink routing. Better performance than Fluentd for Ruby JSON logs.
  • Fluentd: Ruby-based, rich plugin ecosystem but higher memory usage. Good choice if your team already maintains custom Ruby plugins.

For detailed comparisons of these tools, read our breakdown of Fluentd vs Fluent Bit for log shipping. Regardless of choice, configure local buffering with disk fallback to prevent log loss during network blips.

Container and systemd integration

In Docker/Kubernetes, always log to stdout/stderr. Let the container runtime capture output and forward it to the node-level shipper via journald or direct file tailing. Never mount volumes for application logs in production; this couples pod lifecycle to storage availability and complicates scaling. For systemd-managed bare-metal deployments, configure StandardOutput=journal and have your shipper read from the journal API rather than tailing files.

Synchronous (Anti-pattern)Asynchronous (Recommended)Ruby App ThreadNetwork WriteWait ACKResume RequestBLOCKED 5-20msRuby App ThreadSTDOUT WriteImmediate ReturnShipper AgentBuffer + ShipCentralized Store<1ms LATENCY
Latency comparison demonstrating why asynchronous shipping is essential for production logging for Ruby applications.

Implementing Audit-Ready Production Logging for Ruby Applications

Effective production logging for Ruby applications is not just about debugging—it is an audit artifact. Ensure immutable storage, tamper-evident hashing, and access controls on your log backend. Define retention tiers aligned with your compliance obligations before you need them during an audit. Start by converting one service to structured JSON this week, validate the output in your aggregator, then roll out the pattern across your fleet. If you need help designing a compliant observability stack or auditing your current logging posture, reach out to discuss your infrastructure.

Frequently Asked Questions

Semantic Logger remains the top choice for structured production logging in Ruby. It supports multiple backends, offers low overhead via background threads, and integrates natively with Rails 8. Use it over stdlib Logger for high-throughput services requiring JSON output and context propagation.

Set config.logger to ActiveSupport::Logger.new wrapped by SemanticLogger::Appender::File with formatter set to :json. Ensure OTEL_RESOURCE_ATTRIBUTES are injected for trace correlation. This enables log aggregators like Datadog or Grafana Loki to parse fields automatically without custom grok patterns.

Always log to stdout in Kubernetes or ECS environments. Container runtimes capture stdout and forward to centralized systems. File logging adds I/O latency and risks data loss during pod restarts. Reserve file appenders only for local debugging or legacy bare-metal setups lacking log shippers.

Use INFO as baseline. DEBUG generates excessive volume and cost. WARN captures recoverable issues; ERROR flags user-impacting failures. Adjust dynamically via environment variables or admin endpoints without redeploying. Avoid TRACE unless actively diagnosing performance bottlenecks in staging mirrors.

Sample verbose logs at ingestion using Fluent Bit filters before shipping. Retain ERROR and WARN indefinitely; archive INFO after 30 days. Exclude health checks and static asset requests via route-level silencing. Structured tagging enables tiered storage policies in S3 or GCS lifecycle rules.

Yes. Inject trace_id and span_id into every log entry using OpenTelemetry Ruby SDK. Configure Semantic Logger to read from Otel.current_span. This links logs to traces in Jaeger or Tempo, enabling root cause analysis across microservices without manual ID stitching.

Use ActiveSupport::ParameterFilter or custom redactors in Semantic Logger to mask PII, tokens, and passwords before serialization. Never log raw request bodies or headers. Audit log schemas quarterly. Enable field-level encryption in your aggregator if compliance requires encrypted-at-rest sensitive attributes.

Synchronous logging blocks request threads during I/O spikes. Switch to async appenders with bounded queues. Monitor queue saturation metrics; drop oldest entries rather than blocking when full. Increase buffer size or add backpressure signals to upstream services to prevent silent data loss.

Use TestLogger gem to assert log messages, levels, and structured fields in RSpec. Mock external appenders to avoid network calls. Validate JSON schema compliance with json-schema matcher. Include negative tests ensuring redaction works and debug logs are suppressed in test environment configs.

Async Semantic Logger adds less than 2ms p99 latency at 10k RPS on modern hardware. Synchronous JSON formatting can add 15-30ms. Profile with benchmark-ips comparing nil logger vs structured appender. Offload serialization to background threads and reuse frozen format strings to minimize GC pressure.

Use logrotate with copytruncate for file-based logging to avoid reopening handles. For stdout, rely on container runtime rotation. Never implement manual rotation in application code. Signal USR1 to gracefully reopen files if using custom daemons. Validate rotation doesn’t corrupt partial JSON lines during writes.

Vector outperforms Fluentd for Ruby JSON logs due to lower memory footprint and native parsing. Ship directly to ClickHouse or Loki for cost efficiency. Avoid ELK unless you need full-text search. Ensure aggregator preserves original timestamp precision and nested field structures from Semantic Logger output.

Configure per-gem log levels via SemanticLogger::Base.logger_for('GemName').level = :warn. Override verbose gems like Net::HTTP or ActiveRecord at boot time. Use middleware to filter specific routes or controllers. Document silenced components in your runbook to prevent confusion during incident response.

Always include timestamp, severity, service name, version, host, pid, trace_id, and user_id when authenticated. Add request_id for HTTP contexts. Embed deployment SHA and region for multi-environment correlation. Standardize field names across services to enable unified dashboards and alerting queries.

Run dual logging temporarily: keep existing text logger while adding structured appender to separate destination. Validate parsing and field completeness in staging first. Gradually shift traffic monitoring to new logs. Decommission old format only after confirming all alerts and dashboards function correctly with structured data.