Observability for Elixir with OpenTelemetry

Khimananda Oli 8 min read Programming and Languages
Observability for Elixir with OpenTelemetry

By Khimananda Oli | Last reviewed: August 2026

Elixir applications running on the BEAM VM present unique challenges for monitoring because traditional agent-based instrumentation often fails to capture lightweight process spawning and message passing. Implementing observability for Elixir with OpenTelemetry requires leveraging native Erlang/OTP telemetry events rather than external bytecode manipulation. This approach ensures you capture distributed traces, runtime metrics, and structured logs with minimal latency impact. For teams building concurrent systems, understanding this native integration is the difference between blind spots and complete system visibility. If you are new to the broader ecosystem, start by reviewing OpenTelemetry as the observability standard to understand the vendor-neutral data model before configuring Elixir-specific exporters.

BEAM VM / Elixir AppPhoenix Telemetry EventsEcto Query SpansLogger MetadataOTel SDK & ProcessorsBatch Span ProcessorContext PropagationResource DetectorsObservability BackendTempo / Jaeger (Traces)Prometheus (Metrics)Loki / Elastic (Logs)
Native observability for Elixir with OpenTelemetry flows from BEAM telemetry events through the SDK batch processor to your chosen observability backend via OTLP.

How do you configure observability for Elixir with OpenTelemetry in a Phoenix app?

Setting up observability for Elixir with OpenTelemetry starts with dependency management and supervision tree configuration. Unlike Java or Python where agents might attach externally, Elixir requires explicit inclusion of the OTel SDK as part of your application release. This ensures the instrumentation runs within the same BEAM scheduler threads, avoiding cross-process serialization costs.

Add core dependencies to mix.exs

You need the base SDK plus specific instrumentation libraries for each framework component. In 2026, the ecosystem has stabilized around version 1.x for all core packages. Add these to your deps function:

def deps do
  [
    {:opentelemetry, "~> 1.4"},
    {:opentelemetry_api, "~> 1.3"},
    {:opentelemetry_exporter, "~> 1.7"},
    {:opentelemetry_phoenix, "~> 2.0"},
    {:opentelemetry_ecto, "~> 1.2"},
    {:opentelemetry_reqwest, "~> 0.3"}
  ]
end

Configure the SDK in config/runtime.exs

Production configuration should always live in config/runtime.exs to read environment variables at boot time rather than compile time. This is critical for containerized deployments where secrets and endpoints change per environment.

config :opentelemetry,
  resource: %{
    service: %{
      name: System.get_env("OTEL_SERVICE_NAME", "my-elixir-app"),
      version: Application.spec(:my_app, :vsn),
      namespace: System.get_env("DEPLOY_ENV", "production")
    }
  },
  processors: [
    {:otel_batch_processor, %{
      exporter: {:opentelemetry_exporter, %{}},
      max_queue_size: 2048,
      schedule_delay_ms: 5000
    }}
  ]

config :opentelemetry_exporter,
  otlp_protocol: :http_protobuf,
  otlp_endpoint: System.get_env("OTEL_EXPORTER_OTLP_ENDPOINT", "http://localhost:4318"),
  otlp_headers: [
    {"Authorization", "Bearer #{System.get_env("OTEL_AUTH_TOKEN")}"}
  ]

Attach instrumenters in application.ex

The final step is attaching the auto-instrumentation handlers during application startup. This must happen before your endpoint or repo supervisors start so no early requests are missed.

def start(_type, _args) do
  :opentelemetry_cowboy.setup()
  OpentelemetryPhoenix.setup(adapter: :cowboy)
  OpentelemetryEcto.setup([:my_app, :repo])

  children = [
    MyApp.Repo,
    {Phoenix.PubSub, name: MyApp.PubSub},
    MyAppWeb.Endpoint
  ]

  opts = [strategy: :one_for_one, name: MyApp.Supervisor]
  Supervisor.start_link(children, opts)
end

What telemetry signals does the BEAM VM expose natively?

The BEAM virtual machine emits rich telemetry that generic APM tools cannot access. When implementing observability for Elixir with OpenTelemetry, you gain visibility into three distinct signal types that map directly to the OTP runtime behavior. Understanding these signals helps you distinguish between application-level slowness and VM-level resource exhaustion.

  • Distributed Traces: Automatic span creation for Phoenix router dispatch, controller actions, template rendering, and Ecto queries. Context propagates across GenServer calls and Task.async boundaries when using the patched spawn functions.
  • VM Metrics: Scheduler utilization, run queue lengths, memory allocator stats, garbage collection counts, and port/process limits. These are exposed as OTLP metrics at configurable intervals.
  • Structured Logs: Logger metadata automatically enriched with trace_id and span_id attributes, enabling log-trace correlation in backends like Grafana Loki without manual tagging.

A common mistake is treating BEAM metrics like OS metrics. High CPU usage in a BEAM node doesn't necessarily indicate overload; it could mean efficient parallel processing across schedulers. Instead, monitor scheduler wall time and run queue length as primary health indicators. For deeper context on signal selection, see metrics, logs, and traces compared.

HTTP RequestGenServerTask.AsyncEcto RepoRouter Dispatchcall + ctxhandle_callasync + ctxTask Executionquery + ctxDB Query SpanResponse Render
Trace context propagates automatically through GenServer calls and Task.async when using OpenTelemetry-instrumented spawn primitives in Elixir.

How do you add custom spans and attributes in Elixir?

Auto-instrumentation covers framework boundaries, but business logic inside pure functions or complex pipelines requires manual instrumentation. The OpenTelemetry API for Elixir uses macros that compile to no-ops when the SDK is disabled, ensuring zero cost in test environments.

Create spans with semantic attributes

Use the OpenTelemetry.Tracer module to wrap critical code paths. Always include semantic conventions for attribute names to ensure backend compatibility.

require OpenTelemetry.Tracer

def process_order(order_id, user_id) do
  OpenTelemetry.Tracer.with_span "order.processing" do
    OpenTelemetry.Tracer.set_attributes(%{
      "order.id" => order_id,
      "user.id" => user_id,
      "order.items_count" => length(order.items)
    })

    case validate_inventory(order) do
      {:ok, inventory} ->
        OpenTelemetry.Tracer.add_event("inventory.validated", %{
          "warehouse" => inventory.location
        })
        fulfill_order(order, inventory)

      {:error, reason} ->
        OpenTelemetry.Tracer.set_status(:error, to_string(reason))
        {:error, reason}
    end
  end
end

Propagate context across uninstrumented boundaries

When spawning processes manually (not via Task or supervised pools), you must explicitly pass the trace context. Failing to do so creates orphaned spans that break the distributed trace. Use :otel_ctx.get_ctx/0 and :otel_ctx.attach/1 for raw process spawns.

Which OpenTelemetry exporter should you use for Elixir in production?

Choosing the right exporter affects both operational complexity and data fidelity. While the OTLP protocol is standardized, transport mechanisms and batching behaviors differ significantly under high-throughput BEAM workloads.

ExporterProtocolBest ForTrade-offs
opentelemetry_exporterOTLP HTTP/gRPCGrafana Tempo, Datadog, HoneycombStandard choice; HTTP protobuf preferred over gRPC for BEAM due to connection pooling maturity
opentelemetry_zipkinZipkin JSON v2Legacy Zipkin installationsLoses metric/log signals; only traces supported; deprecated for new deployments
Console ExporterSTDOUTLocal development & debuggingNever use in production; blocks schedulers and pollutes logs
Custom ProcessorUser-definedSOC 2 audit trails, PII redactionRequires maintaining custom code; useful for compliance-filtered exports

In practice, I recommend OTLP HTTP protobuf for 95% of Elixir deployments. The BEAM's HTTP client handles connection reuse better than the current gRPC implementation, especially under bursty traffic patterns common in Phoenix LiveView applications. For teams needing guidance on backend selection, Tempo distributed tracing with Grafana provides an excellent open-source pairing with Elixir's OTLP output.

How do you correlate logs and traces in Elixir applications?

Log-trace correlation is where observability for Elixir with OpenTelemetry delivers immediate debugging value. Without it, you have isolated signals; with it, you can jump from a slow trace span directly to the relevant log lines. The Elixir Logger integrates with OTel through metadata injection.

Configure Logger backend for trace context

Add the OpenTelemetry logger handler to your logging pipeline. This automatically injects trace_id and span_id into every log entry emitted within an active span context.

# config/config.exs
config :logger, :default_handler,
  formatter: {LoggerJSON.Formatters.Basic, %{
    metadata: [:trace_id, :span_id, :request_id],
    level: :info
  }}

# Ensure OTel logger bridge is started
config :opentelemetry,
  logger_bridge: true

Verify correlation in your backend

After deployment, trigger a request and navigate to the trace view in your backend. Click on any span and verify that associated logs appear in the linked panel. If logs show "trace_id": null, check that the logger bridge is enabled and that your formatter includes the metadata keys. For structured logging patterns beyond OTel correlation, review structured logging best practices.

Native OpenTelemetry (Recommended)Zero External Agent OverheadRuns inside BEAM schedulers, no NIF latencyFull BEAM VisibilityScheduler queues, GC, message passing capturedVendor Neutral OTLP ExportSwitch backends without code changesCompile-Time SafetyMacros become no-ops when SDK disabledCommunity Maintained InstrumentationPhoenix, Ecto, Req, Broadway adaptersLegacy APM Agents (Avoid)NIF-Based InstrumentationCross-boundary calls add scheduler pressureLimited BEAM AwarenessMisses process spawning and mailbox growthVendor Lock-InProprietary protocols require agent updatesRuntime Overhead Always OnCannot disable in test/dev environments cleanlyDeprecated for Elixir/ErlangMost vendors sunsetted BEAM agent support by 2025
Native observability for Elixir with OpenTelemetry outperforms legacy APM agents in overhead, BEAM visibility, and long-term maintainability.

Implementing Production-Grade Observability for Elixir with OpenTelemetry

Shipping observability for Elixir with OpenTelemetry to production requires more than correct configuration. You must validate data flow, set appropriate sampling rates, and establish alerting baselines before traffic hits real users. Start by deploying to staging with 100% sampling to verify span completeness, then reduce to head-based sampling (typically 10–20%) for production to control backend costs. Monitor the otel_batch_processor queue size metric; sustained growth indicates your exporter cannot keep up with emission rate and you need to increase batch intervals or scale your collector tier. Finally, define SLOs around trace coverage itself — if less than 99% of requests produce complete traces, treat it as a reliability incident. For teams ready to implement this stack or audit existing instrumentation, reach out to discuss your Elixir observability architecture.

Frequently Asked Questions

Add opentelemetry, opentelemetry_api, and opentelemetry_exporter to your mix.exs dependencies. Run mix deps.get, then configure the exporter endpoint and service name in config/runtime.exs for environment-specific observability settings.

Yes, add opentelemetry_phoenix to automatically trace HTTP requests, controllers, and views. This library captures request metadata, response status codes, and timing without manual span creation in your router or controller code.

Typically under two percent CPU overhead in production with sampling enabled. Use probabilistic sampling at ten percent for high-traffic services to reduce export volume while maintaining statistical accuracy for latency analysis.

Yes, OpenTelemetry propagates W3C trace context headers automatically between services. Ensure all Elixir nodes share compatible opentelemetry versions and configure consistent resource attributes like service.name for unified trace visualization in Jaeger or Grafana Tempo.

Configure opentelemetry_exporter with OTLP protocol pointing to your Tempo endpoint. Set OTEL_EXPORTER_OTLP_ENDPOINT environment variable and enable gzip compression to reduce network bandwidth during high-throughput trace ingestion.

OpenTelemetry offers vendor neutrality and lower long-term costs but requires self-hosted infrastructure. AppSignal provides managed Elixir-specific insights with less operational overhead, making it preferable for teams prioritizing developer experience over customization.

Use OpenTelemetry.Tracer.set_attribute inside traced functions to attach business context like user_id or order_total. These attributes become searchable fields in your backend, enabling filtered queries and metric aggregation by dimension.

Verify the exporter endpoint URL, authentication headers, and firewall rules allowing outbound traffic on port 4318. Check application logs for opentelemetry_exporter errors and confirm the service is not dropping spans due to misconfigured sampling rates.

Yes, use opentelemetry_ecto for database calls and manually instrument GenServers with OpenTelemetry.Tracer.with_span. Wrap handle_call and handle_cast callbacks to capture message processing duration and internal state transitions as child spans.

Always use TLS encryption for OTLP exporters in production. Avoid recording PII in span attributes; use redaction processors or attribute filters to strip sensitive fields before export to prevent accidental data exposure in observability backends.

Yes, implement a custom sampler using the ParentBasedSampler with TraceIdRatioBased fallback. Configure it to always record spans with error status while sampling successful requests at a lower rate to optimize storage costs.

The opentelemetry-erlang ecosystem supports Elixir 1.14 and later with OTP 25+. Always check the hex.pm compatibility matrix before upgrading, as breaking changes occur between major releases of the core telemetry packages.

Instrument LiveView mounts and event handlers using opentelemetry_liveview. This captures render times, socket latency, and assign updates as discrete spans, helping identify slow components and optimize real-time user interaction responsiveness.

Yes, always include both.

Set otel_traces_exporter to none in config/test.exs.