Change Data Capture with Debezium

By Khimananda Oli | Last reviewed: August 2026

Implementing Change Data Capture with Debezium solves the latency and coupling problems inherent in dual-write architectures and batch ETL jobs. Instead of polling databases or modifying application code to emit events, Debezium reads transaction logs directly, turning your existing PostgreSQL administration or MySQL infrastructure into a real-time event stream. This approach guarantees ordering and captures deletes, but requires precise configuration to avoid stalling production workloads or exhausting disk space.

How does Change Data Capture with Debezium actually work?

Debezium operates as a source connector within the Kafka Connect framework. Unlike query-based CDC tools that run periodic SELECT statements, Debezium acts as a replication client to the database engine. For PostgreSQL, it uses the logical replication slot interface; for MySQL, it masquerades as a replica reading the binary log. This distinction matters because log-based capture has near-zero impact on write performance and captures every mutation, including deletes and pre-update values, which query-based methods often miss.

The connector maintains its own state via Kafka's internal offset storage. When a connector starts, it reads the last committed offset and resumes from that exact position in the transaction log. If the connector crashes or restarts, no events are lost or duplicated, provided you configure idempotent processing downstream. This guarantee relies heavily on the underlying database's replication protocol being correctly configured.

Logical Replication vs. Query-Based Polling

How do you configure PostgreSQL for Change Data Capture with Debezium?

Before deploying any connector, the source database must be prepared. A common mistake in PostgreSQL replication setups is enabling physical replication but neglecting logical decoding parameters. Debezium requires specific settings in postgresql.conf to expose the WAL in a consumable format.

# postgresql.conf - Required for Debezium CDC
wal_level = logical
max_replication_slots = 4
max_wal_senders = 4
track_commit_timestamp = on  # Optional, useful for audit timing

# Create a dedicated replication user with minimal privileges
CREATE ROLE debezium_user WITH REPLICATION LOGIN PASSWORD 'secure_password';
GRANT SELECT ON ALL TABLES IN SCHEMA public TO debezium_user;

# Create the publication (PG 10+)
CREATE PUBLICATION dbz_publication FOR ALL TABLES;

The wal_level = logical setting instructs PostgreSQL to write additional information to the WAL needed for logical decoding. Without this, the Debezium connector will fail immediately upon startup. The replication slot prevents PostgreSQL from purging WAL segments that the connector hasn't yet processed. Monitor slot lag closely; if the connector stalls, unconsumed WAL can fill your disk and crash the primary database.

Deploying the PostgreSQL Connector

Once the database is configured, deploy the connector via the Kafka Connect REST API. Use the JSON payload below as a baseline, adjusting table includes and topic prefixes to match your environment:

curl -X POST http://kafka-connect:8083/connectors \
  -H "Content-Type: application/json" \
  -d '{
    "name": "inventory-connector",
    "config": {
      "connector.class": "io.debezium.connector.postgresql.PostgresConnector",
      "tasks.max": "1",
      "database.hostname": "postgres-primary",
      "database.port": "5432",
      "database.user": "debezium_user",
      "database.password": "${file:/secrets/db-password.txt:password}",
      "database.dbname": "inventory",
      "topic.prefix": "dbserver1",
      "schema.include.list": "public",
      "table.include.list": "public.orders,public.customers",
      "plugin.name": "pgoutput",
      "slot.name": "debezium_slot",
      "publication.name": "dbz_publication",
      "snapshot.mode": "initial",
      "decimal.handling.mode": "string"
    }
  }'

Note the use of ${file:...} syntax for secrets. Never hardcode credentials in connector configurations, especially in environments subject to SOC 2 or ISO 27001 audits. Externalize secrets using HashiCorp Vault or Kubernetes secrets mounted as files.

How do you set up MySQL binlog streaming with Debezium?

MySQL configuration differs fundamentally from PostgreSQL. Debezium reads the binary log, so the server must be configured as a replication source even if you have no traditional replicas. This setup is distinct from standard MySQL master-slave replication because the connector acts as the slave.

# my.cnf - Required for Debezium MySQL CDC
[mysqld]
server-id         = 1
log_bin           = mysql-bin
binlog_format     = ROW
binlog_row_image  = FULL
expire_logs_days  = 7
gtid_mode         = ON
enforce_gtid_consistency = ON

# Create dedicated user with required privileges
CREATE USER 'debezium'@'%' IDENTIFIED BY 'secure_password';
GRANT SELECT, RELOAD, SHOW DATABASES, REPLICATION SLAVE, REPLICATION CLIENT 
ON *.* TO 'debezium'@'%';

The binlog_row_image = FULL setting is non-negotiable. It ensures both before and after images of changed rows are written to the binlog. Without it, Debezium cannot produce complete change events, breaking consumers that rely on previous state. GTID mode is strongly recommended over file-position tracking because it simplifies failover and recovery when the source database topology changes.

What are the production pitfalls and monitoring strategies for Debezium?

Running Debezium in development is straightforward; running it reliably in production demands attention to failure modes that don't appear in tutorials. After managing CDC pipelines across multiple compliance-regulated environments, these are the issues I encounter most frequently.

Snapshot Locking and Downtime Risk

The default snapshot.mode=initial acquires global locks during the consistent snapshot phase. On large production tables, this can block writes for minutes or hours. In 2026, prefer snapshot.mode=incremental for tables over a few million rows. Incremental snapshots chunk the table by primary key range, releasing locks between chunks and allowing concurrent writes to proceed. Alternatively, use snapshot.mode=no_data if you've already backfilled historical data through another mechanism and only need streaming going forward.

WAL/Binlog Retention and Disk Exhaustion

If the Debezium connector stops consuming (due to Kafka cluster issues, network partitions, or misconfiguration), the source database continues writing transaction logs. PostgreSQL won't recycle WAL segments past the replication slot's confirmed flush position. MySQL won't purge binlogs past the connected replica's position. Set aggressive monitoring alerts on replication slot lag and binlog disk usage. A stalled connector can bring down your primary database within hours on high-write systems.

• Monitor slot lag: Query pg_replication_slots and alert when confirmed_flush_lsn falls behind pg_current_wal_lsn() by more than 1GB.
• Track connector health: Expose Kafka Connect JMX metrics via Prometheus. Key metrics include connected, millis-behind-source, and offset-commit-failed.
• Set retention bounds: Configure max_slot_wal_keep_size (PG 13+) as a safety valve to prevent unbounded WAL growth, accepting potential data loss over database outage.
• Test failover: Regularly simulate connector restarts and verify offset recovery works correctly without duplicates or gaps.

Schema Evolution Handling

Database schema changes propagate through Debezium automatically, but downstream consumers may break if they expect fixed schemas. Enable the Schema Registry integration to enforce compatibility checks. Use AvroConverter or JsonSchemaConverter instead of the default JsonConverter to get schema versioning and evolution support. When altering tables in production, coordinate deployments: update consumers first to handle new fields as optional, then apply the DDL change.

When should you choose Change Data Capture with Debezium over alternatives?

Debezium isn't always the right tool. Understanding its trade-offs prevents costly architectural mistakes. Compare it against common alternatives based on your actual requirements rather than hype.

Choose Debezium when you need real-time, row-level fidelity across multiple database engines and already operate Kafka. Choose managed ELT tools like Fivetran when operational simplicity outweighs latency requirements. Choose application-level domain events when business context matters more than raw data replication. For teams in Nepal or similar regions where cloud-managed CDC options may be limited or cost-prohibitive, self-hosted Debezium on existing infrastructure often provides the best balance of capability and control.

Getting Started with Change Data Capture with Debezium

Start small: pick one non-critical table, deploy the connector in incremental snapshot mode, and validate end-to-end event delivery before expanding scope. Integrate connector health metrics into your existing Prometheus and Grafana monitoring stack from day one. Document your offset management strategy and test recovery procedures quarterly. Change Data Capture with Debezium is powerful infrastructure, but like all distributed systems, it rewards methodical preparation over hasty deployment. If you're planning a CDC implementation and want to avoid common production pitfalls, reach out to discuss your architecture.