How would you design a system to capture changes using Change Data Capture (CDC) from a source database?

To capture changes using CDC, first identify the source database and the type of changes needed (insert, update, delete). Choose a CDC method: log-based (most efficient and non-intrusive) or trigger-based (simpler but can add overhead). For log-based CDC, use the database’s transaction log (e.g., MySQL binlog, PostgreSQL WAL, Oracle redo logs). Tools like Debezium, AWS DMS, or Kafka Connect can tail these logs and emit change events. The pipeline typically includes:

1. Source connector: reads the log and produces change events to a message broker (Kafka, Kinesis).
2. Processing layer: optional stream processing (Kafka Streams, Flink) to transform or enrich events.
3. Sink: writes to the target system (data warehouse, NoSQL store, search index).
4. Change tracking: maintain offsets or checkpoints to ensure exactly-once semantics and enable recovery.

Key decisions: choose a broker that matches latency and throughput needs; use schema registry for evolving schemas; secure the pipeline with encryption and IAM roles; monitor lag and error rates. This architecture scales horizontally, provides near real-time replication, and supports downstream analytics or microservices that react to data changes.