Stream CDC data with Amazon Redshift streaming, Amazon MSK and Debezium Connector

Episode 3: Redshift Serverless Streaming Ingestion   Introduction In the previous episodes, I covered the overall architecture design for this project and Debezium connector configuration for our CDC streaming pipeline. Now I’ll complete the series by diving deep into Amazon Redshift Serverless streaming ingestion — the final piece that enables real-time analytics on your CDC data. This episode focuses on the practical […]

Stream CDC data with Amazon Redshift streaming, Amazon MSK and Debezium Connector

Episode 2: Configuring Debezium Connector for Reliable CDC   Introduction In the previous episode, I covered the overall architecture and infrastructure setup for our CDC streaming pipeline. Now I’ll dive deep into Debezium — the open-source platform that captures row-level database changes in real-time and streams them to Kafka topics through MSK Connect. This episode focuses on the […]

Stream CDC data with Amazon Redshift streaming, Amazon MSK and Debezium Connector

Episode 1: Designing the End-to-End CDC Architecture and IaC Setup Introduction In today’s data-driven landscape, organizations need real-time insights from their operational databases to make informed decisions quickly. I developed a comprehensive Change Data Capture (CDC) streaming pipeline that captures database changes from Aurora MySQL and streams them in real-time to Amazon Redshift data warehouse for analytics. This solution […]