Ingest CDC Data using PostgreSQL, Kafka, and Debezium

This recipe downloads Postgres change data capture (CDC) about dvdrentals and connects Kafka to Debezium. CDC data is written to Kafka, and then consumed by Apache Pinot.

To understand how this recipe processes data, examine the commands executed in the Makefile.

​

Prerequisites

To follow the code examples in this guide, you must install Docker locally and download recipes.

​

Navigate to recipe

1. If you haven’t already, download recipes.
2. In the terminal, navigate to this recipe’s directory:

cd pinot-recipes/recipes/debezium-cdc

1. Spin up a Pinot cluster using the Makefile, which uses Docker compose:

make recipe

1. Navigate to localhost:9000/#/query to see the data in Apache Pinot.

​

Clean up

make clean

​

Troubleshooting

To clean up old Docker installations that may be interfering with your testing of this recipe, run the following command:

docker system prune