Skip to main content
Version: 5.x-preview

Generic PostgreSQL Source Setup for VeloDB CDC

This guide covers the PostgreSQL server configuration required for CDC (Change Data Capture) synchronization with VeloDB using Flink CDC.

Supported Versions​

DatabaseVersions
PostgreSQL10, 11, 12, 13, 14, 15, 16, 17
Amazon Aurora PostgreSQL10.x, 11.x, 12.x, 13.x, 14.x, 15.x, 16.x
Amazon RDS PostgreSQL10.x, 11.x, 12.x, 13.x, 14.x, 15.x, 16.x

Reference: Flink CDC PostgreSQL Connector

Logical Replication Configuration​

Add or modify the following in your PostgreSQL configuration file (postgresql.conf):

# Enable logical replication (required for CDC)
wal_level = logical

# Number of replication slots (at least 1 per CDC connection)
max_replication_slots = 4

# Number of WAL sender processes (at least 1 per CDC connection)
max_wal_senders = 4

Configuration Parameters​

ParameterRequired ValueDescription
wal_levellogicalEnable logical replication (required for CDC)
max_replication_slots≥ 1Number of replication slots (1 per CDC pipeline)
max_wal_senders≥ 1Number of WAL sender processes

Restart PostgreSQL​

After modifying postgresql.conf, restart PostgreSQL:

# SystemD (Ubuntu/Debian/CentOS)
sudo systemctl restart postgresql

# macOS (Homebrew)
brew services restart postgresql@16

# Docker
docker restart <container_name>

Verify Configuration​

SHOW wal_level;
SHOW max_replication_slots;
SHOW max_wal_senders;

Expected result:

    wal_level
-----------------
 logical
(1 row)

 max_replication_slots
-----------------------
 4
(1 row)

 max_wal_senders
-----------------
 4
(1 row)

User Permissions​

Create a dedicated CDC user with replication privileges:

CREATE USER cdc_user WITH REPLICATION LOGIN PASSWORD 'your_password';

-- Grant schema usage
GRANT USAGE ON SCHEMA public TO cdc_user;

-- Grant SELECT on all existing tables
GRANT SELECT ON ALL TABLES IN SCHEMA public TO cdc_user;

-- Grant SELECT on future tables
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO cdc_user;

For PostgreSQL 15+​

PostgreSQL 15+ requires an additional privilege:

GRANT pg_create_subscription TO cdc_user;

Required Privileges​

PrivilegePurpose
REPLICATIONRead WAL (Write-Ahead Log) for CDC
LOGINConnect to database
SELECTRead table data for initial snapshot
pg_create_subscriptionRequired for PostgreSQL 15+

Verify User Permissions​

SELECT usename, userepl FROM pg_user WHERE usename = 'cdc_user';

Expected result:

  usename  | userepl
-----------+---------
 cdc_user  | t
(1 row)

Publication Setup (Optional)​

For Aurora/RDS or production environments, manually create a publication:

-- As superuser (postgres)
CREATE PUBLICATION velodb_publication FOR ALL TABLES;

Verify publication:

SELECT * FROM pg_publication WHERE pubname = 'velodb_publication';

REPLICA IDENTITY (Required for DELETEs)​

To capture full row data for DELETE operations:

ALTER TABLE your_table_name REPLICA IDENTITY FULL;

Note: Without REPLICA IDENTITY FULL, DELETE events only include the primary key.

Connection Test​

Verify the CDC user can:

  1. Connect to PostgreSQL:
psql -h <host> -p 5432 -U cdc_user -d your_database
  1. View tables:
\dt
  1. Select from tables:
SELECT * FROM your_table LIMIT 5;
  1. Check replication status:
SELECT * FROM pg_stat_replication;

Next Steps​

Once PostgreSQL is configured for CDC, follow the Flink CDC Ingestion Guide to set up real-time sync to VeloDB.

Sources​