Import
Data Integration provides a managed ingestion workflow for bringing external data into a VeloDB Cloud warehouse. Use it when you want VeloDB Cloud to create and operate ingestion jobs through a visual wizard.
Import is suitable for the following scenarios:
- Loading existing data from a source system into VeloDB Cloud.
- Continuously ingesting changes from MySQL or PostgreSQL through CDC.
- Consuming records from Kafka-compatible event streams.
- Loading files from object storage on a schedule or as a batch job.
If you want to query external data in place without moving it into VeloDB Cloud, use Add Catalog instead. If you are migrating databases and tables from Apache Doris or VeloDB Enterprise, use Migration.
To create your first job, open Data Integration and click Add Import Job. If the Import Jobs page already contains jobs, in the upper-right corner, click Add Data > Import data into VeloDB. To view existing jobs, open Import Jobs.
Choose your import workflow
- Connect for Import: prepare network access from your warehouse to the source. For RDS or Aurora connections through an AWS Network Load Balancer with IP targets, also configure IP target synchronization.
- Choose a source below, complete its prerequisites, and create the import job. The MySQL guides include source setup for Amazon RDS and Aurora. The Kafka guides include Amazon MSK setup.
- Open Import Jobs in Data Integration to review the job's status and activity.
Choose a data source
| Source | Use case | Guide |
|---|---|---|
| MySQL | Initial load plus incremental CDC, or incremental-only ingestion | MySQL |
| PostgreSQL | Initial load plus incremental CDC, or incremental-only ingestion | PostgreSQL |
| Kafka-compatible streams | Continuous ingestion from Kafka topics, including Apache Kafka, Confluent Cloud, and Amazon MSK | Kafka |
| S3-compatible object storage | Load files into VeloDB tables | Object Storage |
Before you begin
Before creating an import job, make sure you have:
- A running VeloDB Cloud warehouse.
- Source connection information, such as host, port, database, username, password, access key, bucket path, or Kafka bootstrap servers.
- Network access from VeloDB Cloud to the source system. Start with Connect for Import.
- If the source is in a private VPC, configure private connectivity. For SaaS warehouses, see Grant VeloDB access to your VPC.
- Source-side permissions required for the selected ingestion mode.
- A target database in VeloDB Cloud, or permission to create one during the wizard.
Next steps
Choose the source-specific guide for your import job: