Skip to main content

Grant VeloDB Access to Your VPC on AWS

Use this guide when a SaaS warehouse needs to access a private data source in your AWS VPC, such as PostgreSQL, MySQL, Kafka, or another TCP service used by import or integration jobs.

Before you begin, read Grant VeloDB access to your VPC for the connection direction, ownership model, and network-control scope.

In this direction:

  • You create and own an AWS endpoint service in your VPC.
  • The endpoint service is backed by a Network Load Balancer (NLB) that forwards traffic to your private data source.
  • VeloDB Cloud creates and manages the interface endpoint that connects to your endpoint service.

Prerequisites​

  • The private data source is reachable from the VPC and subnets where the NLB is deployed.
  • You have permission to create or manage target groups, Network Load Balancers, endpoint services, and endpoint connection requests in AWS.
  • The endpoint service must be created in the same AWS region as the VeloDB warehouse.
  • Security groups, network ACLs, and data-source allowlists must allow traffic from the NLB to the data source on the required port, such as 3306 for MySQL or 5432 for PostgreSQL.

1. Start the connection setup in VeloDB Cloud​

You can start the setup from either of the following entry points:

  • Warehouse connection page: In VeloDB Cloud, open the target warehouse, go to Connection, select the Private Link tab, and under Grant VeloDB Access to Your VPC, click New Connection.
  • Import workflow: When configuring an import task that needs to access a private data source in your VPC, add a PrivateLink connection from the connection setup step.

VeloDB Cloud shows the warehouse region and the ARN of VeloDB. You will use this ARN in Step 5 to add VeloDB Cloud as an allowed principal for the AWS endpoint service.

2. Create or select a target group​

The target group represents the private service that VeloDB Cloud needs to access.

For example, if you want VeloDB Cloud to import data from an Amazon RDS or Amazon Aurora MySQL/PostgreSQL database, create a target group that points to the private IP address of the database endpoint and uses the database port.

If you already have a suitable target group, you can reuse it. Otherwise, create one in AWS:

  1. Open the AWS console and go to EC2 > Target Groups.
  2. Click Create target group.
  3. Select a target type. For Amazon RDS or Amazon Aurora, select IP addresses because the database is reached through private IP addresses behind the database endpoint.
  4. Set the protocol and port used by the data source. For example, use TCP 3306 for MySQL or TCP 5432 for PostgreSQL.
  5. Select the VPC where the data source runs.
  6. Register the private IP address of the RDS or Aurora endpoint as the target.
  7. Confirm that the targets become healthy.

To get the private IP address of an RDS or Aurora endpoint, resolve the database endpoint DNS name from an environment that can access the VPC private DNS, such as an EC2 instance in the same VPC:

dig +short <your-rds-endpoint>

Use the returned private IP address as the target. If the command returns multiple IP addresses, register all of them.

Note Amazon RDS and Amazon Aurora endpoints are DNS names, and the private IP addresses behind them can change after failover, maintenance, or scaling. For production use, do not rely on a one-time IP registration. Use the Lambda-based automation described in Automatically sync IP targets for RDS and Aurora endpoints to keep the target group up to date. For other services, choose the target type and targets based on how the service is deployed.

3. Create or select a Network Load Balancer​

The endpoint service must be associated with a Network Load Balancer.

If you already have a suitable NLB, you can reuse it. Otherwise, create one in AWS:

  1. Go to EC2 > Load Balancers.
  2. Click Create load balancer and select Network Load Balancer.
  3. Choose Internal as the scheme.
  4. Select the same VPC as the target group.
  5. Select subnets that can route to the private data source.
  6. Add a listener for the data-source port and forward it to the target group.
  7. Create the load balancer.

4. Create the AWS endpoint service​

  1. In the AWS console, open VPC > Endpoint services.

  2. Switch to the same region as the VeloDB warehouse.

  3. Click Create endpoint service.

  4. Configure the endpoint service.

    ParameterDescription
    Load balancer typeSelect Network.
    Available load balancersSelect the NLB that forwards traffic to your data source.
    Acceptance requiredKeep this enabled if you want to manually approve the endpoint connection request from VeloDB Cloud.
    TagsOptional. Add tags according to your organization's AWS resource-management policy.

  5. Click Create.

5. Allow VeloDB Cloud to connect​

After the endpoint service is created, add VeloDB Cloud as an allowed principal.

  1. Open the endpoint service details page.
  2. Go to the Allow principals tab.
  3. Click Allow principals.
  4. Paste the ARN of VeloDB shown in the VeloDB Cloud console.
  5. Save the allowlist.

6. Register the endpoint service in VeloDB Cloud​

  1. On the AWS endpoint service details page, copy the Service ID and Service Name.
  2. Return to the VeloDB Cloud connection setup page.
  3. Paste the Service ID and Service Name.
  4. Click Register.
  5. Enter a connection name and click Create Now.

7. Accept the endpoint connection request​

If Acceptance required is enabled for the endpoint service, approve the request in AWS:

  1. In AWS, open the endpoint service.
  2. Go to the Endpoint connections tab.
  3. Select the connection request from VeloDB Cloud.
  4. Click Accept endpoint connection request.

8. Verify the connection status​

Return to VeloDB Cloud and refresh the Connection page. The endpoint status changes from pendingAcceptance to available after AWS accepts the request and the PrivateLink connection becomes ready.

Automatically sync IP targets for RDS and Aurora endpoints​

Amazon RDS and Amazon Aurora endpoints are DNS names. AWS can change the private IP addresses behind these endpoints during events such as failover, maintenance, or scaling. Because the target group uses IP targets, VeloDB recommends using AWS Lambda to keep the target group synchronized with the current endpoint resolution.

A typical automation works as follows:

  1. Create a Lambda function in the same VPC as the database, or in another network environment that can resolve the database endpoint to private IP addresses.
  2. Configure the function with the database endpoint DNS name and the target group ARN.
  3. On each run, the function resolves the database endpoint, compares the returned private IP addresses with the registered targets, registers new IP addresses, and deregisters stale IP addresses.
  4. Trigger the function on a schedule with Amazon EventBridge, for example every 1 to 5 minutes. You can also trigger it from RDS or Aurora failover and maintenance events if your operations process already captures those events.

The Lambda execution role needs permissions to describe, register, and deregister target group targets, for example:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "elasticloadbalancing:DescribeTargetHealth",
        "elasticloadbalancing:RegisterTargets",
        "elasticloadbalancing:DeregisterTargets"
      ],
      "Resource": "<your-target-group-arn>"
    }
  ]
}

The following Python example resolves the database endpoint and keeps one target group in sync. Configure the Lambda function with the environment variables DB_ENDPOINT, TARGET_GROUP_ARN, and TARGET_PORT.

import os
import socket

import boto3


elbv2 = boto3.client("elbv2")


def resolve_endpoint(endpoint):
    return {
        result[4][0]
        for result in socket.getaddrinfo(endpoint, None, family=socket.AF_INET)
    }


def list_registered_targets(target_group_arn):
    response = elbv2.describe_target_health(TargetGroupArn=target_group_arn)
    return {
        target["Target"]["Id"]
        for target in response["TargetHealthDescriptions"]
    }


def lambda_handler(event, context):
    endpoint = os.environ["DB_ENDPOINT"]
    target_group_arn = os.environ["TARGET_GROUP_ARN"]
    target_port = int(os.environ["TARGET_PORT"])

    resolved_ips = resolve_endpoint(endpoint)
    registered_ips = list_registered_targets(target_group_arn)

    targets_to_add = resolved_ips - registered_ips
    targets_to_remove = registered_ips - resolved_ips

    if targets_to_add:
        elbv2.register_targets(
            TargetGroupArn=target_group_arn,
            Targets=[
                {"Id": ip, "Port": target_port}
                for ip in sorted(targets_to_add)
            ],
        )

    if targets_to_remove:
        elbv2.deregister_targets(
            TargetGroupArn=target_group_arn,
            Targets=[
                {"Id": ip, "Port": target_port}
                for ip in sorted(targets_to_remove)
            ],
        )

    return {
        "endpoint": endpoint,
        "resolved_ips": sorted(resolved_ips),
        "registered_before": sorted(registered_ips),
        "added": sorted(targets_to_add),
        "removed": sorted(targets_to_remove),
    }

Keep the Lambda function lightweight and idempotent. If the resolved IP addresses already match the target group, the function should make no changes. Before using the PrivateLink connection in production, verify that the automation can register new targets and remove stale targets without interrupting database access.

Troubleshooting​

If the status does not become available, check the following items:

  • The AWS endpoint service is in the same region as the VeloDB warehouse.
  • The VeloDB ARN is added to Allow principals for the endpoint service.
  • The endpoint connection request has been accepted in AWS.
  • The NLB listener forwards traffic to the correct target group and port.
  • Target group health checks are passing.
  • Security groups, network ACLs, routing rules, and data-source allowlists permit traffic from the NLB to the data source.