GCP Preparation

Add users to the organization and authorize them

Notice:

If you already have a user, you can skip the users creation step.

Please grant administrator privileges to the user.

In GCP, user accounts are usually managed through Google Workspace or personal Gmail accounts. If you want to add users to your organization, you usually need administrator privileges. Here are the steps to add a new user:

Add a user through Google Workspace (if using Google Workspace)
- Sign in to the Google Admin Console (opens in a new tab).
- In the left navigation bar, click Users.
- Click the Add user button at the top of the page.
- Enter the user's basic information (such as first name, last name, and email address), and then click Add.
- The new user will receive an email with a temporary password to log in to Google Workspace.
Assign a role through IAM (using the GCP console):
- Sign in to the Google Cloud Console (opens in a new tab).
- Select your project or organization.
- Navigate to IAM & Admin > IAM.
- Click the Add button at the top of the page.
- Enter the user's email address.
- Select the role Owner to assign and click Save.

Prepare a VPC and subnet

Notice:

Currently supported region is us-east1.

If you already have a VPC and subnet, you can skip the creation step.

Before creating a BYOC type warehouse, you need to use the above user to create a VPC and subnet in advance. The following are the specific operations.

Open the GCP VPC Network (opens in a new tab) console and switch to the region where you want to deploy the BYOC warehouse.

Click VPC networks > CREATE VPC NETWORK to enter the VPC creation page.

Input VPC Name, Select Custom mode.

Input subnet Name, Select Region, Input IPv4 range.

Notice: Currently supported region is us-east1.

Select off for Private Google Access, Flow logs, Hybrid subnet.

Uncheck the Firewall rule list.

Select Regional for Dynamic Routing mode, Select Legacy for Best path selection mode.

Click CREATE to complete the creation of the VPC and subnet.

Learn about Resource Orchestration

When you create a BYOC type warehouse, the cloud platform's resource orchestration service will first automatically deploy the Agent to complete the private connection between the Agent and the VeloDB Cloud platform, and then the Agent will be responsible for the deployment and management of the BYOC warehouse.

Terraform template description

The Terraform resource orchestration template provided by VeloDB runs under your GCP account, and the template code is visible and auditable, and will not operate on your data and other environments in the VPC. You can get the Terraform template provided by SelectDB through the following link:

https://storage.googleapis.com/selectdb-cloud-online-us-west-1/public/gcp-us-west1-byoc-cf.tf

When you run the above Terraform template through GCP CloudShell, it will automatically create and deploy the Agent. Then the Agent will establish a private connection with VeloDB Cloud and complete the warehouse initialization process.

After the resource orchestration script is executed, you can enter the corresponding warehouse from the VeloDB Cloud platform and start creating a computing cluster for data analysis just like using a normal warehouse.

Permissions that Terraform templates rely on

When executing the Terraform template through CloudShell under your cloud account, cloud resources such as Compute Engine, VPC, Cloud Storage will be created or related operations will be performed, so a series of permissions are required. Before formal execution, please ensure that the user who executes this template has the corresponding permissions, otherwise the template execution may fail.

Note The execution process of the Terraform template is completely carried out under your cloud account, and the resources created also belong to your cloud account. VeloDB will not obtain your cloud account information, nor can it use the corresponding permissions of the account.

Please execute the Terraform script as an account administrator to ensure that you have sufficient permissions.

Resources Information

Executing the Terraform template will generate the following resources, as described below:

Compute Engine and Agent Machine

Name: SelectDBAgent
Purpose: Used to deploy Agent, Prometheus and other programs

Private Service Connect and Endpoint

Name: SelectDBEndpoint
Purpose: Establish a private network connection (PSC) with the SelectDB Manage service, so that you can pull control instructions and push monitoring and logs in one direction

VPC Network and Firewall Rules

Name: SelectDBSecurityGroupIngress, SelectDBSecurityGroupEgress
Purpose: Bind to the GCP instance, and limit the traffic of specific ports and specific subnets through security group rules (allow 443, 22, 5000, 9090, 8888, 8666, 8777 ports from the same subnet to enter the network, and allow all ports to leave the network)

Cloud Storage and Bucket

Name: SelectDBBucket
Purpose: Used to store data warehouse data

Service Account and Custom Role

Name: SelectDBUser (service account), SelectDBBucketKey (Bucket Key), SelectDBComputeRole (role)
Purpose: The created service account has the minimum permissions required by the Agent, and all subsequent business operations are performed using the identity of the service account (All service account information will only be used within the user's VPC and will not be leaked)

Permissions of the created service account

After the template is executed for the first time to create the resource stack, all management and control operations are performed based on the permissions of the service account. The following is an excerpt from the template.

"compute.addresses.useInternal",
"compute.addresses.createInternal",
"compute.addresses.deleteInternal",
"compute.addresses.get",
"compute.disks.create",
"compute.disks.createTagBinding",
"compute.disks.delete",
"compute.disks.deleteTagBinding",
"compute.disks.get",
"compute.disks.list",
"compute.disks.resize",
"compute.disks.setLabels",
"compute.disks.use",
"compute.firewalls.create",
"compute.firewalls.createTagBinding",
"compute.firewalls.delete",
"compute.firewalls.deleteTagBinding",
"compute.firewalls.get",
"compute.firewalls.list",
"compute.firewalls.update",
"compute.forwardingRules.create",
"compute.forwardingRules.delete",
"compute.forwardingRules.get",
"compute.instanceGroups.create",
"compute.instanceGroups.delete",
"compute.instanceGroups.update",
"compute.instances.attachDisk",
"compute.instances.create",
"compute.instances.createTagBinding",
"compute.instances.delete",
"compute.instances.deleteTagBinding",
"compute.instances.detachDisk",
"compute.instances.get",
"compute.instances.getEffectiveFirewalls",
"compute.instances.list",
"compute.instances.listTagBindings",
"compute.instances.osAdminLogin",
"compute.instances.osLogin",
"compute.instances.reset",
"compute.instances.resume",
"compute.instances.setDiskAutoDelete",
"compute.instances.setLabels",
"compute.instances.setMachineType",
"compute.instances.setMetadata",
"compute.instances.setName",
"compute.instances.setServiceAccount",
"compute.instances.setTags",
"compute.instances.start",
"compute.instances.stop",
"compute.instances.suspend",
"compute.instances.update",
"compute.instances.use",
"compute.networks.get",
"compute.networks.list",
"compute.networks.updatePolicy",
"compute.networks.use",
"compute.regionBackendServices.create",
"compute.regionBackendServices.createTagBinding",
"compute.regionBackendServices.delete",
"compute.regionBackendServices.get",
"compute.regionBackendServices.getIamPolicy",
"compute.regionBackendServices.update",
"compute.regionBackendServices.use",
"compute.regionHealthChecks.create",
"compute.regionHealthChecks.delete",
"compute.regionHealthChecks.useReadOnly",
"compute.subnetworks.get",
"compute.subnetworks.use",
"compute.zoneOperations.get",

Compute Engine permissions:

Manage VM instances

"compute.disks.create",
"compute.disks.createTagBinding",
"compute.disks.delete",
"compute.disks.deleteTagBinding",
"compute.disks.get",
"compute.disks.list",
"compute.disks.resize",
"compute.disks.setLabels",
"compute.disks.use",
"compute.instances.attachDisk",
"compute.instances.create",
"compute.instances.createTagBinding",
"compute.instances.delete",
"compute.instances.deleteTagBinding",
"compute.instances.detachDisk",
"compute.instances.get",
"compute.instances.getEffectiveFirewalls",
"compute.instances.list",
"compute.instances.listTagBindings",
"compute.instances.osAdminLogin",
"compute.instances.osLogin",
"compute.instances.reset",
"compute.instances.resume",
"compute.instances.setDiskAutoDelete",
"compute.instances.setLabels",
"compute.instances.setMachineType",
"compute.instances.setMetadata",
"compute.instances.setName",
"compute.instances.setServiceAccount",
"compute.instances.setTags",
"compute.instances.start",
"compute.instances.stop",
"compute.instances.suspend",
"compute.instances.update",
"compute.instances.use",

Manage firewall rules

"compute.firewalls.create",
"compute.firewalls.createTagBinding",
"compute.firewalls.delete",
"compute.firewalls.deleteTagBinding",
"compute.firewalls.get",
"compute.firewalls.list",
"compute.firewalls.update",

VPC Network permissions:

Get VPC-related resource information

"compute.networks.get",
"compute.networks.list",
"compute.networks.updatePolicy",
"compute.networks.use",

Manage LB resources

"compute.addresses.useInternal",
"compute.addresses.createInternal",
"compute.addresses.deleteInternal",
"compute.addresses.get",
"compute.forwardingRules.create",
"compute.forwardingRules.delete",
"compute.forwardingRules.get",
"compute.regionBackendServices.create",
"compute.regionBackendServices.createTagBinding",
"compute.regionBackendServices.delete",
"compute.regionBackendServices.get",
"compute.regionBackendServices.getIamPolicy",
"compute.regionBackendServices.update",
"compute.regionBackendServices.use",
"compute.regionHealthChecks.create",
"compute.regionHealthChecks.delete",
"compute.regionHealthChecks.useReadOnly",

Cloud Storage permissions:
- Manage Buckets and read and write buckets and their contents (for specific buckets)
```
roles/storage.admin
```
Account permissions:
- Allows the newly created service account to be bound to all subsequent VM instances
```
roles/iam.serviceAccountUser
```

GCP Use Warehouse