GCP Preparation

Add users to the organization and authorize them

Create a custom role

Notice: This custom role is the minimum set of permissions required to create a velodb cloud warehouse. If you plan to directly grant the user the Owner permission of the project, you can skip creating a custom role.

For detailed steps on creating a custom role, refer to the GCP documentation (opens in a new tab) .

Switch to the gcloud tab and follow the instructions to create a custom role using cloud shell:

• Activate Cloud Shell on this page.

  

• Execute the vi velodb_cloud_role.yaml command in Cloud Shell to create a yaml file.

  Press i to enter edit mode, copy and paste the following content, press ESC, enter :wq to save and exit.

  title: velodb_cloud_role
  description: velodb cloud role
  stage: GA
  includedPermissions:
  - compute.addresses.create
  - compute.addresses.createInternal
  - compute.addresses.delete
  - compute.addresses.deleteInternal
  - compute.addresses.get
  - compute.addresses.list
  - compute.addresses.use
  - compute.addresses.useInternal
  - compute.disks.create
  - compute.disks.createTagBinding
  - compute.disks.delete
  - compute.disks.deleteTagBinding
  - compute.disks.get
  - compute.disks.list
  - compute.disks.resize
  - compute.disks.setLabels
  - compute.disks.use
  - compute.firewalls.create
  - compute.firewalls.createTagBinding
  - compute.firewalls.delete
  - compute.firewalls.deleteTagBinding
  - compute.firewalls.get
  - compute.firewalls.list
  - compute.firewalls.update
  - compute.forwardingRules.create
  - compute.forwardingRules.delete
  - compute.forwardingRules.get
  - compute.forwardingRules.pscCreate
  - compute.forwardingRules.pscDelete
  - compute.globalOperations.get
  - compute.instanceGroups.create
  - compute.instanceGroups.delete
  - compute.instanceGroups.update
  - compute.instanceGroups.use
  - compute.instances.attachDisk
  - compute.instances.create
  - compute.instances.createTagBinding
  - compute.instances.delete
  - compute.instances.deleteTagBinding
  - compute.instances.detachDisk
  - compute.instances.get
  - compute.instances.getEffectiveFirewalls
  - compute.instances.list
  - compute.instances.listTagBindings
  - compute.instances.osAdminLogin
  - compute.instances.osLogin
  - compute.instances.reset
  - compute.instances.resume
  - compute.instances.setDiskAutoDelete
  - compute.instances.setLabels
  - compute.instances.setMachineType
  - compute.instances.setMetadata
  - compute.instances.setName
  - compute.instances.setServiceAccount
  - compute.instances.setTags
  - compute.instances.start
  - compute.instances.stop
  - compute.instances.suspend
  - compute.instances.update
  - compute.instances.use
  - compute.networks.get
  - compute.networks.list
  - compute.networks.updatePolicy
  - compute.networks.use
  - compute.regionBackendServices.create
  - compute.regionBackendServices.createTagBinding
  - compute.regionBackendServices.delete
  - compute.regionBackendServices.deleteTagBinding
  - compute.regionBackendServices.get
  - compute.regionBackendServices.getIamPolicy
  - compute.regionBackendServices.list
  - compute.regionBackendServices.update
  - compute.regionBackendServices.use
  - compute.regionHealthChecks.create
  - compute.regionHealthChecks.delete
  - compute.regionHealthChecks.useReadOnly
  - compute.regionOperations.get
  - compute.subnetworks.get
  - compute.subnetworks.list
  - compute.subnetworks.setPrivateIpGoogleAccess
  - compute.subnetworks.use
  - compute.zoneOperations.get
  - compute.zones.get
  - compute.zones.list
  - iam.roles.create
  - iam.roles.delete
  - iam.roles.get
  - iam.roles.update
  - iam.serviceAccounts.actAs
  - iam.serviceAccounts.create
  - iam.serviceAccounts.createTagBinding
  - iam.serviceAccounts.delete
  - iam.serviceAccounts.deleteTagBinding
  - iam.serviceAccounts.enable
  - iam.serviceAccounts.get
  - iam.serviceAccounts.getAccessToken
  - iam.serviceAccounts.getIamPolicy
  - iam.serviceAccounts.setIamPolicy
  - iam.serviceAccounts.update
  - resourcemanager.projects.createPolicyBinding
  - resourcemanager.projects.deletePolicyBinding
  - resourcemanager.projects.get
  - resourcemanager.projects.getIamPolicy
  - resourcemanager.projects.setIamPolicy
  - servicedirectory.namespaces.create
  - servicedirectory.namespaces.delete
  - servicedirectory.namespaces.get
  - servicedirectory.services.create
  - servicedirectory.services.delete
  - servicedirectory.services.get
  - serviceusage.services.enable
  - serviceusage.services.use
  - storage.buckets.create
  - storage.buckets.createTagBinding
  - storage.buckets.delete
  - storage.buckets.deleteTagBinding
  - storage.buckets.get
  - storage.buckets.getIamPolicy
  - storage.buckets.setIamPolicy
  - storage.hmacKeys.create
  - storage.hmacKeys.delete
  - storage.hmacKeys.get
  - storage.hmacKeys.update
  - storage.objects.delete
  - storage.objects.get
  - storage.objects.list

• Replace PROJECT_ID with the actual project id and execute the following command

  gcloud iam roles create velodb_cloud_role --project=PROJECT_ID \
  --file=velodb_cloud_role.yaml

  Input y to complete the creation of the custom role.

  

Add users to the organization and authorize them

Notice: If you already have a user, you can skip the users creation step.

In GCP, user accounts are usually managed through Google Workspace or personal Gmail accounts. If you want to add users to your organization, you usually need administrator privileges. Here are the steps to add a new user:

1. Add a user through Google Workspace (if using Google Workspace)
   • Sign in to the Google Admin Console (opens in a new tab).
   • In the left navigation bar, click Users.
   • Click the Add user button at the top of the page.
   • Enter the user's basic information (such as first name, last name, and email address), and then click Add.
   • The new user will receive an email with a temporary password to log in to Google Workspace.
2. Assign a role through IAM (using the GCP console):
   • Sign in to the Google Cloud Console (opens in a new tab).
   • Select your project or organization.
   • Navigate to IAM & Admin > IAM.
   • Click the Add button at the top of the page.
   • Enter the user's email address.
   • Select the role above to assign and click Save.

Prepare a VPC and subnet

Notice:

• Currently supported region is us-east1.
• If you already have a VPC and subnet, you can skip the creation step.

Before creating a BYOC type warehouse, you need to use the above user to create a VPC and subnet in advance. The following are the specific operations.

Open the GCP VPC Network (opens in a new tab) console and switch to the region where you want to deploy the BYOC warehouse.

Click VPC networks > CREATE VPC NETWORK to enter the VPC creation page.

Input VPC Name, Select Custom mode.

Input subnet Name, Select Region, Input IPv4 range.

Notice: Currently supported region is us-east1.

Select off for Private Google Access, Flow logs, Hybrid subnet.

Uncheck the Firewall rule list.

Select Regional for Dynamic Routing mode, Select Legacy for Best path selection mode.

Click CREATE to complete the creation of the VPC and subnet.

Learn about Resource Orchestration

When you create a BYOC type warehouse, the cloud platform's resource orchestration service will first automatically deploy the Agent to complete the private connection between the Agent and the VeloDB Cloud platform, and then the Agent will be responsible for the deployment and management of the BYOC warehouse.

Terraform template description

The Terraform resource orchestration template provided by VeloDB runs under your GCP account, and the template code is visible and auditable, and will not operate on your data and other environments in the VPC. You can get the Terraform template provided by SelectDB through the following link:

https://storage.googleapis.com/selectdb-cloud-online-us-west-1/public/gcp-us-west1-byoc-cf.tf

When you run the above Terraform template through GCP CloudShell, it will automatically create and deploy the Agent. Then the Agent will establish a private connection with VeloDB Cloud and complete the warehouse initialization process.

After the resource orchestration script is executed, you can enter the corresponding warehouse from the VeloDB Cloud platform and start creating a computing cluster for data analysis just like using a normal warehouse.

Permissions that Terraform templates rely on

When executing the Terraform template through CloudShell under your cloud account, cloud resources such as Compute Engine, VPC, Cloud Storage will be created or related operations will be performed, so a series of permissions are required. Before formal execution, please ensure that the user who executes this template has the corresponding permissions, otherwise the template execution may fail.

Note The execution process of the Terraform template is completely carried out under your cloud account, and the resources created also belong to your cloud account. VeloDB will not obtain your cloud account information, nor can it use the corresponding permissions of the account.

Please execute the Terraform script as an account administrator to ensure that you have sufficient permissions.

Resources Information

Executing the Terraform template will generate the following resources, as described below:

Compute Engine and Agent Machine

• Name: VeloDBAgent
• Use: Used to deploy Agent, Prometheus and other programs

Private Service Connect and Endpoint

• Name: VeloDBEndpoint
• Use: Establish a private network connection (PSC) with the VeloDB Manage service, so that you can pull control instructions and push monitoring and logs in one direction

VPC Network and Firewall Rules

• Name: VeloDBSecurityGroupIngress, VeloDBSecurityGroupEgress
• Use: It is bound to the GCP instance, and the security group rules limit the traffic of specific ports and specific subnets to pass (8666 and 8777 ports from the same subnet are allowed to enter the network, and all ports are allowed to leave the network)

Cloud Storage and Bucket

• Name: VeloDBBucket
• Use: Used to store data warehouse data

Service Account and Custom Role

• Name: VeloDBUser (service account), VeloDBBucketKey (Bucket Key), VeloDBComputeRole (role)
• Use: The created service account has the minimum permissions required by the Agent, and all subsequent business operations are performed using the identity of the service account (All service account information will only be used within the user's VPC and will not be leaked)

Permissions of the created service account

After the template is executed for the first time to create the resource stack, all management and control operations are performed based on the permissions of the service account. The following is an excerpt from the template.

compute.addresses.create
compute.addresses.createInternal
compute.addresses.delete
compute.addresses.deleteInternal
compute.addresses.get
compute.addresses.list
compute.addresses.use
compute.addresses.useInternal
compute.disks.create
compute.disks.createTagBinding
compute.disks.delete
compute.disks.deleteTagBinding
compute.disks.get
compute.disks.list
compute.disks.resize
compute.disks.setLabels
compute.disks.use
compute.firewalls.create
compute.firewalls.createTagBinding
compute.firewalls.delete
compute.firewalls.deleteTagBinding
compute.firewalls.get
compute.firewalls.list
compute.firewalls.update
compute.forwardingRules.create
compute.forwardingRules.delete
compute.forwardingRules.get
compute.instanceGroups.create
compute.instanceGroups.delete
compute.instanceGroups.update
compute.instanceGroups.use
compute.instances.attachDisk
compute.instances.create
compute.instances.createTagBinding
compute.instances.delete
compute.instances.deleteTagBinding
compute.instances.detachDisk
compute.instances.get
compute.instances.getEffectiveFirewalls
compute.instances.list
compute.instances.listTagBindings
compute.instances.osAdminLogin
compute.instances.osLogin
compute.instances.reset
compute.instances.resume
compute.instances.setDiskAutoDelete
compute.instances.setLabels
compute.instances.setMachineType
compute.instances.setMetadata
compute.instances.setName
compute.instances.setServiceAccount
compute.instances.setTags
compute.instances.start
compute.instances.stop
compute.instances.suspend
compute.instances.update
compute.instances.use
compute.networks.get
compute.networks.list
compute.networks.updatePolicy
compute.networks.use
compute.regionBackendServices.create
compute.regionBackendServices.createTagBinding
compute.regionBackendServices.delete
compute.regionBackendServices.deleteTagBinding
compute.regionBackendServices.get
compute.regionBackendServices.getIamPolicy
compute.regionBackendServices.list
compute.regionBackendServices.update
compute.regionBackendServices.use
compute.regionHealthChecks.create
compute.regionHealthChecks.delete
compute.regionHealthChecks.useReadOnly
compute.subnetworks.get
compute.subnetworks.list
compute.subnetworks.use
compute.zoneOperations.get

• Compute Engine permissions:

  • Manage VM instances

    compute.disks.create
    compute.disks.createTagBinding
    compute.disks.delete
    compute.disks.deleteTagBinding
    compute.disks.get
    compute.disks.list
    compute.disks.resize
    compute.disks.setLabels
    compute.disks.use
    compute.instanceGroups.create
    compute.instanceGroups.delete
    compute.instanceGroups.update
    compute.instanceGroups.use
    compute.instances.attachDisk
    compute.instances.create
    compute.instances.createTagBinding
    compute.instances.delete
    compute.instances.deleteTagBinding
    compute.instances.detachDisk
    compute.instances.get
    compute.instances.getEffectiveFirewalls
    compute.instances.list
    compute.instances.listTagBindings
    compute.instances.osAdminLogin
    compute.instances.osLogin
    compute.instances.reset
    compute.instances.resume
    compute.instances.setDiskAutoDelete
    compute.instances.setLabels
    compute.instances.setMachineType
    compute.instances.setMetadata
    compute.instances.setName
    compute.instances.setServiceAccount
    compute.instances.setTags
    compute.instances.start
    compute.instances.stop
    compute.instances.suspend
    compute.instances.update
    compute.instances.use
  • Manage firewall rules

    compute.firewalls.create
    compute.firewalls.createTagBinding
    compute.firewalls.delete
    compute.firewalls.deleteTagBinding
    compute.firewalls.get
    compute.firewalls.list
    compute.firewalls.update

• VPC Network permissions:

  • Get VPC-related resource information

    compute.networks.get
    compute.networks.list
    compute.networks.updatePolicy
    compute.networks.use
    compute.subnetworks.get
    compute.subnetworks.list
    compute.subnetworks.use
    compute.zoneOperations.get
  • Manage LB resources

    compute.addresses.create
    compute.addresses.createInternal
    compute.addresses.delete
    compute.addresses.deleteInternal
    compute.addresses.get
    compute.addresses.list
    compute.addresses.use
    compute.addresses.useInternal
    compute.forwardingRules.create
    compute.forwardingRules.delete
    compute.forwardingRules.get
    compute.regionBackendServices.create
    compute.regionBackendServices.createTagBinding
    compute.regionBackendServices.delete
    compute.regionBackendServices.deleteTagBinding
    compute.regionBackendServices.get
    compute.regionBackendServices.getIamPolicy
    compute.regionBackendServices.list
    compute.regionBackendServices.update
    compute.regionBackendServices.use
    compute.regionHealthChecks.create
    compute.regionHealthChecks.delete
    compute.regionHealthChecks.useReadOnly

• Cloud Storage permissions:

  • Manage Buckets and read and write buckets and their contents (for specific buckets)

    roles/storage.admin

• Account permissions:

  • Allows the newly created service account to be bound to all subsequent VM instances

    roles/iam.serviceAccountUser