GCP Preparation

Creating a BYOC warehouse on Google Cloud involves two Google Cloud-side tasks: preparing a VPC and subnet, and (optionally) reviewing the Terraform template and the resources it creates.

Prepare a VPC and subnet

You only need this if you deploy into an existing VPC. If no VPC and subnet meet the requirements, create them in advance.

note

If a VPC and subnet already meet the region, availability zone, and subnet requirements and you want to deploy the BYOC warehouse into them, skip the steps below. For the supported regions, see Supported Regions.

Create a VPC and subnet

Open the GCP VPC Network console and switch to the region where you want to deploy the BYOC warehouse.
Click VPC networks > CREATE VPC NETWORK to open the VPC creation page.
Enter a VPC Name and select Custom mode.
Enter a subnet Name, select a supported Region (see the table above), and enter an IPv4 range.
Select off for Private Google Access, Flow logs, and Hybrid subnet.
Uncheck the firewall rule list.
Select Regional for Dynamic routing mode, and Legacy for Best path selection mode.
Click CREATE to finish creating the VPC and subnet.

Understand the Terraform template (optional)

note

You do not need to do anything in this section. Read on only if you want to understand how the deployment works.

When you run the Terraform template in Google Cloud Shell under your cloud account, it operates on cloud resources such as the VPC, Compute Engine instances, and buckets, so it requires a set of IAM permissions.

Use administrator privileges to run the template, or ask your administrator to do it for you. Otherwise the deployment can fail because of insufficient permissions.

About the template

The Terraform template that VeloDB provides runs under your GCP account. Its code is visible and auditable, and it does not touch your data or other environments in the VPC. You can review the template at this link:

https://storage.googleapis.com/velodb-cloud-online-us-west-1/public/gcp-byoc.tf

When you run the template in Google Cloud Shell, it creates and deploys the Agent. The Agent then establishes a private connection with VeloDB Cloud and completes the warehouse initialization.

After the deployment finishes, open the warehouse from the VeloDB Cloud console and create a compute cluster for data analysis, the same as with any other warehouse.

Resources Information

Compute Engine Instance
- Name: VeloDBAgent
- Purpose: Used to deploy Agent, Prometheus, FluentBit and other programs
Private Service Connect Endpoint
- Name: VeloDBEndpoint
- Purpose: Establishes private network connection with VeloDB Manage service to pull control instructions and enable one-way push of monitoring and logs
VPC Firewall Rules
- Name: VeloDBSecurityGroupIngress, VeloDBSecurityGroupEgress
- Purpose: Bound to the all VM instances launched by VeloDB, and restricts traffic through security group rules (allows all traffic from the same security group to access all ports, traffic from the same subnet to access port 5000, and allows all outbound traffic)
Cloud Storage Bucket
- Name: VeloDBBucket
- Purpose: Used to store data warehouse data
IAM & Service Account & Custom Role
- Names:
  - VeloDBControlPlaneRole (control plane service account)
  - VeloDBControlPlaneRolePolicy (custom role policy)
  - VeloDBBucketKey (bucket key)
  - VeloDBDataAccessRole (kernel service account)
  - Purposes: The created service account has the minimum permission policy required by the Agent, and all subsequent control operations are carried out using the identity of this service account

Permissions of the created service account

After the terraform template is executed for the first time, a service account will be created for subsequent management of data warehouse related components in your VPC. The following is a description of the permissions of the service account.

Permission summary:

compute.addresses.create
compute.addresses.createInternal
compute.addresses.delete
compute.addresses.deleteInternal
compute.addresses.setLabels
compute.addresses.get
compute.addresses.list
compute.addresses.use
compute.addresses.useInternal
compute.disks.create
compute.disks.createTagBinding
compute.disks.delete
compute.disks.deleteTagBinding
compute.disks.get
compute.disks.list
compute.disks.resize
compute.disks.setLabels
compute.disks.use
compute.firewalls.create
compute.firewalls.createTagBinding
compute.firewalls.delete
compute.firewalls.deleteTagBinding
compute.firewalls.get
compute.firewalls.list
compute.firewalls.update
compute.forwardingRules.create
compute.forwardingRules.delete
compute.forwardingRules.get
compute.forwardingRules.setLabels
compute.instanceGroups.create
compute.instanceGroups.delete
compute.instanceGroups.update
compute.instanceGroups.use
compute.instances.attachDisk
compute.instances.create
compute.instances.createTagBinding
compute.instances.delete
compute.instances.deleteTagBinding
compute.instances.detachDisk
compute.instances.get
compute.instances.getEffectiveFirewalls
compute.instances.list
compute.instances.listTagBindings
compute.instances.osAdminLogin
compute.instances.osLogin
compute.instances.reset
compute.instances.resume
compute.instances.setDiskAutoDelete
compute.instances.setLabels
compute.instances.setMachineType
compute.instances.setMetadata
compute.instances.setName
compute.instances.setServiceAccount
compute.instances.setTags
compute.instances.start
compute.instances.stop
compute.instances.suspend
compute.instances.update
compute.instances.use
compute.networks.get
compute.networks.list
compute.networks.updatePolicy
compute.networks.use
compute.regionBackendServices.create
compute.regionBackendServices.createTagBinding
compute.regionBackendServices.delete
compute.regionBackendServices.deleteTagBinding
compute.regionBackendServices.get
compute.regionBackendServices.getIamPolicy
compute.regionBackendServices.list
compute.regionBackendServices.update
compute.regionBackendServices.use
compute.regionHealthChecks.create
compute.regionHealthChecks.delete
compute.regionHealthChecks.useReadOnly
compute.subnetworks.get
compute.subnetworks.list
compute.subnetworks.use
compute.zoneOperations.get

The specific permissions are divided as follows:

Compute Engine permissions:

Manage VM instances

compute.disks.create
compute.disks.createTagBinding
compute.disks.delete
compute.disks.deleteTagBinding
compute.disks.get
compute.disks.list
compute.disks.resize
compute.disks.setLabels
compute.disks.use
compute.instanceGroups.create
compute.instanceGroups.delete
compute.instanceGroups.update
compute.instanceGroups.use
compute.instances.attachDisk
compute.instances.create
compute.instances.createTagBinding
compute.instances.delete
compute.instances.deleteTagBinding
compute.instances.detachDisk
compute.instances.get
compute.instances.getEffectiveFirewalls
compute.instances.list
compute.instances.listTagBindings
compute.instances.osAdminLogin
compute.instances.osLogin
compute.instances.reset
compute.instances.resume
compute.instances.setDiskAutoDelete
compute.instances.setLabels
compute.instances.setMachineType
compute.instances.setMetadata
compute.instances.setName
compute.instances.setServiceAccount
compute.instances.setTags
compute.instances.start
compute.instances.stop
compute.instances.suspend
compute.instances.update
compute.instances.use

Manage firewall rules

compute.firewalls.create
compute.firewalls.createTagBinding
compute.firewalls.delete
compute.firewalls.deleteTagBinding
compute.firewalls.get
compute.firewalls.list
compute.firewalls.update

VPC Network permissions:

Get VPC-related resource information

compute.networks.get
compute.networks.list
compute.networks.updatePolicy
compute.networks.use
compute.subnetworks.get
compute.subnetworks.list
compute.subnetworks.use
compute.zoneOperations.get

Manage LB resources

compute.addresses.create
compute.addresses.createInternal
compute.addresses.delete
compute.addresses.deleteInternal
compute.addresses.setLabels
compute.addresses.get
compute.addresses.list
compute.addresses.use
compute.addresses.useInternal
compute.forwardingRules.create
compute.forwardingRules.delete
compute.forwardingRules.get
compute.forwardingRules.setLabels
compute.regionBackendServices.create
compute.regionBackendServices.createTagBinding
compute.regionBackendServices.delete
compute.regionBackendServices.deleteTagBinding
compute.regionBackendServices.get
compute.regionBackendServices.getIamPolicy
compute.regionBackendServices.list
compute.regionBackendServices.update
compute.regionBackendServices.use
compute.regionHealthChecks.create
compute.regionHealthChecks.delete
compute.regionHealthChecks.useReadOnly

Cloud Storage permissions:
- Manage Buckets and read and write buckets and their contents (for specific buckets)
```
roles/storage.admin
```
Account permissions:
- Allow the newly created service account to be associated with the VM instance so that the VM instance can perform related operations as the service account
```
roles/iam.serviceAccountUser
```

Prepare a VPC and subnet​

Create a VPC and subnet​

Understand the Terraform template (optional)​

About the template​

Resources Information​

Permissions of the created service account​