Skip to main content

Workload Groups

Workload Groups let you control how queries share resources within a cluster. Use them to reserve or cap CPU and memory, limit concurrent queries, queue work, restrict scan resources, and limit read throughput for different workloads.

For background on Workload Groups and in-process resource isolation, see Workload Group.

Workload Groups apply to a single compute cluster. They provide in-process resource isolation. They do not isolate data, shared caches, or shared thread pools, and they do not replace warehouse users and roles. Use a separate cluster when your workload requires stronger compute isolation.

Grant the USAGE privilege on a Workload Group to the warehouse users or roles that need to use it. Then assign the Workload Group through the user's default Workload Group property or the workload_group session variable. The session variable takes precedence. See Warehouse Users and Roles.

Note:

Workload Groups are available only when the warehouse core version supports them. If the Workload Groups entry is unavailable, check the warehouse version or contact VeloDB Support.

Open Workload Groups​

  1. Log in to the VeloDB Cloud console.
  2. In the upper-left corner, select the warehouse that you want to use.
  3. In the left navigation pane, under Compute, click Workload Groups.

The page lists each Workload Group, its compute cluster, active queries, queued queries, and current memory usage. Use the page to review resource pressure before changing a group.

Create a Workload Group​

  1. Open Workload Groups.

  2. Click Add Workload Group.

  3. Enter a unique Name.

  4. Select the Cluster that the Workload Group serves.

  5. Configure the resource and queue limits that apply to this workload:

    • Minimum CPU and Maximum CPU reserve and cap CPU as a percentage of the cluster's available CPU. When CPU is idle, a group can use more than its minimum. Its CPU use does not exceed its maximum.
    • Minimum Memory and Maximum Memory reserve and cap memory as a percentage of the cluster's available memory. When memory is under pressure, VeloDB Cloud can cancel queries to make the reserved memory available. When a group exceeds its maximum, VeloDB Cloud can spill to disk or cancel queries.
    • Maximum Concurrency limits the number of queries that can run at the same time.
    • Maximum Queue Size controls how many additional queries can wait. A value of 0 rejects queries when the concurrency limit is reached.
    • Queue Timeout sets how long a queued query can wait before it fails.

  6. Click Confirm.

Choose limits that leave capacity for other Workload Groups and system activity. A maximum limit is not a reservation. A minimum limit reserves capacity when resources are contested. The sum of all minimum CPU values and the sum of all minimum memory values must each be no more than 100%. A minimum value cannot exceed its corresponding maximum value.

Edit or delete a Workload Group​

  1. Open Workload Groups.
  2. In the Actions column for the target group, click the edit icon to change its limits, or click the delete icon to remove it.
  3. Confirm the operation.

The default normal Workload Group cannot be deleted. Before deleting another Workload Group, remove or update the privileges, user defaults, session settings, and workload policies that refer to it. Queries already running in the group can be affected when its limits change or the group is deleted.

Use SQL instead​

You can manage Workload Groups from the SQL Editor or another SQL client. In Cloud mode, specify the cluster with the FOR <cluster> clause. The clause is required for CREATE, ALTER, and DROP WORKLOAD GROUP statements:

PROPERTIES (
    "max_cpu_percent" = "50%",
    "max_memory_percent" = "50%",
    "max_concurrency" = "20",
    "max_queue_size" = "50",
    "queue_timeout" = "30000",
    "read_bytes_per_second" = "104857600",
    "remote_read_bytes_per_second" = "52428800"
);

The read_bytes_per_second property limits reads from internal tables, including spill-file directories. The remote_read_bytes_per_second property limits reads from external tables. For syntax, supported properties, and inspection commands, see CREATE WORKLOAD GROUP, ALTER WORKLOAD GROUP, DROP WORKLOAD GROUP, and SHOW WORKLOAD GROUPS.

To assign a group with SQL, use one of the following statements after granting USAGE:

SET PROPERTY 'default_workload_group' = 'reporting';
SET workload_group = 'reporting';

The first statement persists the user's default. The second affects only the current session and takes precedence over the default.

Next steps​