Skip to main content

Metrics API

Use the Metrics API to collect warehouse and cluster metrics in Prometheus exposition format. Send the metrics to a Prometheus-compatible collector and visualize them in Grafana dashboards.

Note:

The Metrics API is currently available in Private Preview. Contact the VeloDB Cloud support team to enable it for your organization.

Before you start, make sure that:

  • Metrics API access is enabled for your organization.
  • You have an API key with access to the target warehouse.
  • You know the warehouse's cloudProvider, region, warehouseId, and, for cluster requests, clusterId.

The Metrics catalog describes the metrics available in the console. The API returns the raw samples at warehouse or cluster scope.

Endpoints​

The Metrics API exposes two endpoints:

APIPathDescription
QueryWarehouseMetricsGET /v1/warehouse/{warehouseId}/metricsRaw metrics for the warehouse.
QueryClusterMetricsGET /v1/warehouse/{warehouseId}/cluster/{clusterId}/metricsRaw metrics for the specified cluster.

Both endpoints return Prometheus text exposition format:

Content-Type: text/plain; version=0.0.4; charset=utf-8

Prometheus can scrape the endpoints directly with static_configs or file_sd_configs.

Regional endpoints​

Each warehouse is served by the endpoint for its cloud provider and region:

https://apps-api.<region>.<provider>.velodb.cloud

Use the stable cloudProvider and region identifiers returned by the Management API. Do not use marketing names or localized labels.

To discover available cloud providers and regions:

# List cloud providers.
curl -H "X-API-Key: sk-xxxx" \
https://api.velodb.cloud/v1/cloud-providers

# List regions for a cloud provider.
curl -H "X-API-Key: sk-xxxx" \
https://api.velodb.cloud/v1/cloud-providers/aws/regions

Quick reference:

ProviderRegionEndpoint
awsus-east-1https://apps-api.us-east-1.aws.velodb.cloud
awsap-northeast-1https://apps-api.ap-northeast-1.aws.velodb.cloud
awsap-southeast-1https://apps-api.ap-southeast-1.aws.velodb.cloud
gcpus-central1https://apps-api.us-central1.gcp.velodb.cloud

A warehouse is reachable only through the endpoint for its own region and cloud provider. Cross-region or cross-provider requests can return 404 Not Found or fail authentication.

Private endpoint access​

You can scrape the Metrics API through a private endpoint instead of the public network. VeloDB Cloud uses the same provider-specific regional endpoint service for the Metrics API and warehouse-specific console endpoints. You can reuse an existing endpoint in the warehouse region or create one by following the cloud-provider references in Endpoint Services.

In the provider procedure, configure private DNS for the complete apps-api.<region>.<provider>.velodb.cloud hostname. Follow that procedure to select the correct record type and endpoint target, then complete its DNS, TLS, and HTTP verification steps.

Continue to use https://apps-api.<region>.<provider>.velodb.cloud in Prometheus and other clients. Do not use the private endpoint IP address or its provider-generated hostname because the original hostname is required for TLS validation and request routing. Send an authenticated request as described in Authentication to verify Metrics API access.

Authentication​

Every Metrics API request must include an API key in the HTTP header:

X-API-Key: sk-xxxxxxxxxxxxxxxxxxxx

API keys are bound to an organization and inherit the role selected when the key is created. A key can access only warehouses in its own organization and only the resources allowed by its role.

To create an API key, see API Keys. The full key is shown only once when it is created. Copy and store it immediately.

Failed authentication returns 401 Unauthorized:

{
"code": "Unauthorized.InvalidApiKey",
"message": "invalid API key"
}

Rate limit​

Each API key is allowed 100 read requests per minute per endpoint. Exceeding the limit returns 429 Too Many Requests.

For Prometheus scraping, use a scrape interval of at least 15 seconds. Keep the number of targets per endpoint at about 30 or fewer.

Common errors​

StatusCauseResolution
401 UnauthorizedThe API key is missing, invalid, expired, or not authorized for the organization.Check the X-API-Key header and create a key with access to the target warehouse.
404 Not FoundThe warehouse or cluster ID is invalid, or the request uses the wrong regional endpoint.Confirm the resource IDs and use the endpoint for the warehouse's cloudProvider and region.
429 Too Many RequestsThe API key exceeded the read-request limit for the endpoint.Reduce the scrape frequency or the number of targets, then retry after the rate-limit window.

Query warehouse metrics​

QueryWarehouseMetrics returns raw Prometheus metrics for the front-end (FE) nodes in a warehouse.

Request

GET /v1/warehouse/{warehouseId}/metrics
Host: apps-api.<region>.<provider>.velodb.cloud
X-API-Key: sk-xxxx

Path parameters

ParameterDescription
warehouseIdWarehouse ID, for example XXZ0RP.

Response

FieldDescription
Status200 OK
Content typetext/plain; version=0.0.4; charset=utf-8
BodyPrometheus exposition format text. The returned data covers Resource Metrics and Service Metrics at warehouse scope.

Example:

# TYPE doris_fe_connection_total untyped
doris_fe_connection_total{cid="<cluster-id>",instance="<node-address>",job="<warehouse-id>",product="SAAS",provider="aws",region="us-east-1",wid="<warehouse-id>"} 10 <timestamp-ms>
# TYPE doris_fe_query_total untyped
doris_fe_query_total{cid="<cluster-id>",cluster_id="<cluster-id>",cluster_name="<cluster-name>",instance="<node-address>",job="<warehouse-id>",product="SAAS",provider="aws",region="us-east-1",wid="<warehouse-id>"} 21 <timestamp-ms>
# TYPE node_memory_Buffers_bytes untyped
node_memory_Buffers_bytes{cid="<cluster-id>",instance="<node-address>",job="<warehouse-id>",product="SAAS",provider="aws",region="us-east-1",wid="<warehouse-id>"} 2.03e+08 <timestamp-ms>

The sample names and labels above illustrate the response shape. The API can return additional metrics and labels. Treat metric names, types, and labels as API output rather than a fixed allowlist.

Query cluster metrics​

QueryClusterMetrics returns raw Prometheus metrics for the specified cluster.

Request

GET /v1/warehouse/{warehouseId}/cluster/{clusterId}/metrics
Host: apps-api.<region>.<provider>.velodb.cloud
X-API-Key: sk-xxxx

Path parameters

ParameterDescription
warehouseIdWarehouse ID.
clusterIdCluster ID. An o- prefix means an observer cluster. A c- prefix means a compute cluster.

Response

FieldDescription
Status200 OK
Content typetext/plain; version=0.0.4; charset=utf-8
BodyPrometheus exposition format text. The returned data covers Resource Metrics and Service Metrics at cluster scope.

Example:

# TYPE doris_be_load_bytes untyped
doris_be_load_bytes{cid="<cluster-id>",instance="<node-address>",job="<warehouse-id>-<cluster-id>",product="SAAS",provider="aws",region="us-east-1",wid="<warehouse-id>"} 3.04e+06 <timestamp-ms>
# TYPE doris_be_tablet_base_max_compaction_score untyped
doris_be_tablet_base_max_compaction_score{cid="<cluster-id>",instance="<node-address>",job="<warehouse-id>-<cluster-id>",product="SAAS",provider="aws",region="us-east-1",wid="<warehouse-id>"} 1 <timestamp-ms>

The sample names and labels above illustrate the response shape. The API can return additional metrics and labels at cluster scope.

Integrate with Prometheus​

Because the Metrics API returns Prometheus exposition format directly, Prometheus can treat each endpoint as a scrape target.

Key points:

  • Use https as the scheme.
  • Set the per-target __metrics_path__ label because each path contains the warehouse ID or cluster ID.
  • Pass the API key through the X-API-Key HTTP header.
  • Use file_sd_configs to manage multiple warehouses and clusters. Warehouses in different regions should point to their own endpoint host.

Prometheus 2.43+ supports custom headers through http_headers.

Suggested directory layout:

prometheus/
prometheus.yml
targets/
warehouse.yml

Example prometheus.yml:

scrape_configs:
- job_name: velodb-warehouse
scheme: https
http_headers:
X-API-Key:
values:
- sk-xxxxxxxxxxxxxxxxxxxx
file_sd_configs:
- files:
- targets/warehouse.yml
refresh_interval: 1m

Example targets/warehouse.yml:

- targets:
- apps-api.us-east-1.aws.velodb.cloud
labels:
__metrics_path__: /v1/warehouse/XXZ0RP/metrics
warehouse: XXZ0RP
name: warehouse-a

- targets:
- apps-api.ap-northeast-1.aws.velodb.cloud
labels:
__metrics_path__: /v1/warehouse/XXZ0RQ/metrics
warehouse: XXZ0RQ
name: warehouse-b

- targets:
- apps-api.us-central1.gcp.velodb.cloud
labels:
__metrics_path__: /v1/warehouse/GCP0RS/cluster/c-CLUSTER001/metrics
warehouse: GCP0RS
cluster: c-CLUSTER001
name: cluster-name1

If your Prometheus version does not support http_headers, place a reverse proxy, such as Nginx or Envoy, in front of the scraper to inject X-API-Key. Do not replace X-API-Key with an Authorization header.

Visualize with Grafana​

Once Prometheus is scraping the Metrics API, you can use Grafana to query the collected samples and import pre-built dashboards that mirror the Resource Metrics and Service Metrics shown in the console.

Two ready-to-use dashboard templates are published in the monitoring-stacks repository:

DashboardFileScope
VeloDB Warehouse Monitoringgrafana_dashboard_warehouse.jsonFE node resources, QPS, query latency, connections, load jobs, routine load, workload group.
VeloDB Cluster Monitoringgrafana_dashboard_cluster.jsonBE node resources, dead nodes, query performance, cache hit rate, remote S3 throughput, load throughput, tablet compaction score.

Import a dashboard

  1. In Grafana, open Dashboards, then New, then Import.
  2. Upload the JSON file, or paste the raw content from GitHub.
  3. When prompted, select the Prometheus data source that scrapes the Metrics API. This binds the DS_PROMETHEUS template variable.
  4. Click Import.

Template variables

Both dashboards read the fixed labels described in Metric labels. After import, use the dropdowns at the top of the dashboard to scope the view:

VariableDashboardSourcePurpose
DS_PROMETHEUSBothSelected at importPrometheus data source.
widBothlabel_values(doris_fe_connection_total, wid)Warehouse selector.
cidClusterlabel_values(up{wid="$wid", cid!=""}, cid)Cluster selector, scoped to the selected warehouse.
intervalBothStatic list (30s, 1m, 5m, 10m, 30m, 1h)Rate and aggregation window for time-series panels.

If wid or cid is empty after import, confirm that Prometheus is successfully scraping the Metrics API endpoint and that the X-API-Key header is being injected.

Metric labels​

Every sample includes fixed labels injected by VeloDB Cloud:

LabelMeaning
providerCloud provider, such as aws, gcp, or azure.
regionCloud region, such as us-east-1 or ap-northeast-1.
widWarehouse ID.
cidCluster ID.
instanceNode address.
vpc_idVPC ID hosting the warehouse.
productProduct identifier.

The trailing integer on each sample line is the Prometheus-side collection timestamp in milliseconds.

FAQ​

Is there a PromQL query endpoint such as /api/v1/query?

No. The Metrics API exposes raw metrics endpoints for Prometheus scraping. For interactive exploration, use the built-in Metrics page in the VeloDB Cloud console or query your own Prometheus after scraping the metrics.

Does the API support the standard /metrics path?

No. Use /v1/warehouse/{warehouseId}/metrics or /v1/warehouse/{warehouseId}/cluster/{clusterId}/metrics.

Should I use Authorization or X-API-Key?

Use X-API-Key. HTTP header names are case-insensitive.

How do I find my provider, region, warehouse ID, and cluster ID?

You can find them in the VeloDB Cloud console. For automation, use the Management API:

Are private endpoints or IP allowlists supported?

The Metrics API supports public HTTPS access and private endpoint access. Both paths use API key authentication and rate limiting. For private connectivity, follow Private endpoint access.

Next steps​