Collect VeloDB Cloud metrics with Datadog Agent

Use Datadog Agent's OpenMetrics check to collect selected metrics from the VeloDB Cloud Metrics API and send them to your Datadog organization.

VeloDB Cloud Metrics API
        -> Datadog Agent in your environment
        -> Datadog custom metrics
        -> Metrics Explorer

The Agent pulls metrics over HTTPS and sends them to Datadog. Your VeloDB and Datadog credentials remain in your environment. This integration sends metrics only; it does not send Datadog APM traces.

Scope and compatibility

This is a Datadog-specific, end-to-end configuration guide. Datadog is the currently validated external monitoring example. For protocol-level compatibility with other collectors, see External monitoring integrations.

Private Preview: The Metrics API is currently available in Private Preview. Before starting, confirm that it is enabled for your organization. For endpoint behavior, authentication, rate limits, and labels, see Metrics API.

Before you begin

You need:

A VeloDB Cloud warehouse with Metrics API access enabled.
A VeloDB Cloud API key with the least-privilege role that can read the target warehouse. See Cloud API Keys.
The provider, region, and warehouse ID for the target. You need a cluster ID only if you collect cluster-level metrics.
A Datadog organization and a host on which you can install and configure Datadog Agent.
Network access from the Agent host to the VeloDB Metrics API over HTTPS and to your Datadog site.
A Datadog Agent secret backend for resolving the VeloDB API key used in the example configuration.

Metrics collected by the OpenMetrics check are billed as Datadog custom metrics. Start with a small, explicit metric and label allowlist to control cost and tag cardinality.

1. Choose an endpoint scope

You do not need to configure both endpoint types. Choose the scope that contains the metrics you need:

Scope	Use it for	Endpoint
Warehouse	Warehouse-level data, including FE and service metrics. This is the recommended starting point for the first connection test because it requires only a warehouse ID.	`https://apps-api.<REGION>.<PROVIDER>.velodb.cloud/v1/warehouse/<WAREHOUSE_ID>/metrics`
Cluster	Metrics from a specific compute or observer cluster, such as BE, cache, I/O, load, or compaction metrics.	`https://apps-api.<REGION>.<PROVIDER>.velodb.cloud/v1/warehouse/<WAREHOUSE_ID>/cluster/<CLUSTER_ID>/metrics`

For an initial connectivity test, use one warehouse endpoint. After it works, keep it only if you need warehouse-level metrics and add one separate cluster endpoint for each cluster you want to monitor. Every endpoint is an independent OpenMetrics instance and can expose different metric families.

Use the provider and region identifiers for the warehouse, not display or marketing names.

2. Verify the Metrics API

Verify the endpoint before configuring Datadog Agent. The following request checks the HTTP status and response headers without displaying the response body:

curl -sS -D - -o /dev/null \
  -H 'X-API-Key: <VELODB_API_KEY>' \
  '<VELODB_METRICS_ENDPOINT>'

Confirm that the response has:

Status 200 OK.
Content type text/plain; version=0.0.4 or another compatible Prometheus text content type.

List the metric names returned by the endpoint without printing label values:

curl -fsS \
  -H 'X-API-Key: <VELODB_API_KEY>' \
  '<VELODB_METRICS_ENDPOINT>' \
  | awk '!/^#/ && NF {name=$1; sub(/\{.*/, "", name); print name}' \
  | sort -u

To inspect the declared type for each metric family:

curl -fsS \
  -H 'X-API-Key: <VELODB_API_KEY>' \
  '<VELODB_METRICS_ENDPOINT>' \
  | awk '$1 == "#" && $2 == "TYPE" {print $3, $4}' \
  | sort -u

Do not copy the API key, complete endpoint, response body, or label values into source control, tickets, or screenshots.

Select the initial metrics

Choose one to three exact source metrics for the first connection test. The metrics must be present in the selected endpoint. Start with an always-present metric, then add activity or resource metrics only when they support your monitoring goals.

The examples later in this guide are not required metrics or a recommended monitoring baseline. They demonstrate how to map a current-value gauge, an activity counter, and a resource gauge measured in bytes.

Use the Metrics catalog to choose the monitoring category you need. If the meaning or type of a raw source metric is unclear, contact VeloDB Support before assigning a Datadog type. Do not infer the type only from a metric name or suffix.

The Metrics API recommends a scrape interval of at least 15 seconds. This guide uses 30 seconds.

3. Install Datadog Agent

Install Datadog Agent on a host that can reach both the VeloDB Metrics API and your Datadog site. Follow the Datadog Agent installation guide for your operating system.

Confirm that the Agent is running before adding the OpenMetrics configuration:

sudo datadog-agent status

4. Configure the OpenMetrics check

Create conf.d/openmetrics.d/conf.yaml under the Datadog Agent configuration directory.

The following configuration uses the latest OpenMetrics mode and a secret handle for the VeloDB API key:

init_config:

instances:
  - openmetrics_endpoint: "<VELODB_METRICS_ENDPOINT>"
    namespace: velodb
    min_collection_interval: 30
    tag_by_endpoint: false

    headers:
      Accept: text/plain
      X-API-Key: "ENC[velodb_metrics_api_key]"

    metrics:
      - "<SOURCE_METRIC_NAME>":
          name: "<DATADOG_METRIC_SUFFIX>"
          type: "<METRIC_TYPE>"

    include_labels:
      - provider
      - region
      - wid
      - cid
      - product

    exclude_labels:
      - instance
      - vpc_id
      - user
      - job
      - exported_job
      - custom_scheme
      - cluster_name

Replace all placeholders before restarting the Agent:

Placeholder	Replace with
`<VELODB_METRICS_ENDPOINT>`	The complete warehouse or cluster endpoint selected in step 1.
`<SOURCE_METRIC_NAME>`	An exact, case-sensitive name returned by the selected endpoint.
`<DATADOG_METRIC_SUFFIX>`	The name to append to the `velodb.` namespace.
`<METRIC_TYPE>`	The verified type, such as `gauge` or `counter`.

Add one metrics entry for each selected source metric. Do not use broad regular expressions such as .* or doris_.*; they can create large numbers of Datadog custom metrics.

ENC[velodb_metrics_api_key] works only after you configure a Datadog Agent secret backend that can resolve that handle. Do not replace it with a plaintext production key in the configuration file.

The label lists are a conservative starting point. Keep only the dimensions required for your dashboards and monitors. Review identifiers such as wid and cid under your data-governance policy. Keep node addresses, VPC IDs, user names, and other sensitive or high-cardinality labels excluded unless you have an approved need.

Optional example mappings

Use these mappings only if the selected endpoint returns the same source metrics:

metrics:
  - doris_fe_connection_total:
      name: fe.connections
      type: gauge
  - doris_fe_query_total:
      name: fe.queries.count
      type: counter
  - node_memory_Cached_bytes:
      name: node.memory.cached_bytes
      type: gauge

Source metric	Submitted Datadog metric	Purpose of the example
`doris_fe_connection_total`	`velodb.fe.connections`	Current connection count represented as a gauge, despite the `_total` suffix.
`doris_fe_query_total`	`velodb.fe.queries.count`	Cumulative query activity represented as a counter.
`node_memory_Cached_bytes`	`velodb.node.memory.cached_bytes`	Resource gauge with a byte unit.

These three metrics are examples only. You can replace them with any exact metrics returned by your selected endpoint, provided that you verify their meaning and type first.

VeloDB can expose raw metric families as untyped, so the examples set each type explicitly. Datadog Agent versions can normalize counter names ending in _total. If a source family is declared as a counter, current Agent versions expect the configured source name without _total and append .count. Treat the metric names reported by datadog-agent check openmetrics as authoritative and adjust the mapping if necessary. See Datadog OpenMetrics configuration.

5. Load and validate the configuration

Restart Datadog Agent using the command for your operating system, then run:

sudo datadog-agent configcheck
sudo datadog-agent check openmetrics
sudo datadog-agent status

Confirm all of the following:

configcheck loads conf.d/openmetrics.d/conf.yaml.
The OpenMetrics check reports metric samples greater than zero.
The check has no HTTP, authentication, TLS, parser, secret-resolution, or metric-mapping errors.
The submitted metric names use the expected velodb. namespace.

Allow at least two collection intervals before evaluating counter behavior.

Optional: Kubernetes collector failover with Datadog Cluster Checks

Use Datadog Cluster Checks when you run Datadog Agent on Kubernetes and want metric collection to continue after an individual Agent Pod or its node becomes unavailable. This is a Datadog deployment pattern, not a VeloDB high-availability feature.

Configure each VeloDB Metrics API endpoint once as a Datadog cluster check. The Datadog Cluster Agent dispatches each check to one eligible Agent at a time. If the assigned Agent stops reporting, Datadog can reassign the check to another eligible Agent. Do not configure the same VeloDB endpoint as independent checks on multiple Agents at the same time, as this can result in duplicate metric collection.

VeloDB exposes the same HTTPS Metrics API endpoint for this pattern; no second endpoint or additional VeloDB configuration is required. You are responsible for operating the Datadog Cluster Agent and Cluster Checks, making the VeloDB API key available to Agents eligible to run the check through your secret-management process, and ensuring that each eligible Agent can reach the endpoint over HTTPS.

When you use Cluster Checks, configure the OpenMetrics check through the Datadog Cluster Agent instead of copying the standard Agent configuration to every eligible Agent. Cluster Checks are optional. A single Datadog Agent is the recommended starting point for a basic integration. For Kubernetes setup, configuration delivery, and operational requirements, see the Datadog Cluster Checks documentation.

6. Verify metrics in Datadog

In Datadog, open Metrics > Explorer.
Use the metric query field, not the global Datadog search box.
Search for each configured metric: velodb.<DATADOG_METRIC_SUFFIX>.
Select a time range that includes at least two collection intervals after the Agent restarted.
Enable One graph per query when comparing metrics with different units or magnitudes.
Confirm that recent points have the expected warehouse or cluster tags.
Confirm that excluded labels, including instance and vpc_id, are not present.

The connection is successful when the Metrics API returns 200, the Agent check has samples and no errors, and at least one selected velodb.* metric has a recent point in Metrics Explorer.

7. Expand the integration

After the first metric is visible:

Add small groups of exact metrics based on your monitoring goals.
Confirm each source metric's availability, semantics, and type.
Review the labels and expected custom-metric cardinality.
Rerun the Agent validation commands.
Verify the submitted names and tags in Metrics Explorer.

Use a separate instances entry for every additional warehouse or cluster endpoint. Warehouses in different providers or regions use different endpoint hosts. Create dashboards and monitors after metric names, types, and tags have stabilized.

Troubleshooting

Symptom	What to check
`401` or `403` from the API	API key status, assigned role, organization, and Metrics API entitlement.
`404` from the API	Warehouse status, provider, region, warehouse or cluster ID, and endpoint path.
`429` from the API	Reduce the scrape frequency or number of targets for the key and endpoint.
`200` but no metric samples	Confirm the exact, case-sensitive source metric exists and has an explicit type mapping.
An untyped metric is skipped	Set the verified `gauge` or `counter` type explicitly.
A counter name is unexpected	Check the Agent version's `_total` normalization and the submitted name reported by `check openmetrics`.
A metric is absent from Metrics Explorer	Check the namespace, Agent output, selected time range, and Metrics Explorer query field.
Custom metric usage is unexpectedly high	Remove regular expressions and reduce the metric and label allowlists.

Keep TLS verification enabled. Do not disable certificate validation to work around network or certificate errors.

Security, cost, and cleanup

Keep VeloDB and Datadog API keys in your secret-management system and rotate them according to your security policy.
Keep tag_by_endpoint: false so the complete endpoint is not added as a tag.
Exclude sensitive and high-cardinality labels unless they are required and approved.
Review Datadog custom metric billing before expanding the metric or label allowlist.
To remove the integration, delete its OpenMetrics instance, restart the Agent, and revoke credentials that are no longer required.

For additional Agent options and version-specific behavior, see Datadog OpenMetrics.

Scope and compatibility​

Before you begin​

1. Choose an endpoint scope​

2. Verify the Metrics API​

Select the initial metrics​

3. Install Datadog Agent​

4. Configure the OpenMetrics check​

Optional example mappings​

5. Load and validate the configuration​

Optional: Kubernetes collector failover with Datadog Cluster Checks​

6. Verify metrics in Datadog​

7. Expand the integration​

Troubleshooting​

Security, cost, and cleanup​