Collect VeloDB Cloud metrics with Datadog Agent

Use Datadog Agent's OpenMetrics check to collect selected warehouse or cluster metrics from the VeloDB Cloud Metrics API and send them to your Datadog organization.

VeloDB Cloud Metrics API
        -> Datadog Agent in your environment
        -> Datadog custom metrics
        -> Metrics Explorer

The Agent pulls metrics over HTTPS and sends the collected samples to Datadog. Your VeloDB and Datadog credentials remain in your environment. This integration sends metrics only. It does not send Datadog APM traces.

Scope and compatibility

This is a Datadog-specific, end-to-end configuration guide. Datadog is the currently validated external monitoring example. For protocol-level compatibility with other collectors, see External monitoring integrations.

Note:

The Metrics API is currently available in Private Preview. Confirm that it is enabled for your organization before starting. For endpoint behavior, authentication, rate limits, and labels, see Metrics API.

Before you begin

You need:

A VeloDB Cloud warehouse with Metrics API access enabled.
A VeloDB Cloud Management API key with the least-privilege role that can read the target warehouse. See Management API Keys.
The provider, region, and warehouse ID for the target. You need a cluster ID only if you collect cluster-level metrics.
A Datadog organization and a host on which you can install and configure Datadog Agent.
Network access from the Agent host to the VeloDB Metrics API over HTTPS and to your Datadog site.
A Datadog Agent secret backend that resolves the VeloDB API key used in the example configuration.

Metrics collected by the OpenMetrics check are billed as Datadog custom metrics. Start with a small, explicit metric and label allowlist to control cost and tag cardinality.

Step 1. Choose an endpoint scope

You do not need to configure both endpoint types. Choose the scope that contains the metrics you need:

Scope	Use it for	Endpoint
Warehouse	Warehouse-level data, including FE and service metrics. This is the recommended starting point for the first connection test because it requires only a warehouse ID.	`https://apps-api.<REGION>.<PROVIDER>.velodb.cloud/v1/warehouse/<WAREHOUSE_ID>/metrics`
Cluster	Metrics from a specific compute or observer cluster, such as BE, cache, I/O, load, or compaction metrics.	`https://apps-api.<REGION>.<PROVIDER>.velodb.cloud/v1/warehouse/<WAREHOUSE_ID>/cluster/<CLUSTER_ID>/metrics`

For the initial connectivity test, use one warehouse endpoint. After it works, keep that endpoint only if you need warehouse-level metrics. Add a separate cluster endpoint for each cluster you want to monitor. Each endpoint is an independent OpenMetrics instance and can expose different metric families.

Use the provider and region identifiers for the warehouse, not display or marketing names. For more information about the endpoint format and regional lookup commands, see Regional endpoints.

Step 2. Verify the Metrics API

Verify the endpoint before configuring Datadog Agent. The following request checks the HTTP status and response headers without displaying the response body:

curl -sS -D - -o /dev/null \
  -H 'X-API-Key: <VELODB_API_KEY>' \
  '<VELODB_METRICS_ENDPOINT>'

Confirm that the response has:

Status 200 OK.
Content type text/plain; version=0.0.4 or another compatible Prometheus text content type.

List the metric names returned by the endpoint without printing label values:

curl -fsS \
  -H 'X-API-Key: <VELODB_API_KEY>' \
  '<VELODB_METRICS_ENDPOINT>' \
  | awk '!/^#/ && NF {name=$1; sub(/\{.*/, "", name); print name}' \
  | sort -u

To inspect the declared type for each metric family:

curl -fsS \
  -H 'X-API-Key: <VELODB_API_KEY>' \
  '<VELODB_METRICS_ENDPOINT>' \
  | awk '$1 == "#" && $2 == "TYPE" {print $3, $4}' \
  | sort -u

Do not copy the API key, complete endpoint, response body, or label values into source control, tickets, or screenshots.

Select the initial metrics

Choose one to three exact source metrics for the first connection test. The metrics must be present in the selected endpoint. Start with an always-present metric, then add activity or resource metrics only when they support your monitoring goals.

The examples later in this guide are not required metrics or a recommended monitoring baseline. They demonstrate how to map a current-value gauge, an activity counter, and a resource gauge measured in bytes.

Use the Metrics catalog to choose the monitoring category you need. If the meaning or type of a raw source metric is unclear, contact VeloDB Cloud support before assigning a Datadog type. Do not infer the type only from a metric name or suffix.

The Metrics API recommends a scrape interval of at least 15 seconds. This guide uses 30 seconds.

Step 3. Install Datadog Agent

Install Datadog Agent on a host that can reach both the VeloDB Cloud Metrics API and your Datadog site. Follow the Datadog Agent installation guide for your operating system.

Confirm that the Agent is running before adding the OpenMetrics configuration:

sudo datadog-agent status

Step 4. Configure the OpenMetrics check

Create conf.d/openmetrics.d/conf.yaml under the Datadog Agent configuration directory.

The following configuration uses the latest OpenMetrics mode and a secret handle for the VeloDB API key:

init_config:

instances:
  - openmetrics_endpoint: "<VELODB_METRICS_ENDPOINT>"
    namespace: velodb
    min_collection_interval: 30
    tag_by_endpoint: false

    headers:
      Accept: text/plain
      X-API-Key: "ENC[velodb_metrics_api_key]"

    metrics:
      - "<SOURCE_METRIC_NAME>":
          name: "<DATADOG_METRIC_SUFFIX>"
          type: "<METRIC_TYPE>"

    include_labels:
      - provider
      - region
      - wid
      - cid
      - product

    exclude_labels:
      - instance
      - vpc_id
      - user
      - job
      - exported_job
      - custom_scheme
      - cluster_name

Replace all placeholders before restarting the Agent:

Placeholder	Replace with
`<VELODB_METRICS_ENDPOINT>`	The complete warehouse or cluster endpoint selected in Step 1.
`<SOURCE_METRIC_NAME>`	An exact, case-sensitive name returned by the selected endpoint.
`<DATADOG_METRIC_SUFFIX>`	The name to append to the `velodb.` namespace.
`<METRIC_TYPE>`	The verified type, such as `gauge` or `counter`.

Add one metrics entry for each selected source metric. Do not use broad regular expressions such as .* or doris_.*. They can create large numbers of Datadog custom metrics.

ENC[velodb_metrics_api_key] works only after you configure a Datadog Agent secret backend that can resolve that handle. Do not replace it with a plaintext production key in the configuration file.

The label lists are a conservative starting point. Note the following:

Keep only the dimensions required for your dashboards and monitors.
Review identifiers such as wid and cid under your data-governance policy.
Keep node addresses, VPC IDs, user names, and other sensitive or high-cardinality labels excluded unless you have an approved need.

Optional example mappings

Use these mappings only if the selected endpoint returns the same source metrics:

metrics:
  - doris_fe_connection_total:
      name: fe.connections
      type: gauge
  - doris_fe_query_total:
      name: fe.queries.count
      type: counter
  - node_memory_Cached_bytes:
      name: node.memory.cached_bytes
      type: gauge

Source metric	Submitted Datadog metric	Purpose of the example
`doris_fe_connection_total`	`velodb.fe.connections`	Current connection count represented as a gauge, despite the `_total` suffix.
`doris_fe_query_total`	`velodb.fe.queries.count`	Cumulative query activity represented as a counter.
`node_memory_Cached_bytes`	`velodb.node.memory.cached_bytes`	Resource gauge with a byte unit.

These three metrics are examples only. You can replace them with any exact metrics returned by your selected endpoint, provided that you verify their meaning and type first.

VeloDB Cloud can expose raw metric families as untyped, so the examples set each type explicitly. Datadog Agent versions can normalize counter names ending in _total. If a source family is declared as a counter, current Agent versions expect the configured source name without _total and append .count. Treat the metric names reported by datadog-agent check openmetrics as authoritative and adjust the mapping if necessary. See Datadog OpenMetrics configuration.

Step 5. Load and validate the configuration

First validate the configuration. On a Linux host that uses systemd, run:

sudo datadog-agent configcheck

If the configuration is valid, restart the Agent and run the OpenMetrics check:

sudo systemctl restart datadog-agent
sudo datadog-agent check openmetrics
sudo datadog-agent status

For other operating systems, use the corresponding restart command from the Datadog Agent documentation.

Confirm all of the following:

configcheck loads conf.d/openmetrics.d/conf.yaml without configuration errors.
The OpenMetrics check reports metric samples greater than zero.
The check has no HTTP, authentication, TLS, parser, secret-resolution, or metric-mapping errors.
The submitted metric names use the expected velodb. namespace.

Allow at least two collection intervals before evaluating counter behavior.

Step 6. Verify metrics in Datadog

In Datadog, open Metrics > Explorer.
Use the metric query field, not the global Datadog search box.
Search for each configured metric: velodb.<DATADOG_METRIC_SUFFIX>.
Select a time range that includes at least two collection intervals after the Agent restarted.
Enable One graph per query when comparing metrics with different units or magnitudes.
Confirm that recent points have the expected warehouse or cluster tags.
Confirm that excluded labels, including instance and vpc_id, are not present.

The connection is successful when the Metrics API returns 200, the Agent check has samples and no errors, and at least one selected velodb.* metric has a recent point in Metrics Explorer.

Step 7. Expand the integration

After the first metric is visible:

Add small groups of exact metrics based on your monitoring goals.
Confirm each source metric's availability, semantics, and type.
Review the labels and expected custom-metric cardinality.
Rerun the Agent validation commands.
Verify the submitted names and tags in Metrics Explorer.

Use a separate instances entry for every additional warehouse or cluster endpoint. Warehouses in different providers or regions use different endpoint hosts. Create dashboards and monitors after metric names, types, and tags have stabilized.

Optional: Kubernetes collector failover with Datadog Cluster Checks

Use Datadog Cluster Checks when you run Datadog Agent on Kubernetes and want metric collection to continue after an individual Agent Pod or its node becomes unavailable. This is a Datadog deployment pattern, not a VeloDB Cloud high-availability feature.

Configure each VeloDB Cloud Metrics API endpoint once as a Datadog cluster check. The Datadog Cluster Agent dispatches each check to one eligible Agent at a time. If the assigned Agent stops reporting, Datadog can reassign the check to another eligible Agent. Do not configure the same VeloDB Cloud endpoint as independent checks on multiple Agents at the same time, because this can result in duplicate metric collection.

VeloDB Cloud exposes the same HTTPS Metrics API endpoint for this pattern. No second endpoint or additional VeloDB Cloud configuration is required. You are responsible for operating the Datadog Cluster Agent and Cluster Checks, making the VeloDB Cloud API key available to Agents eligible to run the check through your secret-management process, and ensuring that each eligible Agent can reach the endpoint over HTTPS.

When you use Cluster Checks, configure the OpenMetrics check through the Datadog Cluster Agent instead of copying the standard Agent configuration to every eligible Agent. Cluster Checks are optional. A single Datadog Agent is the recommended starting point for a basic integration.

For Kubernetes setup, configuration delivery, and operational requirements, see the Datadog Cluster Checks documentation.

Troubleshooting

Symptom	What to check
`401` or `403` from the API	API key status, assigned role, organization, and Metrics API entitlement.
`404` from the API	Warehouse status, provider, region, warehouse or cluster ID, and endpoint path.
`429` from the API	Reduce the scrape frequency or number of targets for the key and endpoint.
`200` but no metric samples	Confirm the exact, case-sensitive source metric exists and has an explicit type mapping.
An untyped metric is skipped	Set the verified `gauge` or `counter` type explicitly.
A counter name is unexpected	Check the Agent version's `_total` normalization and the submitted name reported by `check openmetrics`.
A metric is absent from Metrics Explorer	Check the namespace, Agent output, selected time range, and Metrics Explorer query field.
Custom metric usage is unexpectedly high	Remove regular expressions and reduce the metric and label allowlists.

Keep TLS verification enabled. Do not disable certificate validation to work around network or certificate errors.

Best practices for security and cost

Keep VeloDB Cloud and Datadog API keys in your secret-management system and rotate them according to your security policy.
Keep tag_by_endpoint: false so the complete endpoint is not added as a tag.
Exclude sensitive and high-cardinality labels unless they are required and approved.
Review Datadog custom metric billing before expanding the metric or label allowlist.
To remove the integration, delete its OpenMetrics instance, restart the Agent, and revoke credentials that are no longer required.

For additional Agent options and version-specific behavior, see Datadog OpenMetrics.

Scope and compatibility​

Before you begin​

Step 1. Choose an endpoint scope​

Step 2. Verify the Metrics API​

Select the initial metrics​

Step 3. Install Datadog Agent​

Step 4. Configure the OpenMetrics check​

Optional example mappings​

Step 5. Load and validate the configuration​

Step 6. Verify metrics in Datadog​

Step 7. Expand the integration​

Optional: Kubernetes collector failover with Datadog Cluster Checks​

Troubleshooting​

Best practices for security and cost​

Scope and compatibility

Before you begin

Step 1. Choose an endpoint scope

Step 2. Verify the Metrics API

Select the initial metrics

Step 3. Install Datadog Agent

Step 4. Configure the OpenMetrics check

Optional example mappings

Step 5. Load and validate the configuration

Step 6. Verify metrics in Datadog

Step 7. Expand the integration

Optional: Kubernetes collector failover with Datadog Cluster Checks

Troubleshooting

Best practices for security and cost