VeloDB Manager 24.x Installation Manual

VeloDB Manager is a tool introduced by VeloDB for managing and operating Apache Doris and VeloDB Enterprise clusters. You can follow the instructions below to understand the overall architecture of the VeloDB Manager 24.0.0 version service, and download and deploy VeloDB Manager 24.0.0.

Note: VeloDB Manager 24.0.0 is not compatible with the previous 23.x series SSH mutual trust mode. If you want to use Agent mode 24.0.0 VeloDB Manager, you need to reinstall it and cannot upgrade directly from the 23.x series version.

VeloDB Manager Service Structure

Unlike the SSH mutual trust mode of the historical version of the 23.x series, The management and control of VeloDB Manager version 24.0.0 employs the Agent approach. The Agent and Server communicate directly using the HTTP protocol, which can be combined with SSL encryption to ensure security. The overall architecture of the service is shown in the following figure.

VeloDB Manager Web Server

VeloDB manager web server is the server level of the Manager web server, and it is also the core module of the entire VeloDB Manager to achieve automatic operation and maintenance management of Apache Doris and VeloDB Doris cluster. The main functions include:

Manage Doris cluster, including deployment, hosting, expansion, reduction, upgrade, configuration modification, restart and other control operations.
Provide service API for web access.
Store metadata for all business functions of VeloDB Manager.
Provide Doris cluster operation and maintenance monitoring and alarm component function.
Provide a user authentication system for natural persons to access VeloDB Manager, and implement a permission system based on Doris cluster function management.
Manage the Doris cluster nodes in the Agent unified management system and issue control instructions to the Agent.

VeloDB Manager Agent

The Doris cluster controlled by VeloDB Manager requires the installation of a resident Agent process on each Doris cluster node. The default port for Agent is 8972, and it must be connected to the machine where the VeloDB Manager Web service is located. The main functions of Agent include:

Receive control commands issued by VeloDB Manager Web service side, execute and return execution results.
Collect monitoring metrics of Doris Cluster node hosts and proactively report them to the monitoring component storage on the VeloDB Manager Web service side.
Regularly report the heartbeat information of hosts and Doris processes on the Doris Cluster node to the VeloDB Manager Web service.

Deploy VeloDB Manager Web Server

Installation package download and instructions

Download link

https://www.velodb.io/download/enterprise#manager (opens in a new tab)

Installation package instructions

Installation package name
1. velodb-manager-24.0.0-x64-bin.tar.gz
Unzip the installation package

$ tar -zxvf velodb-manager-24.0.0-x64-bin.tar.gz

Installation package directory structure

velodb-manager-24.0.0-x64-bin
    webserver // VeloDB Manager web server component, which is a web portal service that needs to be manually started
       bin  // start/stop script
       conf  // configuration file
       lib  // service binary
       static  // frontend static files
       config-tool  // VeloDB Manager service management tool
       inspection  // inspection script
    deps // VeloDB Manager manager control dependency component
       alertmanager // alert tool
       jdk // jdk dependency package
       prometheus // monitor metrics storage tool
       grafana // monitor dashboard tool
       Doris-Dashboard.json // default dashboard json file, the name is based on the actual
    agent
       manager-agent-24.0.0-x64-bin.tar.gz // The compressed installation package for VeloDB Manager's Agent. Please note that this compressed package cannot be deleted.

Manually deploying VeloDB Manager Web service components

After decompressing the VeloDB Manager installation package, you will obtain velodb-manager-24.0.0-rc1-x64-bindirectory.

Modify the installation package directory name

It is recommended to rename the velodb-manager-24.0.0-rc1-x64-bin directory to something else, such as velodb-manager before starting the service.

$ mv velodb-manager-24.0.0-x64-bin velodb-manager

Enter the installation path

$ cd velodb-manager

Configure web services (optional)

Read the conf/manager.conf file under the webserver path in the installation directory

# VeloDB Manager Web server port that the service component is listening on
MANAGER_PORT=8004

# Service-dependent database type: MySQL, h2, or PostgreSQL. Default is h2
DB_TYPE=h2

# If the database type is h2, the default data storage path is the data path under the webserver directory.
# Users can also specify a custom absolute path.
DATA_PATH=

# If the database type is MySQL, configure the access address of the MySQL
DB_HOST=

# Configure access port of the MySQL/PostgreSQ
DB_PORT=

# Configure access users for MySQL/PostgreSQL
DB_USER=

# Configure user passwords for accessing MySQL/PostgreSQL 
DB_PASS=

# Configure access database name of MySQL/PostgreSQL 
DB_DBNAME=

# The suffix of the MySQL data connection URL
DB_URL_SUFFIX='?useSSL=false&useUnicode=true&characterEncoding=UTF-8'

# Configure the HTTP handshake timeout duration (in seconds). The default is 30 seconds.
HTTP_CONNECT_TIMEOUT=30

# Configure the HTTP response receive timeout duration (in seconds). The default is 60 seconds.
HTTP_SOCKET_TIMEOUT=60

# The IP protocol that the service listens on supports ALL, IPV4, and IPV6. ALL indicates support for both IPV4 and IPV6.
LISTEN_PROTOCOL=ALL

# The minimum free disk space required for the FE module installation path during an upgrade is 10G by default.
FE_MIN_DISK_SPACE_FOR_UPGRADE=10

# The minimum free disk space required for the BE module installation path during an upgrade is 10G by default.
BE_MIN_DISK_SPACE_FOR_UPGRADE=10

Start Web Server

Run the script directly in the webserver directory.

$ sh bin/start.sh

Access Web Server

Directly enter the URL through the browser - http://{Web service machine IP}: {Web server level port}, for example http://172.20.17.107:8004 (opens in a new tab)

Automatically deploy service components through web services

Initialize VeloDB Manager administrator account

When accessing the web service for the first time, enter the Initialize Users page and create the first VeloDB Manager administrator user.

Configure service component deployment information

After creating the VeloDB Manager administrator user, click **Next **to enter the service component configuration page and fill in the configuration information.

The settings are divided into two parts, namely component configuration and installation package configuration.

Component configuration : VeloDB Manager monitors the configuration of components related to the alarm service.
- Monitoring Alarm Service : Optional, used to configure the monitoring alarm module of VeloDB Manager. Here, Grafana, Prometheus, and Alertmanager will be installed. You need to select three available ports on the machine where VeloDB Manager is installed.
  - Email Alarm : Configure the mail server and then use the "mail alerts" channel in alerts.
  - Advanced Settings : Optional for monitoring access addresses and proxy configurations.
    - Monitoring access address: Used to solve the problem that monitoring metrics cannot be viewed when accessing VeloDB Manager through domain name.
    - Proxy configuration: If the production environment and extranet are isolated, you can set the proxy to send notifications to the public network work communication software.
Package Configuration : Configure the local Doris Core and VeloDB Manager installation package storage path for new and upgraded clusters.

Deploy service components

Click Start VeloDB Manager to prompt that VeloDB Manager is being started.

After a successful startup, the user is prompted to enter the VeloDB Manager.

Deploy VeloDB Manager Agent

After the successful deployment of VeloDB Manager Web service, you can deploy or take over the Doris cluster through the service page, and you will be prompted to install the Agent. The specific operation steps are:

View Agent Installation Guide

Click View to enter the Agent installation guide details page:

Note: If you directly copy the compressed package of the Agent in the installation path of the Manager Web service, do not delete the Agent compressed package in the installation path of the Manager Web service after copying.

Install Agent

Note: After installing the Agent on the server, when managing the Doris cluster, the Agent will operate the corresponding instructions. Therefore, it is equivalent to using the machine user who deployed the Agent to deploy the Doris cluster, and attention should be paid to the content of relevant permissions.

Log in to the Doris cluster node, download or copy the installation package to the corresponding Doris cluster node

The download method selected here:

Unzip the Agent installation package and modify the installation path name

Start Agent

Go to the Agent installation directory, and the Agent directory structure is as follows:

manager-agent
   bin  // start/stop script
   conf  // configuration file
   lib  // service binary

Start Agent：

Execute crontab -e to check if the Agent process's resident keep-alive mechanism is effective.

回到 VeloDB Manager 页面，确认 Agent 状态正常

Upgrade VeloDB Manager Service

The Agent mode of VeloDB Manager 24.0.0 version is not compatible with the SSH mode of the previous 23.x series version, so the upgrade here refers to directly upgrading from the Agent mode 24.0.0 future versions. The specific process of the upgrade is as follows:

View the current version

Click on the user information in the lower left corner of the page to view the current version of VeloDB Manager.

Download the new version of VeloDB Manager installation package

The upgrade page will prompt the storage path of the target version installation package on the VeloDB Manager server.

Directly log in to the VeloDB Manager server machine and save the new version of the installation package into the directory. Then refresh the page to view and select the target version.

Click the Upgrade button to enter the upgrade page.

Upgrade VeloDB Manager

Note: The webserver component will also be restarted during the upgrade, so the page will briefly exit automatically. Please log in again.

Uninstall VeloDB Manager Service

Uninstall the current version of VeloDB Manager service

Stop the non-webserver service component

Find the page of the component service and stop all components that appear on that page

Stop the webserver service

Execute the command to remove the automatic pull-up mechanism of the webserver component, crontab -l | grep -v. doris_manager | crontab -
Uninstall webserver service: enter the manager deployment directory, cd webserver; sh bin/stop.sh
Remove the relevant installation path of VeloDB Manager Web service

Stop the Agent on each Doris cluster node

Enter the Agent installation path and execute: sh bin/stop.sh
Delete the relevant installation path of the Agent

Uninstall VeloDB Manager service for 23.x series version

Stop the non-webserver service component

Find the component service page and manually stop all components that appear on the page

Stop the webserver service

Execute the command to remove the automatic pull-up mechanism of the webserver component, crontab -l | grep -v. doris_manager | crontab -
Stop the webserver service: go to the manager deployment directory, cd webserver; sh bin/stop.sh

Version 23.x Version 23.x