WEBUI
VeloDB Cloud WebUI ("WebUI") is a data development platform for data development scenarios. It is a data development platform on the cloud provided by VeloDB, which can assist users in managing and exploring data, and can replace Navicat.
Main Function
- Warehouse Login : Use different database users to log in to the warehouse in the WebUI.
- Data Management: View and manage data in the database, currently supports viewing.
- Data query: An easy-to-use SQL query editor that supports query execution, viewing query profiles, viewing query history and saving queries.
- Data Integration: Easily connect to data in object storage on the cloud, connect to data lakes, and import sample data.
- Privilege Management: Manage users and roles in the database, and grant and revoke permissions to them.
Register and Login
Using the WebUI service
In VeloDB Cloud Manager ("Manager"), each warehouse has a corresponding WebUI service. In the "Connection" module of Manager, you can find the entrance to the WebUI through a private network or a public network.
You can also save the entry address of the WebUI for direct access.
Login to WebUI
You need to enter the Username Password and Warehouse Name of the database on the login page. If you clicked the link to log in from the Manager, the warehouse name should be pre-filled.
We will not record your login account and password, but you can use the recording function that comes with your browser.
Data
The "data" module is the basic function of WebUI to manage the database, and it mainly has two functions:
1、 Check the data and its organizational form, such as database table structure, data size, table creation statement, table field information, data preview, etc.
2、 Add, delete and modify data, including creating and deleting database objects, modifying storage strategies, etc. This part of the functionality is still being implemented and is temporarily unavailable.
The data module is displayed according to the organizational form of the data in the database, and is divided into Catalog -Database -Table /View .
Catalog
Catalog is a collection of databases.
Catalog is divided into internal catalog and external catalog. Internal catalog contains Doris's own database, external catalog can connect to Hive, Iceberg, Hudi, etc. and perform queries.
Currently, you can create an external directory by creating a new "Hive-OSS" integration in the "Integration" module to perform data lake queries.
Database
Database is a collection of tables and views. Database belongs to catalog.
When you select a directory, you can view the database under the directory, as well as the size of the database and the modification time. At the same time, you can delete and create new databases.
Table
Table is the basic unit of VeloDB data warehouse, and table belongs to database.
When a database is selected, you can see the tables under the database, as well as the size of the table, creation and modification time.
In the current version, you can delete the table, and in the future version, you can create the table and manage the life cycle of the table in the WebUI.
Table Details
On the table details page, you can view the table definition and some basic attributes.
In the future, we will support the viewing and management of the life cycle of the table here, and provide more abundant storage-related statistical information.
In the future, VeloDB WebUI will also support uploading local files to tables to complete the lightweight table import function.
Field Information
Here you can see the field information of the Base Table and its Rollup or materialized view.
Index
Here is the index information in the table. There are many index-like acceleration methods in VeloDB. Here, partitions, primary keys, and rollups are not treated as indexes for the time being, and only Zonemap, Bloomfilter, Bitmap/Inverted Index are displayed.
Data Overview
Viewing the overview will consume cluster resources, and you must have an available cluster to view the data overview.
"Total x data" is fetched from the metadata service, so there may be a delay and does not reflect the actual number of table rows.
View
A view is a table based visualization of the result set of an SQL statement. In WebUI, we treat views separately from tables, and materialized views are not within the scope of views.
The view page is roughly similar to the table page, and the attributes that the view does not have (such as index and details) will not be displayed.
You can preview the data, but the preview view data will initiate queries and consume cluster resources.
Query
The query result will be returned below the edit box, and the error or success status and information returned by the query will also be displayed at the query result.
At the same time, you can click the drop-down button on the right side of Run (LIMIT 1000) and switch to Run and Download to download your query results.
Reminder Currently, if multiple queries are executed at one time, the result of the first Query will be returned, and we will add the function of viewing other returns in later versions.
At the same time, when the "query" module initiates a query, there is no context association. Therefore, you may encounter the problem of missing context such as USE [DATABASE] not taking effect. We will add context association when querying in the next version.
You can click the Query ID of the query history to view the query details.
NOTE There is no Query ID for non-query statements, nor for failed statements.
By default, query plans are enabled for queries initiated in the WebUI, which will not affect the performance of a single query. Click "Query Statement" to enter the execution plan page.
You can find "Save Query" and "Save as new query" in the upper right corner of the editor.
Open SQL from "Saved SQL" and make changes later, these changes will be saved.
In the saved query, you can see the test statement of our pre-made test data set.
Integrations
Integrations are portals connecting VeloDB Cloud with data outside the warehouse.
New Integration
Currently, you can create two new integrations, namely Stage integration (object storage) and sample data.
Object Storage
By creating a new object storage integration, you can establish a Connection with data in object storage. Through the Integrate + Copy Into command, you can Import the data in the object storage to the warehouse.
When creating a new object storage integration, you need to enter the following:
- Integration Name: Consistent with the database object naming rules, up to 64 characters, letters, numbers, and underscores can be used.
- Comments: Integrated comments.
- Bucket : The bucket you need to integrate.
- Default file path: The file path to be accessed in the bucket. VeloDB will only access the files under the path you fill in. If you do not fill in, the default is that the data in the entire bucket can be accessed.
- Access Authorization: The way to allow VeloDB to access your bucket. It is divided into Access key and cross-account authorization. We recommend using cross-account authorization for better security. For guidelines on cross-account authorization, please refer to: Cross Account IAM Guide (opens in a new tab)。You must pass the permissions check to successfully create an integration.
- Advanced Configuration: Details below.
Advanced Configuration
Divided into File Type and Import Configuration. These are the parameters that you may use when importing integrated data. You can set them here, or specify them when importing. If you do not set or specify them, the system will execute the import task of the integration with the default configuration.
File Configuration
- File type: The default type of the integrated storage file, currently supports
csv,json,orc,parquet. The default is that the system infers from the filename suffix. - Compression method: The default compression type of the integrated storage file, currently supports
gz,bz2,lz4,lzo,deflate. The default is that the system infers from the filename suffix. - Column separator: The default column separator of the integrated storage file, the default
\t. - Line separator: The default line separator of the integrated storage file, the default
\n. - File size: When importing files under this integration, the default import size limit is unlimited by default.
Import Configuration
- On Error: When importing files under this integration, when the data quality is unqualified, the default error handling method. There are three types: continue importing, stop importing, and continue importing when the proportion of error data does not exceed a certain value.
- Strict Mode: Strictly filter the column type conversion during the import process. Default is off.
Sample Data
Creating a new sample data integration will import sample data into the database on the basis of creating an object storage integration. Therefore, you need to select the cluster to complete the new creation. TPCH, Github Event, SSB-FLAT test data size has the following choices: sf1 (1GB), sf10 (10GB), sf100 (100GB), select through the drop-down menu, and the test warehouse can only choose 1sf (1GB).
Clickbench only has the option of sf100 (100GB), we recommend that you use a larger cluster to import Clickbench sample data.
You can view the import progress in the sample data details.
Permissions
User
Display the users in the VeloDB repository. Note that the root user will not be displayed here.
Only users with Admin authority can add and modify other users.
You can create a new user on this page, except for the username, other content is optional. However, we strongly recommend that you add passwords for users and restrict access to hosts for enhanced security.
Role
Here you can manage the roles in VeloDB, and also perform authorized operations on the roles.
Only users with Admin permissions can add and modify other roles.
VeloDB currently does not support managing users under roles through roles, which means you need to specify your roles when creating or modifying users.
Authorize
On the details page of a user or role, you can perform authorization/revocation operations.
You need to have Admin or the Grant permission at the corresponding level to perform authorization/revocation work.
In WebUI, we divide permissions into four categories, namely:
- Global: Global permissions are the permissions at the Global level. With global permissions, you automatically have the corresponding permissions for all corresponding objects in the database.
- Data: Refers to the authority of data resources. You can authorize according to the level. With the authority of the parent level, you automatically have the corresponding authority of its child content.
- Integration: The authority to external integration, currently it is actually the authority to object storage integration.
- Cluster: The permission of computing resources that can be used.
