Core Release Notes

This article describes the release notes for VeloDB Core.

3.0

Note VeloDB Core 3.0 was developed based on Apache Doris 2.0.

March 31, 2024（3.0.7）

New Features

Supported audit log plug-in (audit log table needs to be created manually).

Improvements

Merged all features, improvements and bug fixes from Apache Doris 2.0.6 (opens in a new tab).
If the user does not set a default cluster when executing the query, the available (live, authorized) clusters will be selected in dictionary order of cluster names.
After the cluster automatically stopped, it is no longer necessary to explicitly use cluster to trigger automatic start of the cluster.
Optimized availability and performance of reverse index and log analysis scenarios.
The new optimizer optimizes ambiguous projection having syntax.

Bug Fixes

Fixed the issue where a large number of delete requests triggered import failure.
Fixed the issue where the statistics logic repeatedly triggers SQL, causing the cluster's automatic stop logic to fail.
Fixed the issue with importing specified UTC time zone conversion errors.

February 05, 2024（3.0.6）

New Features

Supported automatic analyze.
Supported wide table import.
Supported group commit (batch/real-time import).
Supported in left join infer.

Improvements

Merged all features, improvements and bug fixes from Apache Doris 2.0.4 (opens in a new tab).
Optimized the error message when import fails.

Bug Fixes

Fixed the issue where cluster restart may cause cluster permissions to become invalid.
Fixed the problem of failure to upgrade from 2.4.x series version to 3.0.x series version.
Fixed the problem of misaligned output of the show processlist command.
Fixed the problem that altering table‘s attribute TTL may not take effect (the displayed value is 0).
Fixed the issue where the queries would occasionally receive the error failed to initialize storage reader.
Fixed the issue where abnormal paths in parquet reader may cause memory leaks.
Fixed the pushdown problem of date_sub, data_add, to_date functions.
Fixed the issue where the colocate jion plan failed due to colocate participating in balance.
Fixed the problem that altering colocate‘s attribute replay would fail.
Fixed the issue where data cannot be queried due to special characters in the delete condition.
Fixed the issue of read query reporting NOT_IMPLEMENTED_ERROR with a small probability.
Fixed the issue where importing time zone conversions resulted in incorrect data.
Fixed the issue where passing negative values to substr() would cause coredump.
Fixed several stability issues.

December 20, 2023（3.0.5）

New Features

Provided stream load compatible API.

/api/{db}/{table}/_stream_load_2pc

Opened the whitelist of show /proc/statistic.

Improvements

Merged all features (it is not recommended to enable automatic analyze function), improvements and bug fixes from Apache Doris 2.0.3 (opens in a new tab).
Improved the stability of mow table import.
Improved the speed of dynamic table creation.
Improved the performance of Plan generation.
Optimized decimal operation behavior.
Improved stability.

Bug Fixes

Fixed the issue where insufficient S3 buffer may cause writing to become stuck.
Fixed the problem that the query may get an error "failed to find version path" in multi-cluster scenarios.
Fixed the problem that the Plan may cross clusters in multi-cluster scenarios.
Fixed the problem that dynamic partition cannot create a new partition due to changing the length of varchar column type.
Fixed the issue that show create table decimal is not compatible with SmartBI.
Fixed the issue where column type conversion functionality might become unavailable.
Fixed the problem that row storage table attributes did not take effect.
Fixed the issue where base compaction could fail.
Fixed the problem that single point query may not find the latest data.
Fixed the issue where copy into duplicate files needs to wait for timeout.
Fixed the issue where query_parallel_instance_num could be divided by 0.
Fixed the issue where cache hit rate calculation may be inaccurate.
Fixed the problem that ngram index does not take effect.

November 23, 2023（3.0.4）

New Features

Supported show proc current_stmt。

Improvements

Enhanced access control security.
Enhanced the reliability of uploaded data verification.
Improved warehouse metadata service availability.
Optimized the error message when creating table dynamic_partition.replication_num.

Bug Fixes

Fixed an issue where upgrading VeloDB Core 2.4 to 3.0 may cause copy into to fail.
Fixed an issue where importing authorization using external stage may fail.
Fixed an issue where routine load import may fail.
Fixed an issue where MOW table import may fail.
Fixed an issue where real-time writing of insert statements may fail.
Fixed tpcds performance degradation issue.
Fixed the problem that non-global session variables may not take effect.
Fixed the issue where a large number of where conditions in a query would cause high CPU consumption and slow execution.

November 02, 2023（3.0.3）

New Features

Supported Doris Clean Label.
Stream Load cloud API is compatible with Doris community behavior.
The warehouse had built-in JDBC Driver.

Improvements

The new optimizer top n supported runtime filter clipping by default.

Bug Fixes

Fixed the problem of slow query response due to resource leakage in the warehouse.
Fixed the problem that function statement disabling does not take effect.
Fixed an issue where schema change would fail when re-arranging.

October 17, 2023（3.0.2）

New Features

Added JDBC Driver URL whitelist function for S3 protocol import.

Improvements

Merged all features, improvements and bug fixes from Apache Doris 2.0.2 (opens in a new tab).
Merged all features, improvements and bug fixes from VeloDB Core 2.3.10 and 2.4.4.
Reduced the time to obtain metadata during Plan and speeds up Plan generation.
Added time-consuming statistics for query profile accessing metadata.
Balanced and optimized resource utilization.

Bug Fixes

Fixed the issue of read and write errors after setting File Cache TTL.
Fixed the problem of service unavailability caused by querying the backends table or rowsets table.

September 18, 2023（3.0.1-beta）

Improvements

Merged all features, improvements and bug fixes from Apache Doris 2.0.1 (opens in a new tab).
Merged all features, improvements and bug fixes from VeloDB Core 2.4.3.
Optimized the time consumption of get version.

2.4

Note VeloDB Core 2.4 was developed based on Apache Doris 1.2.

November 14, 2023（2.4.6）

Bug Fixes

Fixed several compute node memory leaks.

November 06, 2023（2.4.5）

Improvements

Optimized the balance strategy.

Bug Fixes

Fixed the query failure problem caused by replace tmp partition.
Fixed the deadlock problem caused by mixing pthread and bthread.

September 20, 2023（2.4.4）

Improvements

Merged all features, improvements and bug fixes from VeloDB Core 2.3.9 and 2.3.10.
Improved the usability of the import function.

Bug Fixes

Fixed the problem of unresponsive mysql session (requesting truncate table, which contains a large number of partitions).

September 11, 2023（2.4.3）

New Features

Supported suspension of service due to arrears.

Improvements

Merged all features, improvements and bug fixes from VeloDB Core 2.3.8.
Compatible with s3 load endpoint with http string.

Bug Fixes

Fixed the docker environment time zone issue.

August 28, 2023（2.4.2）

New Features

Supported the copy into import method to set constant values.

Improvements

Improved the accuracy of storage information statistics.

Bug Fixes

Fixed the issue where cloud stream load might fail.

August 3, 2023（2.4.1）

New Features

Supported on-demand cluster auto start when receiving a request.
Supported stream load.

Improvements

Merged all features, improvements and bug fixes from VeloDB Core 2.3.5, 2.3.6 and 2.3.7.

Bug Fixes

Fixed the issue where inverted index queries could fail.

July 3, 2023（2.4.0）

New Features

Supported controlling whether SQL uses cache through session variable.
Supported incremental data synchronization of imported data between clusters.
Supported cache data preheating function based on table cluster.
Supported querying cache usage through SQL.
Supported on-demand cluster auto stop when idle.

Improvements

Enhanced data security.
Improved the response speed when the query misses the cache.

Bug Fixes

Fixed the issue where set default cluster might fail.
Fixed the issue where the upgrade process could fail.

2.3

Note VeloDB Core 2.3 was developed based on Apache Doris 1.2.

October 16, 2023 (2.3.11)

Improvements

Optimized the balance strategy.

Bug Fixes

Fixed the query failure problem caused by replace tmp partition.

September 20, 2023 (2.3.10)

Bug Fixes

Fixed the problem of time extension in copy into import method.

September 11, 2023 (2.3.9)

Bug Fixes

Fixed the problem that the in operator for bitmap type columns caused the query to be unavailable.
Fixed decimal calculation accuracy problem.
Fixed the bug that when the alter statement changes more columns, the key column changes will also go through lightweight schema change.

August 29, 2023 (2.3.8)

Improvements

Optimized the speed of importing and uploading small objects.

Bug Fixes

Fixed the problem of incorrect division accuracy in decimal v3 in some cases (increased 100 times accuracy).

August 2, 2023 (2.3.7)

Bug Fixes

Fixed the issue where incorrect type inference for decimal v2 calculations resulted in inaccurate precision.
Fixed the issue where create catalog relied on an external driver, causing the warehouse to fail to start.
Fixed the issue where indexes will be created for all columns when adding bitmap index (which may cause data import failure).
Fixed the issue where the default value of the copy into import method does not take effect.

July 25, 2023 (2.3.6)

Improvements

Improved the usability of the import function.
The round precision behavior of decimal v2 is supported and is compatible with decimal v3.

Bug Fixes

Fixed the issue where queries could get stuck.
Fixed the problem of access failure that may report 403 when importing error_log.

July 19, 2023 (2.3.5)

Improvements

Reduced the delay when importing small batches of data with partition skew.
Added the limit configuration of the length of the audit log insert into statement, and the default is 4KB truncation.
Improved remote IO efficiency and accelerated query speed.

Bug Fixes

Fixed the issue where an error would be prompted if the column name in the delete statement condition was capitalized.
Fixed known memory leak issue.
Fixed the issue where tablet balance could be uneven.
Fixed the issue where node restart might cause query failure.
Fixed the problem of OSS S3 load error and added AWS_PROVIDER attribute.

July 7, 2023 (2.3.4)

Bug Fixes

Fixed the issue where queries might become unavailable.

July 3, 2023 (2.3.3)

Improvements

Speeded up node restart speed and improved availability.

Bug Fixes

Fixed several memory leak issues.

June 27, 2023 (2.3.2)

Improvements

Upgraded the warehouse mariadb client version.
Reduced the dimension statistics of file cache metrics.
Added indicators for warehouse Fragment execution.

Bug Fixes

Fixed the issue where the query is canceled but the hash table is not released during query execution.
Fixed the bug that inverted index files were not recycled after compaction.
Added null check to sql parser.
Fixed the bug that some segments could not be opened after limit optimization.

May 16, 2023 (2.3.0）

New Features

Supported MySQL TLS connection (closed by default, enabling it requires separate configuration, and requires mysql client 5.0.38 or above).
Supported Prepare stmt.
Supported smooth node reduction, reducing query performance jitter during cluster size reduction.
Supported smooth upgrade and improved the import success rate during version upgrade.

Improvements

Further improved data security.
Improved the stability of the copy into import method.
Provideed better account/data isolation mechanism.

Bug Fixes

Synchronized several bug fixes of doris-1.2-lts.
Fixed the problem that hard disk space could not be cleared in time.

2.2

Note VeloDB Core 2.2 was developed based on Apache Doris 1.2.

June 1, 2023（2.2.34）

Bug Fixes

Fixed the problem of import being stuck due to inconsistent object storage size.

May 15, 2023（2.2.33）

Bug Fixes

Fixed the issue that caused inaccessibility after JDBC URL or mysql -D was used with cluster.

April 27, 2023（2.2.32）

Improvements

Added file cache metrics.

Bug Fixes

Synchronized several bug fixes of doris-1.2-lts.
Fixed the problem of segment file size being out of order.

April 19, 2023（2.2.31）

New Features

Supported dryrun testing when configuring external resources (HMS, external stage).
Supported lightweight schema change to modify the length of Varchar columns.
Supported the copy into import method of split compressed file.

Improvements

Disabled routine load.
Enhanced data security.

Bug Fixes

Fixed issues related to querying top n.
Fixed the problem of query failure due to elimination of file cache.
Fixed the issue where importing a large number of tablets at a time might fail.
Fixed the issue of inverted index memory instability.
Fixed the issue where inverted index could cause query failure.
Fixed the issue where the file was overwritten during copy into, causing the import to get stuck.
Fixed the issue of "no backend" caused by renaming a cluster with the same name.

March 23, 2023（2.2.21）

Bug Fixes

Fixed the issue where cumulative compaction may not be triggered.
Fixed the problem that pure computing plan nodes may cross clusters.
Fixed name inconsistency after executing rename cluster.
Fixed the problem that the schema change job cannot fail automatically when executing drop cluster.
Fixed the issue where base compaction failed due to delete predicate.
Fixed issue with create table like.

February 28, 2023（2.2.20）

Improvements

Implemented several cache optimizations.
Improved copy into import throughput.
Optimized empty table query performance.

Bug Fixes

Fixed the bug of incorrect data when executing add inverted index.
Fixed the issue where parquet import could get stuck.
Fixed several issues related to data lake analysis in Apache Doris 1.2.4.

February 9, 2023（2.2.1）

New Features

Supported external stage cross-account authorization.

Improvements

Optimized copy into upload speed.
Optimized show roles for cluster and stage permission display.

Bug Fixes

Fixed the issue where the computing node may get stuck.
Fixed the problem of resource leakage that may be caused by temp partition.
Fixed the issue where show cluster cannot be executed except by admin user.

January 17, 2023（2.2.0）

New Features

Supported inverted index of String type.
Supported BKD multidimensional numerical index of numeric and date types.
Supported inverted index and BKD multi-dimensional numerical index of array type.
Supported dynamic table to dynamically modify schema.

Improvements

Optimized incremental ADD/DROP INDEX.
Optimized Index searcher cache.
Optimized Top n query (ORDER BY c1, c2 LIMIT n) performance.

Bug Fixes

Fixed the problem of not reading original columns when index filtering.

2.0

Note VeloDB Core 2.0 was developed based on Apache Doris 1.1.

January 5, 2023（2.0.15）

New Features

Supported rename warehouse.

Improvements

Optimized copy into delete sign.
Supported the ability to automatically determine whether the object information is valid when executing copy into to create a stage.
Optimized partition level tablet balance .
Added checking of cache configuration parameter types.
Optimized tablet sink profile.
Improved the permission system.

Bug Fixes

Fixed the problem of misjudgment of stale request in sync_tablet_rowsets.
Fixed load job leak issue.
Fixed partition level balance bug.

December 14, 2022（2.0.14）

New Features

Supported the select statement to set cluster through hint.
Supported retaining several copy job for each table.

Improvements

Optimized the performance of import and write to s3.
After the imported Segment index is uploaded to the object storage, it will be entered into the read cache in advance.

Bug Fixes

Fixed the issue where the data types of planning and execution did not match when importing parquet, causing log import to fail.
Fixed issue with importing across clusters.

December 2, 2022（2.0.13）

New Features

Supported the copy into statement additional condition delete on.
Supported the show data statement to display the data volume of the entire warehouse.

Improvements

Restored use of show processlist statement.
Improved the speed of recycling dropped stages.
Improved the speed of cache data writing to disk.

Bug Fixes

Fixed the issue of inconsistent warehouse behavior when executing drop cluster.

November 26, 2022（2.0.12）

Improvements

Supported verifying the validity of object storage information when executing create stage.

Bug Fixes

Fixed the issue where the write cache may be deleted prematurely.
Fixed the issue where cache might not be hit.
Fixed the issue where non-colocate table tablets could be unevenly distributed.

Platform Release Notes Getting Started