v3.0

This article describes the release notes for VeloDB Core v3.0.

Note VeloDB Core v3.0 was developed based on Apache Doris v2.0 (opens in a new tab).

v3.0.10 (August 23, 2024)

Improvements

• Merged all features, improvements and bug fixes from Apache Doris v2.0.13 (opens in a new tab).

v3.0.9 (June 25, 2024)

Improvements

• Merged all features, improvements and bug fixes from Apache Doris v2.0.11 (opens in a new tab).
• Merged all features, improvements and bug fixes from VeloDB Core v3.0.8.
• Optimized the calculation of delete bitmap in mow table, improving import speed and robustness.
• Optimized the response speed of show data.
• Optimized the reverse index and semi-structured data query performance.

Bug Fixes

• Fixed the problem that too many copy into tasks caused warehouse OOM.
• Fixed the problem that too many alter jobs may cause warehouse OOM.
• Fixed the problem that warehouse metrics may be missing in multi-cluster scenarios.
• Fixed the issue that some tablets are missing versions in extreme cases when importing multiple tablets in a single transaction.
• Fixed the issue that the warehouse cannot immediately perceive the change in file cache size.
• Fixed several JDBC external table connection issues.
• Fixed the issue that ES Catalog cannot use esquery.
• Fixed the issue of uneven balance and decommission stuck.

v3.0.8 (May 15, 2024)

New Features

• Supported recycler speed limit.
• Supported show proc statement.
• Supported getting all json keys.

Improvements

• Merged all features, improvements and bug fixes from Apache Doris v2.0.10 (opens in a new tab).
• Optimized the time consumption of table creation.
• Optimized the calculation speed delete bitmap in mow table to improve import throughput.
• Optimized the query speed of mow table high-frequency import (multiple versions).
• Optimized query efficiency.

Bug Fixes

• Fixed the problem of incorrect indicators after rename cluster.
• Fixed the problem of inaccurate traffic statistics after rename cluster.
• Fixed the problem of incorrect format of show processlist.
• Fixed the problem of failure due to insufficient memory of schema change.
• Fixed the problem of import permission of audit log component.
• Fixed the error of datetimev2 round function.
• Fixed the problem that nullable planning may cause coredump.
• Fixed the problem that nullable planning may cause wrong results.

v3.0.7 (March 31, 2024)

New Features

• Supported audit log plug-in (audit log table needs to be created manually).

Improvements

• Merged all features, improvements and bug fixes from Apache Doris v2.0.6 (opens in a new tab).
• If the user does not set a default cluster when executing the query, the available (live, authorized) clusters will be selected in dictionary order of cluster names.
• After the cluster automatically stopped, it is no longer necessary to explicitly use cluster to trigger automatic start of the cluster.
• Optimized availability and performance of reverse index and log analysis scenarios.
• The new optimizer optimizes ambiguous projection having syntax.
• Improved the performance of reverse index compound in list.

Bug Fixes

• Fixed the issue where a large number of delete requests triggered import failure.
• Fixed the issue where the statistics logic repeatedly triggers SQL, causing the cluster's automatic stop logic to fail.
• Fixed the issue with importing specified UTC time zone conversion errors.
• Fixed the error of show process list from JDBC.
• Fixed the problem of table attribute ttl_seconds being cleared to 0 after restart after checkpoint.
• Fixed the date/month/year add function.

v3.0.6 (February 5, 2024)

New Features

• Supported automatic analyze.
• Supported wide table import.
• Supported group commit (batch/real-time import).
• Supported in left join infer.

Improvements

• Merged all features, improvements and bug fixes from Apache Doris v2.0.4 (opens in a new tab).
• Optimized the error message when import fails.
• Improved stability.

Bug Fixes

• Fixed the issue where cluster restart may cause cluster permissions to become invalid.
• Fixed the problem of failure to upgrade from VeloDB Core v2.4.x series version to v3.0.x series version.
• Fixed the problem of misaligned output of the show processlist command.
• Fixed the problem that altering table‘s attribute TTL may not take effect (the displayed value is 0).
• Fixed the issue where the queries would occasionally receive the error failed to initialize storage reader.
• Fixed the issue where abnormal paths in parquet reader may cause memory leaks.
• Fixed the pushdown problem of date_sub, data_add, to_date functions.
• Fixed the issue where the colocate jion plan failed due to colocate participating in balance.
• Fixed the problem that altering colocate‘s attribute replay would fail.
• Fixed the issue where data cannot be queried due to special characters in the delete condition.
• Fixed the issue of read query reporting NOT_IMPLEMENTED_ERROR with a small probability.
• Fixed the issue where importing time zone conversions resulted in incorrect data.
• Fixed the issue where passing negative values to substr() would cause coredump.

v3.0.5 (December 20, 2023)

New Features

• Provided stream load compatible API.

/api/{db}/{table}/_stream_load_2pc

• Opened the whitelist of show /proc/statistic.

Improvements

• Merged all features (it is not recommended to enable automatic analyze function), improvements and bug fixes from Apache Doris v2.0.3 (opens in a new tab).
• Improved the stability of mow table import.
• Improved the speed of dynamic table creation.
• Improved the performance of Plan generation.
• Optimized decimal operation behavior.
• Improved stability.

Bug Fixes

• Fixed the issue where insufficient S3 buffer may cause writing to become stuck.
• Fixed the problem that the query may get an error "failed to find version path" in multi-cluster scenarios.
• Fixed the problem that the Plan may cross clusters in multi-cluster scenarios.
• Fixed the problem that dynamic partition cannot create a new partition due to changing the length of varchar column type.
• Fixed the issue that show create table decimal is not compatible with SmartBI.
• Fixed the issue where column type conversion functionality might become unavailable.
• Fixed the problem that row storage table attributes did not take effect.
• Fixed the issue where base compaction could fail.
• Fixed the problem that single point query may not find the latest data.
• Fixed the issue where copy into duplicate files needs to wait for timeout.
• Fixed the issue where query_parallel_instance_num could be divided by 0.
• Fixed the issue where cache hit rate calculation may be inaccurate.
• Fixed the problem that ngram index does not take effect.

v3.0.4 (November 23, 2023)

New Features

• Supported show proc current_stmt。

Improvements

• Enhanced access control security.
• Enhanced the reliability of uploaded data verification.
• Improved warehouse metadata service availability.
• Optimized the error message when creating table dynamic_partition.replication_num.

Bug Fixes

• Fixed an issue where upgrading VeloDB Core v2.4 to v3.0 may cause copy into to fail.
• Fixed an issue where importing authorization using external stage may fail.
• Fixed an issue where routine load import may fail.
• Fixed an issue where mow table import may fail.
• Fixed an issue where real-time writing of insert statements may fail.
• Fixed tpcds performance degradation issue.
• Fixed the problem that non-global session variables may not take effect.
• Fixed the issue where a large number of where conditions in a query would cause high CPU consumption and slow execution.

v3.0.3 (November 2, 2023)

New Features

• Supported Doris Clean Label.
• Stream Load cloud API is compatible with Doris community behavior.
• The warehouse had built-in JDBC Driver.

Improvements

• The new optimizer top n supported runtime filter clipping by default.

Bug Fixes

• Fixed the problem of slow query response due to resource leakage in the warehouse.
• Fixed the problem that function statement disabling does not take effect.
• Fixed an issue where schema change would fail when re-arranging.

v3.0.2 (October 17, 2023)

New Features

• Added JDBC Driver URL whitelist function for S3 protocol import.

Improvements

• Merged all features, improvements and bug fixes from Apache Doris v2.0.2 (opens in a new tab).
• Merged all features, improvements and bug fixes from VeloDB Core v2.3.10 and v2.4.4.
• Reduced the time to obtain metadata during Plan and speeds up Plan generation.
• Added time-consuming statistics for query profile accessing metadata.
• Balanced and optimized resource utilization.

Bug Fixes

• Fixed the issue of read and write errors after setting File Cache TTL.
• Fixed the problem of service unavailability caused by querying the backends table or rowsets table.

v3.0.1-beta (September 18, 2023)

Improvements

• Merged all features, improvements and bug fixes from Apache Doris v2.0.1 (opens in a new tab).
• Merged all features, improvements and bug fixes from VeloDB Core v2.4.3.
• Optimized the time consumption of get version.