VeloDB Cloud
Release Notes
Core Release Notes

Core Release Notes

This article describes the release notes for VeloDB Core.


3.0

Note VeloDB Core 3.0 was developed based on Apache Doris 2.0.

February 05, 2024(3.0.6)

New Features

  • Supported automatic analyze.
  • Supported wide table import.
  • Supported group commit (batch/real-time import).
  • Supported in left join infer.

Improvements

Bug Fixes

  • Fixed the problem of failure to upgrade from 2.4.x series version to 3.0.x series version.
  • Fixed the problem of misaligned output of the show processlist command.
  • Fixed the problem that altering table‘s attribute TTL may not take effect (the displayed value is 0).
  • Fixed the issue where the queries would occasionally receive the error failed to initialize storage reader.
  • Fixed the issue where abnormal paths in parquet reader may cause memory leaks.
  • Fixed the pushdown problem of date_sub, data_add, to_date functions.
  • Fixed the issue where the colocate jion plan failed due to colocate participating in balance.
  • Fixed the problem that altering colocate‘s attribute replay would fail.
  • Fixed several stability issues.

December 20, 2023(3.0.5)

New Features

  • Provided stream load compatible API.
/api/{db}/{table}/_stream_load_2pc
  • Opened the whitelist of show /proc/statistic.

Improvements

  • Merged all features (it is not recommended to enable automatic analyze function), improvements and bug fixes from Apache Doris 2.0.3 (opens in a new tab).
  • Improved the stability of mow table import.
  • Improved the speed of dynamic table creation.
  • Improved the performance of Plan generation.
  • Optimized decimal operation behavior.
  • Improved stability.

Bug Fixes

  • Fixed the issue where insufficient S3 buffer may cause writing to become stuck.
  • Fixed the problem that the query may get an error "failed to find version path" in multi-cluster scenarios.
  • Fixed the problem that the Plan may cross clusters in multi-cluster scenarios.
  • Fixed the problem that dynamic partition cannot create a new partition due to changing the length of varchar column type.
  • Fixed the issue that show create table decimal is not compatible with SmartBI.
  • Fixed the issue where column type conversion functionality might become unavailable.
  • Fixed the problem that row storage table attributes did not take effect.
  • Fixed the issue where base compaction could fail.
  • Fixed the problem that single point query may not find the latest data.
  • Fixed the issue where copy into duplicate files needs to wait for timeout.
  • Fixed the issue where query_parallel_instance_num could be divided by 0.
  • Fixed the issue where cache hit rate calculation may be inaccurate.
  • Fixed the problem that ngram index does not take effect.

November 23, 2023(3.0.4)

New Features

  • Supported show proc current_stmt

Improvements

  • Enhanced access control security.
  • Enhanced the reliability of uploaded data verification.
  • Improved warehouse metadata service availability.
  • Optimized the error message when creating table dynamic_partition.replication_num.

Bug Fixes

  • Fixed an issue where upgrading VeloDB Core 2.4 to 3.0 may cause copy into to fail.
  • Fixed an issue where importing authorization using external stage may fail.
  • Fixed an issue where routine load import may fail.
  • Fixed an issue where MOW table import may fail.
  • Fixed an issue where real-time writing of insert statements may fail.
  • Fixed tpcds performance degradation issue.
  • Fixed the problem that non-global session variables may not take effect.
  • Fixed the issue where a large number of where conditions in a query would cause high CPU consumption and slow execution.

November 02, 2023(3.0.3)

New Features

  • Supported Doris Clean Label.
  • Stream Load cloud API is compatible with Doris community behavior.
  • The warehouse had built-in JDBC Driver.

Improvements

  • The new optimizer top n supported runtime filter clipping by default.

Bug Fixes

  • Fixed the problem of slow query response due to resource leakage in the warehouse.
  • Fixed the problem that function statement disabling does not take effect.
  • Fixed an issue where schema change would fail when re-arranging.

October 17, 2023(3.0.2)

New Features

  • Added JDBC Driver URL whitelist function for S3 protocol import.

Improvements

  • Merged all features, improvements and bug fixes from Apache Doris 2.0.2 (opens in a new tab).
  • Merged all features, improvements and bug fixes from VeloDB Core 2.3.10 and 2.4.4.
  • Reduced the time to obtain metadata during Plan and speeds up Plan generation.
  • Added time-consuming statistics for query profile accessing metadata.
  • Balanced and optimized resource utilization.

Bug Fixes

  • Fixed the issue of read and write errors after setting File Cache TTL.
  • Fixed the problem of service unavailability caused by querying the backends table or rowsets table.

September 18, 2023(3.0.1-beta)

Improvements

  • Merged all features, improvements and bug fixes from Apache Doris 2.0.1 (opens in a new tab).
  • Merged all features, improvements and bug fixes from VeloDB Core 2.4.3.
  • Optimized the time consumption of get version.

2.4

Note VeloDB Core 2.4 was developed based on Apache Doris 1.2.

November 14, 2023(2.4.6)

Bug Fixes

  • Fixed several compute node memory leaks.

November 06, 2023(2.4.5)

Improvements

  • Optimized the balance strategy.

Bug Fixes

  • Fixed the query failure problem caused by replace tmp partition.
  • Fixed the deadlock problem caused by mixing pthread and bthread.

September 20, 2023(2.4.4)

Improvements

  • Merged all features, improvements and bug fixes from VeloDB Core 2.3.9 and 2.3.10.
  • Improved the usability of the import function.

Bug Fixes

  • Fixed the problem of unresponsive mysql session (requesting truncate table, which contains a large number of partitions).

September 11, 2023(2.4.3)

New Features

  • Supported suspension of service due to arrears.

Improvements

  • Merged all features, improvements and bug fixes from VeloDB Core 2.3.8.
  • Compatible with s3 load endpoint with http string.

Bug Fixes

  • Fixed the docker environment time zone issue.

August 28, 2023(2.4.2)

New Features

  • Supported the copy into import method to set constant values.

Improvements

  • Improved the accuracy of storage information statistics.

Bug Fixes

  • Fixed the issue where cloud stream load might fail.

August 3, 2023(2.4.1)

New Features

  • Supported on-demand cluster auto start when receiving a request.
  • Supported stream load.

Improvements

  • Merged all features, improvements and bug fixes from VeloDB Core 2.3.5, 2.3.6 and 2.3.7.

Bug Fixes

  • Fixed the issue where inverted index queries could fail.

July 3, 2023(2.4.0)

New Features

  • Supported controlling whether SQL uses cache through session variable.
  • Supported incremental data synchronization of imported data between clusters.
  • Supported cache data preheating function based on table cluster.
  • Supported querying cache usage through SQL.
  • Supported on-demand cluster auto stop when idle.

Improvements

  • Enhanced data security.
  • Improved the response speed when the query misses the cache.

Bug Fixes

  • Fixed the issue where set default cluster might fail.
  • Fixed the issue where the upgrade process could fail.

2.3

Note VeloDB Core 2.3 was developed based on Apache Doris 1.2.

October 16, 2023 (2.3.11)

Improvements

  • Optimized the balance strategy.

Bug Fixes

  • Fixed the query failure problem caused by replace tmp partition.

September 20, 2023 (2.3.10)

Bug Fixes

  • Fixed the problem of time extension in copy into import method.

September 11, 2023 (2.3.9)

Bug Fixes

  • Fixed the problem that the in operator for bitmap type columns caused the query to be unavailable.
  • Fixed decimal calculation accuracy problem.
  • Fixed the bug that when the alter statement changes more columns, the key column changes will also go through lightweight schema change.

August 29, 2023 (2.3.8)

Improvements

  • Optimized the speed of importing and uploading small objects.

Bug Fixes

  • Fixed the problem of incorrect division accuracy in decimal v3 in some cases (increased 100 times accuracy).

August 2, 2023 (2.3.7)

Bug Fixes

  • Fixed the issue where incorrect type inference for decimal v2 calculations resulted in inaccurate precision.
  • Fixed the issue where create catalog relied on an external driver, causing the warehouse to fail to start.
  • Fixed the issue where indexes will be created for all columns when adding bitmap index (which may cause data import failure).
  • Fixed the issue where the default value of the copy into import method does not take effect.

July 25, 2023 (2.3.6)

Improvements

  • Improved the usability of the import function.
  • The round precision behavior of decimal v2 is supported and is compatible with decimal v3.

Bug Fixes

  • Fixed the issue where queries could get stuck.
  • Fixed the problem of access failure that may report 403 when importing error_log.

July 19, 2023 (2.3.5)

Improvements

  • Reduced the delay when importing small batches of data with partition skew.
  • Added the limit configuration of the length of the audit log insert into statement, and the default is 4KB truncation.
  • Improved remote IO efficiency and accelerated query speed.

Bug Fixes

  • Fixed the issue where an error would be prompted if the column name in the delete statement condition was capitalized.
  • Fixed known memory leak issue.
  • Fixed the issue where tablet balance could be uneven.
  • Fixed the issue where node restart might cause query failure.
  • Fixed the problem of OSS S3 load error and added AWS_PROVIDER attribute.

July 7, 2023 (2.3.4)

Bug Fixes

  • Fixed the issue where queries might become unavailable.

July 3, 2023 (2.3.3)

Improvements

  • Speeded up node restart speed and improved availability.

Bug Fixes

  • Fixed several memory leak issues.

June 27, 2023 (2.3.2)

Improvements

  • Upgraded the warehouse mariadb client version.
  • Reduced the dimension statistics of file cache metrics.
  • Added indicators for warehouse Fragment execution.

Bug Fixes

  • Fixed the issue where the query is canceled but the hash table is not released during query execution.
  • Fixed the bug that inverted index files were not recycled after compaction.
  • Added null check to sql parser.
  • Fixed the bug that some segments could not be opened after limit optimization.

May 16, 2023 (2.3.0)

New Features

  • Supported MySQL TLS connection (closed by default, enabling it requires separate configuration, and requires mysql client 5.0.38 or above).
  • Supported Prepare stmt.
  • Supported smooth node reduction, reducing query performance jitter during cluster size reduction.
  • Supported smooth upgrade and improved the import success rate during version upgrade.

Improvements

  • Further improved data security.
  • Improved the stability of the copy into import method.
  • Provideed better account/data isolation mechanism.

Bug Fixes

  • Synchronized several bug fixes of doris-1.2-lts.
  • Fixed the problem that hard disk space could not be cleared in time.

2.2

Note VeloDB Core 2.2 was developed based on Apache Doris 1.2.

June 1, 2023(2.2.34)

Bug Fixes

  • Fixed the problem of import being stuck due to inconsistent object storage size.

May 15, 2023(2.2.33)

Bug Fixes

  • Fixed the issue that caused inaccessibility after JDBC URL or mysql -D was used with cluster.

April 27, 2023(2.2.32)

Improvements

  • Added file cache metrics.

Bug Fixes

  • Synchronized several bug fixes of doris-1.2-lts.
  • Fixed the problem of segment file size being out of order.

April 19, 2023(2.2.31)

New Features

  • Supported dryrun testing when configuring external resources (HMS, external stage).
  • Supported lightweight schema change to modify the length of Varchar columns.
  • Supported the copy into import method of split compressed file.

Improvements

  • Disabled routine load.
  • Enhanced data security.

Bug Fixes

  • Fixed issues related to querying top n.
  • Fixed the problem of query failure due to elimination of file cache.
  • Fixed the issue where importing a large number of tablets at a time might fail.
  • Fixed the issue of inverted index memory instability.
  • Fixed the issue where inverted index could cause query failure.
  • Fixed the issue where the file was overwritten during copy into, causing the import to get stuck.
  • Fixed the issue of "no backend" caused by renaming a cluster with the same name.

March 23, 2023(2.2.21)

Bug Fixes

  • Fixed the issue where cumulative compaction may not be triggered.
  • Fixed the problem that pure computing plan nodes may cross clusters.
  • Fixed name inconsistency after executing rename cluster.
  • Fixed the problem that the schema change job cannot fail automatically when executing drop cluster.
  • Fixed the issue where base compaction failed due to delete predicate.
  • Fixed issue with create table like.

February 28, 2023(2.2.20)

Improvements

  • Implemented several cache optimizations.
  • Improved copy into import throughput.
  • Optimized empty table query performance.

Bug Fixes

  • Fixed the bug of incorrect data when executing add inverted index.
  • Fixed the issue where parquet import could get stuck.
  • Fixed several issues related to data lake analysis in Apache Doris 1.2.4.

February 9, 2023(2.2.1)

New Features

  • Supported external stage cross-account authorization.

Improvements

  • Optimized copy into upload speed.
  • Optimized show roles for cluster and stage permission display.

Bug Fixes

  • Fixed the issue where the computing node may get stuck.
  • Fixed the problem of resource leakage that may be caused by temp partition.
  • Fixed the issue where show cluster cannot be executed except by admin user.

January 17, 2023(2.2.0)

New Features

  • Supported inverted index of String type.
  • Supported BKD multidimensional numerical index of numeric and date types.
  • Supported inverted index and BKD multi-dimensional numerical index of array type.
  • Supported dynamic table to dynamically modify schema.

Improvements

  • Optimized incremental ADD/DROP INDEX.
  • Optimized Index searcher cache.
  • Optimized Top n query (ORDER BY c1, c2 LIMIT n) performance.

Bug Fixes

  • Fixed the problem of not reading original columns when index filtering.

2.0

Note VeloDB Core 2.0 was developed based on Apache Doris 1.1.

January 5, 2023(2.0.15)

New Features

  • Supported rename warehouse.

Improvements

  • Optimized copy into delete sign.
  • Supported the ability to automatically determine whether the object information is valid when executing copy into to create a stage.
  • Optimized partition level tablet balance .
  • Added checking of cache configuration parameter types.
  • Optimized tablet sink profile.
  • Improved the permission system.

Bug Fixes

  • Fixed the problem of misjudgment of stale request in sync_tablet_rowsets.
  • Fixed load job leak issue.
  • Fixed partition level balance bug.

December 14, 2022(2.0.14)

New Features

  • Supported the select statement to set cluster through hint.
  • Supported retaining several copy job for each table.

Improvements

  • Optimized the performance of import and write to s3.
  • After the imported Segment index is uploaded to the object storage, it will be entered into the read cache in advance.

Bug Fixes

  • Fixed the issue where the data types of planning and execution did not match when importing parquet, causing log import to fail.
  • Fixed issue with importing across clusters.

December 2, 2022(2.0.13)

New Features

  • Supported the copy into statement additional condition delete on.
  • Supported the show data statement to display the data volume of the entire warehouse.

Improvements

  • Restored use of show processlist statement.
  • Improved the speed of recycling dropped stages.
  • Improved the speed of cache data writing to disk.

Bug Fixes

  • Fixed the issue of inconsistent warehouse behavior when executing drop cluster.

November 26, 2022(2.0.12)

Improvements

  • Supported verifying the validity of object storage information when executing create stage.

Bug Fixes

  • Fixed the issue where the write cache may be deleted prematurely.
  • Fixed the issue where cache might not be hit.
  • Fixed the issue where non-colocate table tablets could be unevenly distributed.