Enterprise Core

Enterprise Core 2.1.x

Enterprise Core 2.1.2

April 18, 2024

Add processlist table in information_schema database, users could use this table to query active connections.
Add a new table valued function LOCAL to allow access file system like shared storage.
Set the default value of the data_consistence property of EXPORT to partition to make export more stable during load.
Some of MySQL Connector (eg, dotnet MySQL.Data) rely on variable's column type to make connection.
Add rollup table name in profile to help find the mv selection problem.
Add test connection function to DB2 database to allow user check the connection when create DB2 Catalog.
Add DNS Cache for FQDN to accelerate the connect process among BEs in K8s env.
Refresh external table's rowcount async to make the query plan more stable.

Enterprise Core 2.1.1

April 8, 2024

Change float type output format to improve float type serialization performance.
Fix issues during rolling upgrade from 2.0.x to 2.1.x, including backend node core problems and JDBC Catalog query errors.
Enable proxy protocol to support IP transparency. Using this protocol, IP transparency for load balancing can be achieved, so that after load balancing, Doris can still obtain the client's real IP and implement permission control such as whitelisting.
Add new system table backend_active_tasks to monitor the realtime query statics on every BE.
Add inverted index support for CCR.
Support arrow serialization for varint type.
Fixed 20 bugs, including occasional core issues in the BE during the restore process.

Enterprise Core 2.1.0

March 18, 2024

Further improvement in the performance of complex SQL queries has been achieved, with over 100% performance enhancement on the TPC-DS 1TB test dataset, positioning query performance at the forefront of the industry.
Performance improvements in data lake analytics scenarios, with 4-6 times better performance compared to Trino and Spark, have been made. Additionally, compatibility with multiple SQL dialects has been introduced, enabling seamless migration from existing systems to Apache Doris.
For data science and other forms of large-scale data reading scenarios, a high-speed reading interface based on Arrow Flight has been introduced, resulting in a 100-fold improvement in data transfer efficiency.
In semi-structured data analysis scenarios, new Variant and IP data types have been introduced, along with enhancements to a series of analytical functions, making storage and analysis of complex semi-structured data more convenient.
The introduction of asynchronous materialized views based on multiple tables has improved query performance. This includes support for transparent rewriting acceleration, automatic refreshing, external-to-internal table materialized views, and direct querying of materialized views. Leveraging these capabilities, materialized views can also be used for data warehouse tiered modeling, job scheduling, and data processing.
In terms of data storage, capabilities such as auto-increment columns, automatic partitioning, MemTable forwarding, and server-side batching have been introduced to improve the efficiency of real-time data writing at scale.
Further improvements have been made in workload management, enhancing the isolation capability of Workload Group resource groups and adding the ability to view SQL resource usage at runtime, thereby enhancing stability in multi-load scenarios.

Enterprise Core 2.0.x

Enterprise Core 2.0.9

April 24, 2024

Allow predicate appearing on both key and value mv columns
Enable mv with bitmap_union(bitmap_from_array())
Introduce a configuration to forcibly replicate allocation for all olap tables within the cluster
Add timezone support for date literals in the new optimizer Nereids
Enable the use of "slop" in full-text search's match_phrase to specify word distances
Display index ID in SHOW PROC INDEXES
Incorporate a secondary argument in first_value/last_value to overlook NULL values
Allow the use of 0 as the offset parameter in the lead/lag function

Enterprise Core 2.0.8

April 12, 2024

Make Inverted Index work with TopN opt in Nereids
Limit the max string length to 1024 while collecting column stats to control BE memory usage
JDBC Catalog close when JDBC client is not empty
Accept all Iceberg database and do not check the name format of database
Refresh external table's rowcount async to avoid cache miss and unstable query plan
Simplify the isSplitable method of hive external table to avoid too many hadoop metrics

Enterprise Core 2.0.7

March 26, 2024

Support make miss slot as null alias when converting outer join to anti join to speed up query.
Add DEFAULT_ENCRYPTION column in information_schema table and add processlist table for better compatibility for BI tools.
Automatically test connectivity by default when creating a JDBC Catalog.
Enhance auto resume to keep routine load stable.
Use lowercase by default for Chinese tokenizer in inverted index.
Add error msg if exceeded maximum default value in repeat function.
Skip hidden file and dir in hive table.
Reduce file meta cache size and disable cache for some cases to avoid OOM.
Reduce jvm heap memory consumed by profiles of BrokerLoadJob.
Remove sort which is under table sink to speed up query like INSERT INTO t1 SELECT * FROM t2 ORDER BY k.

Enterprise Core 2.0.6

March 14, 2024

Support match a function with alias in materialized-view.
Add a command to drop a tablet replica safely on backend.
Add row count cache for external table.
Support analyze rollup to gather statistics for optimizer.
Improve tablet schema cache memory by using deterministic way to serialize protobuf.
Improve show column stats performance.
Support estimate row count for iceberg and paimon.
Support sqlserver timestamp type read for JDBC catalog.

Enterprise Core 2.0.3

December 8, 2023

Supports automatic statistics collection, which aids the optimizer in understanding data distribution characteristics. This enables the selection of more optimal plans, significantly improving query efficiency. Starting from version 2.0.3, this feature is officially supported and is enabled by default throughout the day.
Data Lake supports JAVA UDF, JDBC, Hudi MOR, and more system support for complex data types.
Cross-Cluster Replication (CCR) now supports features like throttling and table truncation enhancements.
Additional built-in functions such as SHA, JSON have been added.
Over 20 performance improvements including inverted index, case when, predicate pushdown, etc.
Enhanced distributed replica management, including skipping deleted partitions, colocate group, continuous write balancing failure, and inability to balance cold-hot tiered tables.
Stability improvements in complex data types, inverted index, materialized views, import and compaction, Data Lake compatibility, SQL planning, and more.

Enterprise Core 2.0.2

October 8, 2023

Improved usability, including optimized priority network matching logic and support for role-based authorization at the row level.
Enhanced statistics data collection in the new optimizer, Nereids, including the elimination of file caching during the execution of analysis tasks and support for basic JDBC external table statistics collection.
Performance optimization and enhancement in inverted index queries, including the addition of BKD indices for improved efficiency and optimization of multi-word conjunction queries.
Improved support for multiple types of data sources in the multi-catalog feature for data lakes, including JDBC, HDFS, Hive, MySQL, MaxCompute, and more.
Optimization of array functions, with the array_union now supporting multiple parameters.

Enterprise Core 2.0.1

September 10, 2023

Improved the functionality and stability of complex data types such as arrays and maps, including nested complex types in inner tables and nesting outer tables with ORC/Parquet formats.
Enhanced performance in inverted index queries, covering tokenization, numerical processing, predicate pushdown, and more.
Improved query performance, including enhancements in bitmap operations, LIKE queries, scans, and aggregate functions.
Refined and stabilized Cross-Cluster Replication (CCR) functionality.
Accelerated and improved accuracy in the collection of statistics by the query optimizer, resulting in enhanced automatic query optimization.
Enhanced functionality and performance in the multi-catalog feature for data lakes, including performance optimizations for Iceberg and support for complex data types.

Enterprise Core 1.2.x

Enterprise Core 1.2.8

September 05, 2023

Fixed several decimal-related issues.
Resolved the problem where "show tables" couldn't display tables for which the user had select permissions.
Addressed issues related to replica scheduling.
Fixed several query planning problems.
Addressed an issue of file handle leakage in BE processes under certain circumstances.
Fixed a problem with table creation timing out in certain scenarios.
Resolved errors when reading ORC format files.
Fixed an issue where closing the FileSystem in Broker caused read errors.
Optimized the logic for calculating replica sizes in Auto Bucket.
Fixed a NullPointerException issue in Spark Load under certain circumstances.

Apache Doris Docs Enterprise Manager