This is an automated email from the ASF dual-hosted git repository.
zykkk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new fa1f86556c0 [typo](docs) Remove features that are not useful in 1.2
(#269)
fa1f86556c0 is described below
commit fa1f86556c0d8fb4800f1924b59fcb567aea4ddd
Author: zy-kkk <zhongy...@gmail.com>
AuthorDate: Sat Jul 22 11:58:30 2023 +0800
[typo](docs) Remove features that are not useful in 1.2 (#269)
---
.../version-1.2/data-table/dynamic-schema-table.md | 133 -------
.../version-1.2/data-table/index/inverted-index.md | 403 --------------------
.../data-table/index/ngram-bloomfilter-index.md | 84 -----
.../hight-concurrent-point-query.md | 103 ------
.../version-1.2/query-acceleration/nereids.md | 87 -----
.../pipeline-execution-engine.md | 78 ----
.../version-1.2/data-table/dynamic-schema-table.md | 135 -------
.../version-1.2/data-table/index/inverted-index.md | 405 ---------------------
.../data-table/index/ngram-bloomfilter-index.md | 82 -----
.../hight-concurrent-point-query.md | 106 ------
.../version-1.2/query-acceleration/nereids.md | 87 -----
.../pipeline-execution-engine.md | 78 ----
versioned_sidebars/version-1.2-sidebars.json | 6 -
13 files changed, 1787 deletions(-)
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-1.2/data-table/dynamic-schema-table.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-1.2/data-table/dynamic-schema-table.md
deleted file mode 100644
index a60b35686dd..00000000000
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-1.2/data-table/dynamic-schema-table.md
+++ /dev/null
@@ -1,133 +0,0 @@
----
-{
- "title": "动态schema表",
- "language": "zh-CN"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# 动态表
-
-<version since="2.0.0">
-
-动态表
-
-</version>
-
-动态schema表是一种特殊的表，其schema随着导入自动进行扩展。目前该功能，主要用于半结构数据，例如JSON等的导入、自动列生成。因为JSON是类型自描述的，所以我们可以从原始文档中提取schema信息，推断最终类型信息。这种特殊的表可以减少人工schema
change的操作，并轻松导入半结构数据并自动扩展其schema。
-
-## 名词解释
-- schema change, 改变表的结构， 例如增加列、减少列， 修改列类型
-- 静态列， 在建表时指定的列， 例如分区列、主键列
-- 动态列， 随着导入自动识别并增加的列
-
-## 建表
-
-```sql
-CREATE DATABASE test_dynamic_table;
-
--- 建表， 并指定静态列类型， 导入遇到对应列会自动转换成静态列的类型
--- 选择随机分桶方式
-CREATE TABLE IF NOT EXISTS test_dynamic_table (
- qid bigint,
- `answers.date` array<datetime>,
- `title` string,
- ... -- ...标识该表是动态表， 是动态表的语法
- )
-DUPLICATE KEY(`qid`)
-DISTRIBUTED BY RANDOM BUCKETS 5
-properties("replication_num" = "1");
-
--- 可以看到三列Column在表中默认添加， 类型是指定类型
-mysql> DESC test_dynamic_table;
-+--------------+-----------------+------+-------+---------+-------+
-| Field | Type | Null | Key | Default | Extra |
-+--------------+-----------------+------+-------+---------+-------+
-| qid | BIGINT | Yes | true | NULL | |
-| answers.date | ARRAY<DATETIME> | Yes | false | NULL | NONE |
-| user | TEXT | Yes | false | NULL | NONE |
-+--------------+-----------------+------+-------+---------+-------+
-3 rows in set (0.00 sec)
-```
-
-## 导入数据
-
-``` sql
--- example1.json
-'{
- "title": "Display Progress Bar at the Time of Processing",
- "qid": "1000000",
- "answers": [
- {"date": "2009-06-16T09:55:57.320", "user": "Micha\u0142 Niklas
(22595)"},
- {"date": "2009-06-17T12:34:22.643", "user": "Jack Njiri (77153)"}
- ],
- "tag": ["vb6", "progress-bar"],
- "user": "Jash",
- "creationdate": "2009-06-16T07:28:42.770"
-}'
-
-curl -X PUT -T example1.json --location-trusted -u root: -H
"read_json_by_line:false" -H "format:json"
http://127.0.0.1:8147/api/regression_test_dynamic_table/test_dynamic_table/_stream_load
-
--- 新增 title，answers.user， tag， title， creationdate 五列
--- 且 qid，answers.date，user三列类型与建表时保持一致
--- 新增数组类型默认Default值为空数组[]
-mysql> DESC test_dynamic_table;
-+--------------+-----------------+------+-------+---------+-------+
-| Field | Type | Null | Key | Default | Extra |
-+--------------+-----------------+------+-------+---------+-------+
-| qid | BIGINT | Yes | true | NULL | |
-| answers.date | ARRAY<DATETIME> | Yes | false | NULL | NONE |
-| title | TEXT | Yes | false | NULL | NONE |
-| answers.user | ARRAY<TEXT> | No | false | [] | NONE |
-| tag | ARRAY<TEXT> | No | false | [] | NONE |
-| user | TEXT | Yes | false | NULL | NONE |
-| creationdate | TEXT | Yes | false | NULL | NONE |
-| date | TEXT | Yes | false | NULL | NONE |
-+--------------+-----------------+------+-------+---------+-------+
-
--- 批量导入数据
-
--- 指定 -H "read_json_by_line:true"， 逐行解析JSON
-curl -X PUT -T example_batch.json --location-trusted -u root: -H
"read_json_by_line:true" -H "format:json"
http://127.0.0.1:8147/api/regression_test_dynamic_table/test_dynamic_table/_stream_load
-
--- 指定 -H "strip_outer_array:true"， 整个文件当做一个JSON array解析， array中的每个元素是一行，
解析效率更高效
-curl -X PUT -T example_batch_array.json --location-trusted -u root: -H
"strip_outer_array:true" -H "format:json"
http://127.0.0.1:8147/api/regression_test_dynamic_table/test_dynamic_table/_stream_load
-```
-对于dynamic table， 你也可以使用S3load或者Routine load, 使用方式类似
-
-## 对动态列增加索引
-```sql
--- 将在titile列上新建倒排索引， 并按照english分词
-CREATE INDEX title_idx ON test_dynamic_table (`title`) using inverted
PROPERTIES("parser"="english")
-```
-
-## 类型冲突
-在第一批导入会自动推断出统一的类型， 并以此作为最终的Column类型，所以建议保持Column类型的一致, 例如
-```
-{"id" : 123}
-{"id" : "123"}
--- 类型会被最终推断为Text类型， 如果在后续导入{"id" : 123}则类型会被自动转成String类型
-
-对于无法统一的类型， 例如
-{"id" : [123]}
-{"id" : 123}
--- 导入将会报错
-```
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-1.2/data-table/index/inverted-index.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-1.2/data-table/index/inverted-index.md
deleted file mode 100644
index a624d31fa55..00000000000
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-1.2/data-table/index/inverted-index.md
+++ /dev/null
@@ -1,403 +0,0 @@
----
-{
- "title": "倒排索引",
- "language": "zh-CN"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# [Experimental] 倒排索引
-
-<version since="2.0.0">
-
-</version>
-
-从2.0.0版本开始，Doris支持倒排索引，可以用来进行文本类型的全文检索、普通数值日期类型的等值范围查询，快速从海量数据中过滤出满足条件的行。本文档主要介绍如何倒排索引的创建、删除、查询等使用方式。
-
-
-## 名词解释
-
-- [inverted
index](https://en.wikipedia.org/wiki/Inverted_index)：[倒排索引](https://zh.wikipedia.org/wiki/%E5%80%92%E6%8E%92%E7%B4%A2%E5%BC%95)，是信息检索领域常用的索引技术，将文本分割成一个个词，构建
词 -> 文档编号 的索引，可以快速查找一个词在哪些文档出现。
-
-
-## 原理介绍
-
-Doris使用[CLucene](https://clucene.sourceforge.net/)作为底层的倒排索引库。CLucene是一个用C++实现的高性能、稳定的Lucene倒排索引库。Doris进一步优化了CLucene，使得它更简单、更快、更适合数据库场景。
-
-在Doris的倒排索引实现中，table的一行对应一个文档、一列对应文档中的一个字段，因此利用倒排索引可以根据关键词快速定位包含它的行，达到WHERE子句加速的目的。
-
-与Doris中其他索引不同的是，在存储层倒排索引使用独立的文件，跟segment文件有逻辑对应关系、但存储的文件相互独立。这样的好处是可以做到创建、删除索引不用重写tablet和segment文件，大幅降低处理开销。
-
-
-## 功能介绍
-
-Doris倒排索引的功能简要介绍如下：
-
-- 增加了字符串类型的全文检索
- - 支持字符串全文检索，包括同时匹配多个关键字MATCH_ALL、匹配任意一个关键字MATCH_ANY
- - 支持字符串数组类型的全文检索
- - 支持英文、中文分词
-- 加速普通等值、范围查询，覆盖bitmap索引的功能，未来会代替bitmap索引
- - 支持字符串、数值、日期时间类型的 =, !=, >, >=, <, <= 快速过滤
- - 支持字符串、数字、日期时间数组类型的 =, !=, >, >=, <, <=
-- 支持完善的逻辑组合
- - 新增索引对OR NOT逻辑的下推
- - 支持多个条件的任意AND OR NOT组合
-- 灵活、快速的索引管理
- - 支持在创建表上定义倒排索引
- - 支持在已有的表上增加倒排索引，而且支持增量构建倒排索引，无需重写表中的已有数据
- - 支持删除已有表上的倒排索引，无需重写表中的已有数据
-
-## 语法
-
-- 建表时定义倒排索引，语法说明如下
- - USING INVERTED 是必须的，用于指定索引类型是倒排索引
- - PROPERTIES 是可选的，用于指定倒排索引的额外属性，目前有一个属性parser指定分词器
- - 默认不指定代表不分词
- - english是英文分词，适合被索引列是英文的情况，用空格和标点符号分词，性能高
- - chinese是中文分词，适合被索引列有中文或者中英文混合的情况，采用jieba分词库，性能比english分词低
- - COMMENT 是可选的，用于指定注释
-
-```sql
-CREATE TABLE table_name
-(
- columns_difinition,
- INDEX idx_name1(column_name1) USING INVERTED [PROPERTIES("parser" =
"english|chinese")] [COMMENT 'your comment']
- INDEX idx_name2(column_name2) USING INVERTED [PROPERTIES("parser" =
"english|chinese")] [COMMENT 'your comment']
-)
-table_properties;
-```
-
-- 已有表增加倒排索引
-```sql
--- 语法1
-CREATE INDEX idx_name ON table_name(column_name) USING INVERTED
[PROPERTIES("parser" = "english|chinese")] [COMMENT 'your comment'];
--- 语法2
-ALTER TABLE table_name ADD INDEX idx_name(column_name) USING INVERTED
[PROPERTIES("parser" = "english|chinese")] [COMMENT 'your comment'];
-```
-
-- 删除倒排索引
-```sql
--- 语法1
-DROP INDEX idx_name ON table_name;
--- 语法2
-ALTER TABLE table_name DROP INDEX idx_name;
-```
-
-- 利用倒排索引加速查询
-```sql
--- 1. 全文检索关键词匹配，通过MATCH_ANY MATCH_ALL完成
-SELECT * FROM table_name WHERE column_name MATCH_ANY | MATCH_ALL 'keyword1
...';
-
--- 1.1 logmsg中包含keyword1的行
-SELECT * FROM table_name WHERE logmsg MATCH_ANY 'keyword1';
-
--- 1.2 logmsg中包含keyword1或者keyword2的行，后面还可以添加多个keyword
-SELECT * FROM table_name WHERE logmsg MATCH_ANY 'keyword2 keyword2';
-
--- 1.3 logmsg中同时包含keyword1和keyword2的行，后面还可以添加多个keyword
-SELECT * FROM table_name WHERE logmsg MATCH_ALL 'keyword2 keyword2';
-
-
--- 2. 普通等值、范围、IN、NOT IN，正常的SQL语句即可，例如
-SELECT * FROM table_name WHERE id = 123;
-SELECT * FROM table_name WHERE ts > '2023-01-01 00:00:00';
-SELECT * FROM table_name WHERE op_type IN ('add', 'delete');
-```
-
-## 使用示例
-
-用hackernews 100万条数据展示倒排索引的创建、全文检索、普通查询，包括跟无索引的查询性能进行简单对比。
-
-### 建表
-
-```sql
-
-CREATE DATABASE test_inverted_index;
-
-USE test_inverted_index;
-
--- 创建表的同时创建了comment的倒排索引idx_comment
--- USING INVERTED 指定索引类型是倒排索引
--- PROPERTIES("parser" = "english")
指定采用english分词，还支持"chinese"中文分词，如果不指定"parser"参数表示不分词
-CREATE TABLE hackernews_1m
-(
- `id` BIGINT,
- `deleted` TINYINT,
- `type` String,
- `author` String,
- `timestamp` DateTimeV2,
- `comment` String,
- `dead` TINYINT,
- `parent` BIGINT,
- `poll` BIGINT,
- `children` Array<BIGINT>,
- `url` String,
- `score` INT,
- `title` String,
- `parts` Array<INT>,
- `descendants` INT,
- INDEX idx_comment (`comment`) USING INVERTED PROPERTIES("parser" =
"english") COMMENT 'inverted index for comment'
-)
-DUPLICATE KEY(`id`)
-DISTRIBUTED BY HASH(`id`) BUCKETS 10
-PROPERTIES ("replication_num" = "1");
-
-```
-
-
-### 导入数据
-
-- 通过stream load导入数据
-
-```
-
-wget
https://doris-build-1308700295.cos.ap-beijing.myqcloud.com/regression/index/hacknernews_1m.csv.gz
-
-curl --location-trusted -u root: -H "compress_type:gz" -T
hacknernews_1m.csv.gz
http://127.0.0.1:8030/api/test_inverted_index/hackernews_1m/_stream_load
-{
- "TxnId": 2,
- "Label": "a8a3e802-2329-49e8-912b-04c800a461a6",
- "TwoPhaseCommit": "false",
- "Status": "Success",
- "Message": "OK",
- "NumberTotalRows": 1000000,
- "NumberLoadedRows": 1000000,
- "NumberFilteredRows": 0,
- "NumberUnselectedRows": 0,
- "LoadBytes": 130618406,
- "LoadTimeMs": 8988,
- "BeginTxnTimeMs": 23,
- "StreamLoadPutTimeMs": 113,
- "ReadDataTimeMs": 4788,
- "WriteDataTimeMs": 8811,
- "CommitAndPublishTimeMs": 38
-}
-```
-
-- SQL运行count()确认导入数据成功
-
-```sql
-mysql> SELECT count() FROM hackernews_1m;
-+---------+
-| count() |
-+---------+
-| 1000000 |
-+---------+
-1 row in set (0.02 sec)
-```
-
-### 查询
-
-#### 全文检索
-
-- 用LIKE匹配计算comment中含有'OLAP'的行数，耗时0.18s
-```sql
-mysql> SELECT count() FROM hackernews_1m WHERE comment LIKE '%OLAP%';
-+---------+
-| count() |
-+---------+
-| 34 |
-+---------+
-1 row in set (0.18 sec)
-```
-
-- 用基于倒排索引的全文检索MATCH_ANY计算comment中含有'OLAP'的行数，耗时0.02s，加速9倍，在更大的数据集上效果会更加明显
- - 这里结果条数的差异，是因为倒排索引对comment分词后，还会对词进行进行统一成小写等归一化处理，因此MATCH_ANY比LIKE的结果多一些
-```sql
-mysql> SELECT count() FROM hackernews_1m WHERE comment MATCH_ANY 'OLAP';
-+---------+
-| count() |
-+---------+
-| 35 |
-+---------+
-1 row in set (0.02 sec)
-```
-
-- 同样的对比统计'OLTP'出现次数的性能，0.07s vs 0.01s，由于缓存的原因LIKE和MATCH_ANY都有提升，倒排索引仍然有7倍加速
-```sql
-mysql> SELECT count() FROM hackernews_1m WHERE comment LIKE '%OLTP%';
-+---------+
-| count() |
-+---------+
-| 48 |
-+---------+
-1 row in set (0.07 sec)
-
-mysql> SELECT count() FROM hackernews_1m WHERE comment MATCH_ANY 'OLTP';
-+---------+
-| count() |
-+---------+
-| 51 |
-+---------+
-1 row in set (0.01 sec)
-```
-
-- 同时出现'OLAP'和'OLTP'两个词，0.13s vs 0.01s，13倍加速
- - 要求多个词同时出现时（AND关系）使用 MATCH_ALL 'keyword1 keyword2 ...'
-```sql
-mysql> SELECT count() FROM hackernews_1m WHERE comment LIKE '%OLAP%' AND
comment LIKE '%OLTP%';
-+---------+
-| count() |
-+---------+
-| 14 |
-+---------+
-1 row in set (0.13 sec)
-
-mysql> SELECT count() FROM hackernews_1m WHERE comment MATCH_ALL 'OLAP OLTP';
-+---------+
-| count() |
-+---------+
-| 15 |
-+---------+
-1 row in set (0.01 sec)
-```
-
-- 任意出现'OLAP'和'OLTP'其中一个词，0.12s vs 0.01s，12倍加速
- - 只要求多个词任意一个或多个出现时（OR关系）使用 MATCH_ANY 'keyword1 keyword2 ...'
-```sql
-mysql> SELECT count() FROM hackernews_1m WHERE comment LIKE '%OLAP%' OR
comment LIKE '%OLTP%';
-+---------+
-| count() |
-+---------+
-| 68 |
-+---------+
-1 row in set (0.12 sec)
-
-mysql> SELECT count() FROM hackernews_1m WHERE comment MATCH_ANY 'OLAP OLTP';
-+---------+
-| count() |
-+---------+
-| 71 |
-+---------+
-1 row in set (0.01 sec)
-```
-
-
-#### 普通等值、范围查询
-
-- DataTime类型的列范围查询
-```sql
-mysql> SELECT count() FROM hackernews_1m WHERE timestamp > '2007-08-23
04:17:00';
-+---------+
-| count() |
-+---------+
-| 999081 |
-+---------+
-1 row in set (0.03 sec)
-```
-
-- 为timestamp列增加一个倒排索引
-```sql
--- 对于日期时间类型USING INVERTED，不用指定分词
--- CREATE INDEX 是第一种建索引的语法，另外一种在后面展示
-mysql> CREATE INDEX idx_timestamp ON hackernews_1m(timestamp) USING INVERTED;
-Query OK, 0 rows affected (0.03 sec)
-```
-
-- 查看索引创建进度，通过FinishTime和CreateTime的差值，可以看到100万条数据对timestamp列建倒排索引只用了1s
-```sql
-mysql> SHOW ALTER TABLE COLUMN;
-+-------+---------------+-------------------------+-------------------------+---------------+---------+---------------+---------------+---------------+----------+------+----------+---------+
-| JobId | TableName | CreateTime | FinishTime |
IndexName | IndexId | OriginIndexId | SchemaVersion | TransactionId | State
| Msg | Progress | Timeout |
-+-------+---------------+-------------------------+-------------------------+---------------+---------+---------------+---------------+---------------+----------+------+----------+---------+
-| 10030 | hackernews_1m | 2023-02-10 19:44:12.929 | 2023-02-10 19:44:13.938 |
hackernews_1m | 10031 | 10008 | 1:1994690496 | 3 |
FINISHED | | NULL | 2592000 |
-+-------+---------------+-------------------------+-------------------------+---------------+---------+---------------+---------------+---------------+----------+------+----------+---------+
-1 row in set (0.00 sec)
-```
-
-- 索引创建后，范围查询用同样的查询方式，Doris会自动识别索引进行优化，但是这里由于数据量小性能差别不大
-```sql
-mysql> SELECT count() FROM hackernews_1m WHERE timestamp > '2007-08-23
04:17:00';
-+---------+
-| count() |
-+---------+
-| 999081 |
-+---------+
-1 row in set (0.01 sec)
-```
-
-- 在数值类型的列parent进行类似timestamp的操作，这里查询使用等值匹配
-```sql
-mysql> SELECT count() FROM hackernews_1m WHERE parent = 11189;
-+---------+
-| count() |
-+---------+
-| 2 |
-+---------+
-1 row in set (0.01 sec)
-
--- 对于数值类型USING INVERTED，不用指定分词
--- ALTER TABLE t ADD INDEX 是第二种建索引的语法
-mysql> ALTER TABLE hackernews_1m ADD INDEX idx_parent(parent) USING INVERTED;
-Query OK, 0 rows affected (0.01 sec)
-
-mysql> SHOW ALTER TABLE COLUMN;
-+-------+---------------+-------------------------+-------------------------+---------------+---------+---------------+---------------+---------------+----------+------+----------+---------+
-| JobId | TableName | CreateTime | FinishTime |
IndexName | IndexId | OriginIndexId | SchemaVersion | TransactionId | State
| Msg | Progress | Timeout |
-+-------+---------------+-------------------------+-------------------------+---------------+---------+---------------+---------------+---------------+----------+------+----------+---------+
-| 10030 | hackernews_1m | 2023-02-10 19:44:12.929 | 2023-02-10 19:44:13.938 |
hackernews_1m | 10031 | 10008 | 1:1994690496 | 3 |
FINISHED | | NULL | 2592000 |
-| 10053 | hackernews_1m | 2023-02-10 19:49:32.893 | 2023-02-10 19:49:33.982 |
hackernews_1m | 10054 | 10008 | 1:378856428 | 4 |
FINISHED | | NULL | 2592000 |
-+-------+---------------+-------------------------+-------------------------+---------------+---------+---------------+---------------+---------------+----------+------+----------+---------+
-
-mysql> SELECT count() FROM hackernews_1m WHERE parent = 11189;
-+---------+
-| count() |
-+---------+
-| 2 |
-+---------+
-1 row in set (0.01 sec)
-```
-
-- 对字符串类型的author建立不分词的倒排索引，等值查询也可以利用索引加速
-```sql
-mysql> SELECT count() FROM hackernews_1m WHERE author = 'faster';
-+---------+
-| count() |
-+---------+
-| 20 |
-+---------+
-1 row in set (0.03 sec)
-
--- 这里只用了USING INVERTED，不对author分词，整个当做一个词处理
-mysql> ALTER TABLE hackernews_1m ADD INDEX idx_author(author) USING INVERTED;
-Query OK, 0 rows affected (0.01 sec)
-
--- 100万条author数据增量建索引仅消耗1.5s
-mysql> SHOW ALTER TABLE COLUMN;
-+-------+---------------+-------------------------+-------------------------+---------------+---------+---------------+---------------+---------------+----------+------+----------+---------+
-| JobId | TableName | CreateTime | FinishTime |
IndexName | IndexId | OriginIndexId | SchemaVersion | TransactionId | State
| Msg | Progress | Timeout |
-+-------+---------------+-------------------------+-------------------------+---------------+---------+---------------+---------------+---------------+----------+------+----------+---------+
-| 10030 | hackernews_1m | 2023-02-10 19:44:12.929 | 2023-02-10 19:44:13.938 |
hackernews_1m | 10031 | 10008 | 1:1994690496 | 3 |
FINISHED | | NULL | 2592000 |
-| 10053 | hackernews_1m | 2023-02-10 19:49:32.893 | 2023-02-10 19:49:33.982 |
hackernews_1m | 10054 | 10008 | 1:378856428 | 4 |
FINISHED | | NULL | 2592000 |
-| 10076 | hackernews_1m | 2023-02-10 19:54:20.046 | 2023-02-10 19:54:21.521 |
hackernews_1m | 10077 | 10008 | 1:1335127701 | 5 |
FINISHED | | NULL | 2592000 |
-+-------+---------------+-------------------------+-------------------------+---------------+---------+---------------+---------------+---------------+----------+------+----------+---------+
-
--- 创建索引后，字符串等值匹配也有明显加速
-mysql> SELECT count() FROM hackernews_1m WHERE author = 'faster';
-+---------+
-| count() |
-+---------+
-| 20 |
-+---------+
-1 row in set (0.01 sec)
-
-```
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-1.2/data-table/index/ngram-bloomfilter-index.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-1.2/data-table/index/ngram-bloomfilter-index.md
deleted file mode 100644
index ea6304253a1..00000000000
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-1.2/data-table/index/ngram-bloomfilter-index.md
+++ /dev/null
@@ -1,84 +0,0 @@
----
-{
- "title": "NGram BloomFilter索引",
- "language": "zh-CN"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# [Experimental] NGram BloomFilter索引及使用使用场景
-
-<version since="2.0.0">
-</version>
-
-为了提升like的查询性能，增加了NGram BloomFilter索引，其实现主要参照了ClickHouse的ngrambf。
-
-## NGram BloomFilter创建
-
-表创建时指定：
-
-```sql
-CREATE TABLE `table3` (
- `siteid` int(11) NULL DEFAULT "10" COMMENT "",
- `citycode` smallint(6) NULL COMMENT "",
- `username` varchar(32) NULL DEFAULT "" COMMENT "",
- INDEX idx_ngrambf (`username`) USING NGRAM_BF PROPERTIES("gram_size"="3",
"bf_size"="256") COMMENT 'username ngram_bf index'
-) ENGINE=OLAP
-AGGREGATE KEY(`siteid`, `citycode`, `username`) COMMENT "OLAP"
-DISTRIBUTED BY HASH(`siteid`) BUCKETS 10
-PROPERTIES (
-"replication_num" = "1"
-);
-
--- PROPERTIES("gram_size"="3", "bf_size"="256")，分别表示gram的个数和bloom filter的字节数。
--- gram的个数跟实际查询场景相关，通常设置为大部分查询字符串的长度，bloom
filter字节数，可以通过测试得出，通常越大过滤效果越好，可以从256开始进行验证测试看看效果。当然字节数越大也会带来索引存储、内存cost上升。
--- 如果数据基数比较高，字节数可以不用设置过大，如果基数不是很高，可以通过增加字节数来提升过滤效果。
-```
-
-## 查看NGram BloomFilter索引
-
-查看我们在表上建立的NGram BloomFilter索引是使用:
-
-```sql
-show index from example_db.table3;
-```
-
-## 删除NGram BloomFilter索引
-
-
-```sql
-alter table example_db.table3 drop index idx_ngrambf;
-```
-
-## 修改NGram BloomFilter索引
-
-为已有列新增NGram BloomFilter索引：
-
-```sql
-alter table example_db.table3 add index idx_ngrambf(username) using NGRAM_BF
PROPERTIES("gram_size"="2", "bf_size"="512")comment 'username ngram_bf index'
-```
-
-## **Doris NGram BloomFilter使用注意事项**
-
-1. NGram BloomFilter只支持字符串列
-2. NGram BloomFilter索引和BloomFilter索引为互斥关系，即同一个列只能设置两者中的一个
-3. NGram大小和BloomFilter的字节数，可以根据实际情况调优，如果NGram比较小，可以适当增加BloomFilter大小
-4. 如果要查看某个查询是否命中了NGram Bloom Filter索引，可以通过查询的Profile信息查看
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-1.2/query-acceleration/hight-concurrent-point-query.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-1.2/query-acceleration/hight-concurrent-point-query.md
deleted file mode 100644
index 8f962166045..00000000000
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-1.2/query-acceleration/hight-concurrent-point-query.md
+++ /dev/null
@@ -1,103 +0,0 @@
----
-{
- "title": "高并发点查",
- "language": "zh-CN"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# 高并发点查
-
-<version since="2.0.0"></version>
-
-## 背景
-
-Doris 基于列存格式引擎构建，在高并发服务场景中，用户总是希望从系统中获取整行数据。但是，当表宽时，列存格式将大大放大随机读取 IO。Doris
查询引擎和计划对于某些简单的查询（如点查询）来说太重了。需要一个在 FE 的查询规划中规划短路径来处理这样的查询。FE 是 SQL 查询的访问层服务，使用
Java 编写，分析和解析 SQL 也会导致高并发查询的高 CPU 开销。为了解决上述问题，我们在 Doris
中引入了行存、短查询路径、PreparedStatement 来解决上诉问题，下面是开启这些优化的指南。
-
-## 行存
-
-用户可以在 Olap
表中开启行存模式，但是需要额外的空间来存储行存。目前的行存实现是将行存编码后存在单独的一列中，这样做是用于简化行存的实现。行存模式默认是关闭的，如果您想开启则可以在建表语句的
property 中指定如下属性
-
-```
-"store_row_column" = "true"
-```
-
-## 在 Unique 模型下的点查优化
-
-上述的行存用于在 Unique 模型下开启 Merge-On-Write 策略是减少点查时的 IO
开销。当`enable_unique_key_merge_on_write`与`store_row_column`在创建 Unique
表开启时，对于主键的点查会走短路径来对 SQL 执行进行优化，仅需要执行一次 RPC 即可执行完成查询。下面是点查结合行存在 在 Unique 模型下开启
Merge-On-Write 策略的一个例子:
-
-```sql
-CREATE TABLE `tbl_point_query` (
- `key` int(11) NULL,
- `v1` decimal(27, 9) NULL,
- `v2` varchar(30) NULL,
- `v3` varchar(30) NULL,
- `v4` date NULL,
- `v5` datetime NULL,
- `v6` float NULL,
- `v7` datev2 NULL
-) ENGINE=OLAP
-UNIQUE KEY(`key`)
-COMMENT 'OLAP'
-DISTRIBUTED BY HASH(`key`) BUCKETS 1
-PROPERTIES (
- "replication_allocation" = "tag.location.default: 1",
- "enable_unique_key_merge_on_write" = "true",
- "light_schema_change" = "true",
- "store_row_column" = "true"
-);
-```
-
-**注意：**
-1. `enable_unique_key_merge_on_write`应该被开启， 存储引擎需要根据主键来快速点查
-2. 当条件只包含主键时，如`select * from tbl_point_query where key = 123`，类似的查询会走短路径来优化查询
-3. `light_schema_change`应该被开启， 因为主键点查的优化依赖了轻量级 Schema Change 中的`column unique
id`来定位列
-
-## 使用 `PreparedStatement`
-
-为了减少 SQL 解析和表达式计算的开销， 我们在 FE 端提供了与 MySQL
协议完全兼容的`PreparedStatement`特性（目前只支持主键点查）。当`PreparedStatement`在 FE 开启，SQL
和其表达式将被提前计算并缓存到 Session 级别的内存缓存中，后续的查询直接使用缓存对象即可。当 CPU 成为主键点查的瓶颈， 在开启
`PreparedStatement` 后，将会有 4 倍+的性能提升。下面是在 JDBC 中使用 `PreparedStatement` 的例子
-
-1. 设置 JDBC url 并在 Server 端开启 prepared statement
-
-```
-url = jdbc:mysql://127.0.0.1:9030/ycsb?useServerPrepStmts=true
-```
-
-2. 使用 `PreparedStatement`
-
-```java
-// use `?` for placement holders, readStatement should be reused
-PreparedStatement readStatement = conn.prepareStatement("select * from
tbl_point_query where key = ?");
-...
-readStatement.setInt(1234);
-ResultSet resultSet = readStatement.executeQuery();
-...
-readStatement.setInt(1235);
-resultSet = readStatement.executeQuery();
-...
-```
-
-## 开启行缓存
-
-Doris 中有针对 Page 级别的 Cache，每个 Page 中存的是某一列的数据, 所以 Page cache
是针对列的缓存，对于前面提到的行存，一行里包括了多列数据，缓存可能被大查询给刷掉，为了增加行缓存命中率，单独引入了行存缓存，行缓存复用了 Doris 中的
LRU Cache 机制来保障内存的使用，通过指定下面的的BE配置来开启
-
-- `disable_storage_row_cache`是否开启行缓存， 默认不开启
-- `row_cache_mem_limit`指定 Row cache 占用内存的百分比， 默认 20% 内存
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-1.2/query-acceleration/nereids.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-1.2/query-acceleration/nereids.md
deleted file mode 100644
index 6868d102743..00000000000
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-1.2/query-acceleration/nereids.md
+++ /dev/null
@@ -1,87 +0,0 @@
----
-{
- "title": "Nereids 全新优化器",
- "language": "zh-CN"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Nereids 全新优化器
-
-<version since="dev"></version>
-
-## 研发背景
-
-现代查询优化器面临更加复杂的查询语句、更加多样的查询场景等挑战。同时，用户也越来越迫切的希望尽快获得查询结果。旧优化器的陈旧架构，难以满足今后快速迭代的需要。基于此，我们开始着手研发现代架构的全新查询优化器。在更高效的处理当前Doris场景的查询请求同时，提供更好的扩展性，为处理今后
Doris 所需面临的更复杂的需求打下良好的基础。
-
-## 新优化器的优势
-
-### 更智能
-
-新优化器将每个 RBO 和 CBO
的优化点以规则的形式呈现。对于每一个规则，新优化器都提供了一组用于描述查询计划形状的模式，可以精确的匹配可优化的查询计划。基于此，新优化器更好的可以支持诸如多层子查询嵌套等更为复杂的查询语句。
-
-同时新优化器的 CBO 基于先进的 Cascades 框架，使用了更为丰富的数据统计信息，并应用了维度更科学的代价模型。这使得新优化器在面对多表 Join
的查询时，更加得心应手。
-
-TPC-H SF100 查询速度比较。环境为 3BE，新优化器使用原始 SQL ，执行 SQL 前收集了统计信息。旧优化器使用手工调优
SQL。可以看到，新优化器在无需手工优化查询的情况下，总体查询时间与旧优化器手工优化后的查询时间相近。
-
-
-
-### 更健壮
-
-新优化器的所有优化规则，均在逻辑执行计划树上完成。当查询语法语义解析完成后，变转换为一颗树状结构。相比于旧优化器的内部数据结构更为合理、统一。以子查询处理为例，新优化器基于新的数据结构，避免了旧优化器中众多规则对子查询的单独处理。进而减少了优化规则逻辑错误的可能。
-
-### 更灵活
-
-新优化器的架构设计更合理，更现代。可以方便地扩展优化规则和处理阶段。能够更为迅速的响应用户的需求。
-
-## 使用方法
-
-开启新优化器
-
-```sql
-SET enable_nereids_planner=true;
-```
-
-开启自动回退到旧优化器
-
-```sql
-SET enable_fallback_to_original_planner=true;
-```
-
-## 已知问题和暂不支持的功能
-
-### 暂不支持的功能
-
-> 如果开启了自动回退，则会自动回退到旧优化器执行
-
-- Json、Array、Map、Struct 类型：查询的表含有以上类型，或者查询中的函数会输出以上类型
-- DML：所有的 DML 语句，例如 Insert Into Select，Create Table As Select，Update，Delete 等
-- 函数别名
-- Java UDF 和 HDFS UDF
-- 高并发点查询优化
-- 倒排索引
-
-### 已知问题
-
-- 不支持命中 Query Cache 和 Partition Cache
-- 不支持选中多表物化视图
-- 不支持选中使用 2.0 版本新创建物化视图
-- 部分不支持的子查询用法会产生错误结果而不是报错
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-1.2/query-acceleration/pipeline-execution-engine.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-1.2/query-acceleration/pipeline-execution-engine.md
deleted file mode 100644
index f3bcd2b7e7d..00000000000
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-1.2/query-acceleration/pipeline-execution-engine.md
+++ /dev/null
@@ -1,78 +0,0 @@
----
-{
- "title": "Pipeline 执行引擎",
- "language": "zh-CN",
- "toc_min_heading_level": 2,
- "toc_max_heading_level": 4
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Pipeline 执行引擎
-
-<version since="2.0.0"></version>
-
-Pipeline 执行引擎 是 Doris 在 2.0 版本加入的实验性功能。目标是为了替换当前 Doris 的火山模型的执行引擎，充分释放多核 CPU
的计算能力，并对 Doris 的查询线程的数目进行限制，解决 Doris 的执行线程膨胀的问题。
-
-它的具体设计、实现和效果可以参阅 [DSIP-027]([DSIP-027: Support Pipeline Exec Engine - DORIS -
Apache Software
Foundation](https://cwiki.apache.org/confluence/display/DORIS/DSIP-027%3A+Support+Pipeline+Exec+Engine))。
-
-## 原理
-
-当前的Doris的SQL执行引擎是基于传统的火山模型进行设计，在单机多核的场景下存在下面的一些问题：
-* 无法充分利用多核计算能力，提升查询性能，**多数场景下进行性能调优时需要手动设置并行度**，在生产环境中几乎很难进行设定。
-
-* 单机查询的每个 Instance 对应线程池的一个线程，这会带来额外的两个问题。
- *
线程池一旦打满。**Doris的查询引擎会进入假性死锁**，对后续的查询无法响应。**同时有一定概率进入逻辑死锁**的情况：比如所有的线程都在执行一个
Instance 的 Probe 任务。
- * 阻塞的算子会占用线程资源，**而阻塞的线程资源无法让渡给能够调度的 Instance**，整体资源利用率上不去。
-
-* 阻塞算子依赖操作系统的线程调度机制，**线程切换开销较大（尤其在系统混布的场景中）**
-
-由此带来的一系列问题驱动 Doris 需要实现适应现代多核 CPU 的体系结构的执行引擎。
-
-而如下图所示（引用自[Push versus pull-based loop fusion in query engines]([jfp_1800010a
(cambridge.org)](https://www.cambridge.org/core/services/aop-cambridge-core/content/view/D67AE4899E87F4B5102F859B0FC02045/S0956796818000102a.pdf/div-class-title-push-versus-pull-based-loop-fusion-in-query-engines-div.pdf))），Pipeline执行引擎基于多核CPU的特点，重新设计由数据驱动的执行引擎：
-
-
-
-1. 将传统 Pull 拉取的逻辑驱动的执行流程改造为 Push 模型的数据驱动的执行引擎
-2. 阻塞操作异步化，减少了线程切换，线程阻塞导致的执行开销，对于 CPU 的利用更为高效
-3. 控制了执行线程的数目，通过时间片的切换的控制，在混合负载的场景中，减少大查询对于小查询的资源挤占问题
-
-从而提高了 CPU 在混合负载 SQL 上执行时的效率，提升了 SQL 查询的性能。
-
-## 使用方式
-
-### 设置Session变量
-
-#### enable_pipeline_engine
-
-将session变量`enable_pipeline_engine `设置为`true`，则 BE 在进行查询执行时就会默认将 SQL 的执行模型转变
Pipeline 的执行方式。
-
-```
-set enable_pipeline_engine = true;
-```
-
-#### parallel_fragment_exec_instance_num
-
-`parallel_fragment_exec_instance_num`代表了 SQL 查询进行查询并发的 Instance
数目。Doris默认的配置为`1`,这个配置会影响非 Pipeline 执行引擎的查询线程数目，而在 Pipeline
执行引擎中不会有线程数目膨胀的问题。这里推荐配置为`16`，用户也可以实际根据自己的查询情况进行调整。
-
-```
-set parallel_fragment_exec_instance_num = 16;
-```
diff --git a/versioned_docs/version-1.2/data-table/dynamic-schema-table.md
b/versioned_docs/version-1.2/data-table/dynamic-schema-table.md
deleted file mode 100644
index ffb788bebf3..00000000000
--- a/versioned_docs/version-1.2/data-table/dynamic-schema-table.md
+++ /dev/null
@@ -1,135 +0,0 @@
----
-{
- "title": "[Experimental] Dynamie schema table",
- "language": "en"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Dynamic Table
-
-<version since="2.0.0">
-
-Dynamic Table
-
-</version>
-
-
-A dynamic schema table is a special kind of table which schema expands
automatically with the import procedure. Currently, this feature is mainly used
for importing semi-structured data such as JSON. Because JSON is
self-describing, we can extract the schema information from the original
document and infer the final type information. This special table can reduce
manual schema change operations and easily import semi-structured data and
automatically expand its schema.
-
-## Terminology
-- Schema change, changing the structure of the table, such as adding columns,
reducing columns, changing column types
-- Static column, column specified during table creation, such as partition
columns, primary key columns
-- Dynamic column, columns automatically recognized and added during import
-
-## Create dynamic table
-
-```sql
-CREATE DATABASE test_dynamic_table;
-
--- Create table and specify static column types, import will automatically
convert to the type of static column
--- Choose random bucketing
-CREATE TABLE IF NOT EXISTS test_dynamic_table (
- qid bigint,
- `answers.date` array<datetime>,
- `title` string,
- ... -- ... Identifying a table as a dynamic table and
its syntax for dynamic tables.
- )
-DUPLICATE KEY(`qid`)
-DISTRIBUTED BY RANDOM BUCKETS 5
-properties("replication_num" = "1");
-
--- Three Columns are added to the table by default, and their types are
specified
-mysql> DESC test_dynamic_table;
-+--------------+-----------------+------+-------+---------+-------+
-| Field | Type | Null | Key | Default | Extra |
-+--------------+-----------------+------+-------+---------+-------+
-| qid | BIGINT | Yes | true | NULL | |
-| answers.date | ARRAY<DATETIME> | Yes | false | NULL | NONE |
-| user | TEXT | Yes | false | NULL | NONE |
-+--------------+-----------------+------+-------+---------+-------+
-3 rows in set (0.00 sec)
-```
-
-## Importing data
-
-``` sql
--- example1.json
-'{
- "title": "Display Progress Bar at the Time of Processing",
- "qid": "1000000",
- "answers": [
- {"date": "2009-06-16T09:55:57.320", "user": "Micha\u0142 Niklas
(22595)"},
- {"date": "2009-06-17T12:34:22.643", "user": "Jack Njiri (77153)"}
- ],
- "tag": ["vb6", "progress-bar"],
- "user": "Jash",
- "creationdate": "2009-06-16T07:28:42.770"
-}'
-
-curl -X PUT -T example1.json --location-trusted -u root: -H
"read_json_by_line:false" -H "format:json"
http://127.0.0.1:8147/api/regression_test_dynamic_table/test_dynamic_table/_stream_load
-
--- Added five new columns: `title`, `answers.user`, `tag`, `title`,
`creationdate`
--- The types of the three columns: `qid`, `answers.date`, `user` remain the
same as with the table was created
--- The default value of the new array type is an empty array []
-mysql> DESC test_dynamic_table;
-+--------------+-----------------+------+-------+---------+-------+
-| Field | Type | Null | Key | Default | Extra |
-+--------------+-----------------+------+-------+---------+-------+
-| qid | BIGINT | Yes | true | NULL | |
-| answers.date | ARRAY<DATETIME> | Yes | false | NULL | NONE |
-| title | TEXT | Yes | false | NULL | NONE |
-| answers.user | ARRAY<TEXT> | No | false | [] | NONE |
-| tag | ARRAY<TEXT> | No | false | [] | NONE |
-| user | TEXT | Yes | false | NULL | NONE |
-| creationdate | TEXT | Yes | false | NULL | NONE |
-| date | TEXT | Yes | false | NULL | NONE |
-+--------------+-----------------+------+-------+---------+-------+
-
--- Batch import data
--- Specifying -H "read_json_by_line:true", parsing JSON line by line
-curl -X PUT -T example_batch.json --location-trusted -u root: -H
"read_json_by_line:true" -H "format:json"
http://127.0.0.1:8147/api/regression_test_dynamic_table/test_dynamic_table/_stream_load
-
--- Specifying -H "strip_outer_array:true", parsing the entire file as a JSON
array, each element in the array is the same, more efficient parsing way
-curl -X PUT -T example_batch_array.json --location-trusted -u root: -H
"strip_outer_array:true" -H "format:json"
http://127.0.0.1:8147/api/regression_test_dynamic_table/test_dynamic_table/_stream_load
-```
-For a dynamic table, you can also use S3load or Routine load, with similar
usage.
-
-
-## Adding Index to Dynamic Columns
-```sql
--- Create an inverted index on the title column, using English parsing.
-CREATE INDEX title_idx ON test_dynamic_table (`title`) using inverted
PROPERTIES("parser"="english")
-```
-
-## Type conflict resolution
-
-In the first batch import, the unified type will be automatically inferred and
used as the final Column type, so it is recommended to keep the Column type
consistent, for example:
-```
-{"id" : 123}
-{"id" : "123"}
--- The type will finally be inferred as Text type, and if {"id" : 123} is
imported later, the type will automatically be converted to String type
-
-For types that cannot be unified, such as:
-{"id" : [123]}
-{"id" : 123}
--- Importing will result in an error."
-```
\ No newline at end of file
diff --git a/versioned_docs/version-1.2/data-table/index/inverted-index.md
b/versioned_docs/version-1.2/data-table/index/inverted-index.md
deleted file mode 100644
index 34f2b4a258b..00000000000
--- a/versioned_docs/version-1.2/data-table/index/inverted-index.md
+++ /dev/null
@@ -1,405 +0,0 @@
----
-{
- "title": "Inverted index",
- "language": "en"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# [Experimental] Inverted Index
-
-<version since="2.0.0">
-
-</version>
-
-From version 2.0.0, Doris implemented inverted index to support fulltext
search on text field, normal eq and range filter on text, numeric, datetime
field. This doc introduce inverted index usage, including create, drop and
query.
-
-
-## Glossary
-
-- [inverted index](https://en.wikipedia.org/wiki/Inverted_index) is a index
techlogy used in information retirval commonly. It split text into word terms
and construct a term to doc index. This index is called inverted index and can
be used to find the docs where a specific term appears.
-
-
-## Basic Principles
-
-Doris use [CLucene](https://clucene.sourceforge.net/) as its underlying lib
for inverted index. CLucene is a high performance and robust implementation of
the famous Lucene inverted index library written in C++. Doris optimize CLucene
to be more simple, fast and suitable for a database.
-
-In the inverted index of Doris, a row in a table corresponds to a doc in
CLucene, a column corresponds to a field in doc. So using inverted index, doris
can get the rows that meet the filter of SQL WHERE clause, and then get the
rows quickly without reading other unrelated rows.
-
-Doris use a seperate file to store inverted index. It's related to segment
file in logic, but iosolated with each other. The advantange is that, create
and drop inverted index does not need to rewrite tablet and segment file, which
is very heavy work.
-
-
-## Features
-
-The features for inverted index is as follows:
-
-- add fulltext search on text(string, varchar, char) field
- - MATCH_ALL matches all keywords, MATCH_ANY matches any keywords
- - support fulltext on array of text field
- - support english and chinese word parser
-- accelerate normal equal, range query, replacing bitmap index in the future
- - suport =, !=, >, >=, <, <= on text, numeric, datetime types
- - suport =, !=, >, >=, <, <= on array of text, numeric, datetime types
-- complete suport for logic combination
- - add index filter push down for OR, NOT
- - support combination of AND, OR, NOT
-- flexiable and fast index management
- - support inverted index definition on table creation
- - support add inverted index on existed table, without rewrite data
- - support delete inverted index on existed table, without rewrite data
-
-
-## Syntax
-
-- The inverted index definition syntax on table creation is as follows
- - USING INVERTED is mandatory, it specify index type to be inverted index
- - PROPERTIES is optional, it allows user to specify additional properties
for index, "parser" is for type of word tokenizor/parser
- - missing stands for no parser, the whole field is considered to be a term
- - "english" stands for english parser
- - "chinese" stands for chinese parser
- - COMMENT is optional
-
-```sql
-CREATE TABLE table_name
-(
- columns_difinition,
- INDEX idx_name1(column_name1) USING INVERTED [PROPERTIES("parser" =
"english|chinese")] [COMMENT 'your comment']
- INDEX idx_name2(column_name2) USING INVERTED [PROPERTIES("parser" =
"english|chinese")] [COMMENT 'your comment']
-)
-table_properties;
-```
-
-- add an inverted index to existed table
-```sql
--- syntax 1
-CREATE INDEX idx_name ON table_name(column_name) USING INVERTED
[PROPERTIES("parser" = "english|chinese")] [COMMENT 'your comment'];
--- syntax 2
-ALTER TABLE table_name ADD INDEX idx_name(column_name) USING INVERTED
[PROPERTIES("parser" = "english|chinese")] [COMMENT 'your comment'];
-```
-
-- drop an inverted index
-```sql
--- syntax 1
-DROP INDEX idx_name ON table_name;
--- syntax 2
-ALTER TABLE table_name DROP INDEX idx_name;
-```
-
-- speed up query using inverted index
-```sql
--- 1. fulltext search using MATCH_ANY OR MATCH_ALL
-SELECT * FROM table_name WHERE column_name MATCH_ANY | MATCH_ALL 'keyword1
...';
-
--- 1.1 find rows that logmsg contains keyword1
-SELECT * FROM table_name WHERE logmsg MATCH_ANY 'keyword1';
-
--- 1.2 find rows that logmsg contains keyword1 or keyword2 or more keywords
-SELECT * FROM table_name WHERE logmsg MATCH_ANY 'keyword2 keyword2';
-
--- 1.3 find rows that logmsg contains both keyword1 and keyword2 and more
keywords
-SELECT * FROM table_name WHERE logmsg MATCH_ALL 'keyword2 keyword2';
-
-
--- 2. normal equal, range query
-SELECT * FROM table_name WHERE id = 123;
-SELECT * FROM table_name WHERE ts > '2023-01-01 00:00:00';
-SELECT * FROM table_name WHERE op_type IN ('add', 'delete');
-```
-
-## Example
-
-This example will demostrate inverted index creation, fulltext query, normal
query using a hackernews dataset with 1 million rows. The performanc
comparation between using and without inverted index will also be showed.
-
-### Create table
-
-```sql
-
-CREATE DATABASE test_inverted_index;
-
-USE test_inverted_index;
-
--- define inverted index idx_comment for comment column on table creation
--- USING INVERTED specify using inverted index
--- PROPERTIES("parser" = "english") specify english word parser
-CREATE TABLE hackernews_1m
-(
- `id` BIGINT,
- `deleted` TINYINT,
- `type` String,
- `author` String,
- `timestamp` DateTimeV2,
- `comment` String,
- `dead` TINYINT,
- `parent` BIGINT,
- `poll` BIGINT,
- `children` Array<BIGINT>,
- `url` String,
- `score` INT,
- `title` String,
- `parts` Array<INT>,
- `descendants` INT,
- INDEX idx_comment (`comment`) USING INVERTED PROPERTIES("parser" =
"english") COMMENT 'inverted index for comment'
-)
-DUPLICATE KEY(`id`)
-DISTRIBUTED BY HASH(`id`) BUCKETS 10
-PROPERTIES ("replication_num" = "1");
-
-```
-
-
-### Load data
-
-- load data by stream load
-
-```
-
-wget
https://doris-build-1308700295.cos.ap-beijing.myqcloud.com/regression/index/hacknernews_1m.csv.gz
-
-curl --location-trusted -u root: -H "compress_type:gz" -T
hacknernews_1m.csv.gz
http://127.0.0.1:8030/api/test_inverted_index/hackernews_1m/_stream_load
-{
- "TxnId": 2,
- "Label": "a8a3e802-2329-49e8-912b-04c800a461a6",
- "TwoPhaseCommit": "false",
- "Status": "Success",
- "Message": "OK",
- "NumberTotalRows": 1000000,
- "NumberLoadedRows": 1000000,
- "NumberFilteredRows": 0,
- "NumberUnselectedRows": 0,
- "LoadBytes": 130618406,
- "LoadTimeMs": 8988,
- "BeginTxnTimeMs": 23,
- "StreamLoadPutTimeMs": 113,
- "ReadDataTimeMs": 4788,
- "WriteDataTimeMs": 8811,
- "CommitAndPublishTimeMs": 38
-}
-```
-
-- check loaded data by SQL count()
-
-```sql
-mysql> SELECT count() FROM hackernews_1m;
-+---------+
-| count() |
-+---------+
-| 1000000 |
-+---------+
-1 row in set (0.02 sec)
-```
-
-### Query
-
-#### Fulltext search query
-
-- count the rows that comment contains 'OLAP' using LIKE, cost 0.18s
-```sql
-mysql> SELECT count() FROM hackernews_1m WHERE comment LIKE '%OLAP%';
-+---------+
-| count() |
-+---------+
-| 34 |
-+---------+
-1 row in set (0.18 sec)
-```
-
-- count the rows that comment contains 'OLAP' using MATCH_ANY fulltext search
based on inverted index , cost 0.02s and 9x speedup, the speedup will be even
larger on larger dataset
- - the difference of count is due to feature of fulltext. Word parser will
not only split text to words, but also do some normalization such transform to
lower case letters. So the result of MATCH_ANY will be a little more.
-```sql
-mysql> SELECT count() FROM hackernews_1m WHERE comment MATCH_ANY 'OLAP';
-+---------+
-| count() |
-+---------+
-| 35 |
-+---------+
-1 row in set (0.02 sec)
-```
-
-- Semilarly, count on 'OLTP' shows 0.07s vs 0.01s. Due to the cache in Doris,
both LIKE and MATCH_ANY is faster, but there is still 7x speedup.
-```sql
-mysql> SELECT count() FROM hackernews_1m WHERE comment LIKE '%OLTP%';
-+---------+
-| count() |
-+---------+
-| 48 |
-+---------+
-1 row in set (0.07 sec)
-
-mysql> SELECT count() FROM hackernews_1m WHERE comment MATCH_ANY 'OLTP';
-+---------+
-| count() |
-+---------+
-| 51 |
-+---------+
-1 row in set (0.01 sec)
-```
-
-
-- search for both 'OLAP' and 'OLTP', 0.13s vs 0.01s，13x speedup
- - using MATCH_ALL if you need the keywords all appears
-```sql
-mysql> SELECT count() FROM hackernews_1m WHERE comment LIKE '%OLAP%' AND
comment LIKE '%OLTP%';
-+---------+
-| count() |
-+---------+
-| 14 |
-+---------+
-1 row in set (0.13 sec)
-
-mysql> SELECT count() FROM hackernews_1m WHERE comment MATCH_ALL 'OLAP OLTP';
-+---------+
-| count() |
-+---------+
-| 15 |
-+---------+
-1 row in set (0.01 sec)
-```
-
-- search for at least one of 'OLAP' or 'OLTP', 0.12s vs 0.01s，12x speedup
- - using MATCH_ALL if you only need at least one of the keywords appears
-```sql
-mysql> SELECT count() FROM hackernews_1m WHERE comment LIKE '%OLAP%' OR
comment LIKE '%OLTP%';
-+---------+
-| count() |
-+---------+
-| 68 |
-+---------+
-1 row in set (0.12 sec)
-
-mysql> SELECT count() FROM hackernews_1m WHERE comment MATCH_ANY 'OLAP OLTP';
-+---------+
-| count() |
-+---------+
-| 71 |
-+---------+
-1 row in set (0.01 sec)
-```
-
-
-#### normal equal, range query
-
-- range query on DateTime column
-```sql
-mysql> SELECT count() FROM hackernews_1m WHERE timestamp > '2007-08-23
04:17:00';
-+---------+
-| count() |
-+---------+
-| 999081 |
-+---------+
-1 row in set (0.03 sec)
-```
-
-- add inverted index for timestamp column
-```sql
--- for timestamp column, there is no need for word parser, so just USING
INVERTED without PROPERTIES
--- this is the first syntax for CREATE INDEX
-mysql> CREATE INDEX idx_timestamp ON hackernews_1m(timestamp) USING INVERTED;
-Query OK, 0 rows affected (0.03 sec)
-```
-
-- progress of building index can be view by SQL. It just costs 1s (compare
FinishTime and CreateTime) to build index for timestamp column with 1 million
rows.
-```sql
-mysql> SHOW ALTER TABLE COLUMN;
-+-------+---------------+-------------------------+-------------------------+---------------+---------+---------------+---------------+---------------+----------+------+----------+---------+
-| JobId | TableName | CreateTime | FinishTime |
IndexName | IndexId | OriginIndexId | SchemaVersion | TransactionId | State
| Msg | Progress | Timeout |
-+-------+---------------+-------------------------+-------------------------+---------------+---------+---------------+---------------+---------------+----------+------+----------+---------+
-| 10030 | hackernews_1m | 2023-02-10 19:44:12.929 | 2023-02-10 19:44:13.938 |
hackernews_1m | 10031 | 10008 | 1:1994690496 | 3 |
FINISHED | | NULL | 2592000 |
-+-------+---------------+-------------------------+-------------------------+---------------+---------+---------------+---------------+---------------+----------+------+----------+---------+
-1 row in set (0.00 sec)
-```
-
-- after the index is build, Doris will automaticaly use index for range query,
but the performance is almost the same since it's already fast on the small
dataset
-```sql
-mysql> SELECT count() FROM hackernews_1m WHERE timestamp > '2007-08-23
04:17:00';
-+---------+
-| count() |
-+---------+
-| 999081 |
-+---------+
-1 row in set (0.01 sec)
-```
-
-- similary test for parent column with numeric type, using equal query
-```sql
-mysql> SELECT count() FROM hackernews_1m WHERE parent = 11189;
-+---------+
-| count() |
-+---------+
-| 2 |
-+---------+
-1 row in set (0.01 sec)
-
--- do not use word parser for numeric type USING INVERTED
--- use the second syntax ALTER TABLE
-mysql> ALTER TABLE hackernews_1m ADD INDEX idx_parent(parent) USING INVERTED;
-Query OK, 0 rows affected (0.01 sec)
-
-mysql> SHOW ALTER TABLE COLUMN;
-+-------+---------------+-------------------------+-------------------------+---------------+---------+---------------+---------------+---------------+----------+------+----------+---------+
-| JobId | TableName | CreateTime | FinishTime |
IndexName | IndexId | OriginIndexId | SchemaVersion | TransactionId | State
| Msg | Progress | Timeout |
-+-------+---------------+-------------------------+-------------------------+---------------+---------+---------------+---------------+---------------+----------+------+----------+---------+
-| 10030 | hackernews_1m | 2023-02-10 19:44:12.929 | 2023-02-10 19:44:13.938 |
hackernews_1m | 10031 | 10008 | 1:1994690496 | 3 |
FINISHED | | NULL | 2592000 |
-| 10053 | hackernews_1m | 2023-02-10 19:49:32.893 | 2023-02-10 19:49:33.982 |
hackernews_1m | 10054 | 10008 | 1:378856428 | 4 |
FINISHED | | NULL | 2592000 |
-+-------+---------------+-------------------------+-------------------------+---------------+---------+---------------+---------------+---------------+----------+------+----------+---------+
-
-mysql> SELECT count() FROM hackernews_1m WHERE parent = 11189;
-+---------+
-| count() |
-+---------+
-| 2 |
-+---------+
-1 row in set (0.01 sec)
-```
-
-- for text column author, inverted index can also be used to speedup equal
query
-```sql
-mysql> SELECT count() FROM hackernews_1m WHERE author = 'faster';
-+---------+
-| count() |
-+---------+
-| 20 |
-+---------+
-1 row in set (0.03 sec)
-
--- do not use any word parser for author to treat it as a whole
-mysql> ALTER TABLE hackernews_1m ADD INDEX idx_author(author) USING INVERTED;
-Query OK, 0 rows affected (0.01 sec)
-
--- costs 1.5s to build index for author column with 1 million rows.
-mysql> SHOW ALTER TABLE COLUMN;
-+-------+---------------+-------------------------+-------------------------+---------------+---------+---------------+---------------+---------------+----------+------+----------+---------+
-| JobId | TableName | CreateTime | FinishTime |
IndexName | IndexId | OriginIndexId | SchemaVersion | TransactionId | State
| Msg | Progress | Timeout |
-+-------+---------------+-------------------------+-------------------------+---------------+---------+---------------+---------------+---------------+----------+------+----------+---------+
-| 10030 | hackernews_1m | 2023-02-10 19:44:12.929 | 2023-02-10 19:44:13.938 |
hackernews_1m | 10031 | 10008 | 1:1994690496 | 3 |
FINISHED | | NULL | 2592000 |
-| 10053 | hackernews_1m | 2023-02-10 19:49:32.893 | 2023-02-10 19:49:33.982 |
hackernews_1m | 10054 | 10008 | 1:378856428 | 4 |
FINISHED | | NULL | 2592000 |
-| 10076 | hackernews_1m | 2023-02-10 19:54:20.046 | 2023-02-10 19:54:21.521 |
hackernews_1m | 10077 | 10008 | 1:1335127701 | 5 |
FINISHED | | NULL | 2592000 |
-+-------+---------------+-------------------------+-------------------------+---------------+---------+---------------+---------------+---------------+----------+------+----------+---------+
-
--- equal qury on text field autor get 3x speedup
-mysql> SELECT count() FROM hackernews_1m WHERE author = 'faster';
-+---------+
-| count() |
-+---------+
-| 20 |
-+---------+
-1 row in set (0.01 sec)
-
-```
diff --git
a/versioned_docs/version-1.2/data-table/index/ngram-bloomfilter-index.md
b/versioned_docs/version-1.2/data-table/index/ngram-bloomfilter-index.md
deleted file mode 100644
index e3e04eb3156..00000000000
--- a/versioned_docs/version-1.2/data-table/index/ngram-bloomfilter-index.md
+++ /dev/null
@@ -1,82 +0,0 @@
----
-{
- "title": "NGram BloomFilter Index",
- "language": "en"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# [Experimental] NGram BloomFilter Index
-
-<version since="2.0.0">
-</version>
-
-In order to improve the like query performance, the NGram BloomFilter index
was implemented, which referenced to the ClickHouse's ngrambf skip indices;
-
-## Create Column With NGram BloomFilter Index
-
-During create table：
-
-```sql
-CREATE TABLE `table3` (
- `siteid` int(11) NULL DEFAULT "10" COMMENT "",
- `citycode` smallint(6) NULL COMMENT "",
- `username` varchar(100) NULL DEFAULT "" COMMENT "",
- INDEX idx_ngrambf (`username`) USING NGRAM_BF PROPERTIES("gram_size"="3",
"bf_size"="256") COMMENT 'username ngram_bf index'
-) ENGINE=OLAP
-AGGREGATE KEY(`siteid`, `citycode`, `username`) COMMENT "OLAP"
-DISTRIBUTED BY HASH(`siteid`) BUCKETS 10
-PROPERTIES (
-"replication_num" = "1"
-);
-
--- PROPERTIES("gram_size"="3", "bf_size"="1024")，indicate the number of gram
and bytes of bloom filter respectively.
--- the gram size set to same as the like query pattern string length. and the
suitable bytes of bloom filter can be get by test, more larger more better, 256
maybe is a good start.
--- Usually, if the data's cardinality is small, you can increase the bytes of
bloom filter to improve the efficiency.
-```
-
-## Show NGram BloomFilter Index
-
-```sql
-show index from example_db.table3;
-```
-
-## Drop NGram BloomFilter Index
-
-
-```sql
-alter table example_db.table3 drop index idx_ngrambf;
-```
-
-## Add NGram BloomFilter Index
-
-Add NGram BloomFilter Index for old column:
-
-```sql
-alter table example_db.table3 add index idx_ngrambf(username) using NGRAM_BF
PROPERTIES("gram_size"="3", "bf_size"="512")comment 'username ngram_bf index'
-```
-
-## **Some notes about Doris NGram BloomFilter**
-
-1. NGram BloomFilter only support CHAR/VARCHAR/String column.
-2. NGram BloomFilter index and BloomFilter index should be exclusive on same
column
-3. The gram number and bytes of BloomFilter can be adjust and optimize. Like
if gram is too small, you can increase the bytes of BloomFilter.
-4. To find some query whether use the NGram BloomFilter index, you can check
the query profile.
diff --git
a/versioned_docs/version-1.2/query-acceleration/hight-concurrent-point-query.md
b/versioned_docs/version-1.2/query-acceleration/hight-concurrent-point-query.md
deleted file mode 100644
index 76a3b98d378..00000000000
---
a/versioned_docs/version-1.2/query-acceleration/hight-concurrent-point-query.md
+++ /dev/null
@@ -1,106 +0,0 @@
----
-{
- "title": "High-concurrency point query",
- "language": "en"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# High-concurrency point query
-
-<version since="2.0.0"></version>
-
-## Background
-
-Doris is built on a columnar storage format engine. In high-concurrency
service scenarios, users always want to retrieve entire rows of data from the
system. However, when tables are wide, the columnar format greatly amplifies
random read IO. Doris query engine and planner are too heavy for some simple
queries, such as point queries. A short path needs to be planned in the FE's
query plan to handle such queries. FE is the access layer service for SQL
queries, written in Java. Parsing and [...]
-
-## Row Store format
-
-We support a row format for olap table to reduce point lookup io cost,
-but to enable this format, you need to spend more disk space for row format
store.
-Currently, we store row in an extra column called `row column` for
simplicity.Row store is disabled by default,
-users can enable it by adding the following property when create table
-
-```
-"store_row_column" = "true"
-```
-
-## Accelerate point query for unique model
-
-The above row storage is used to enable the Merge-On-Write strategy under the
Unique model to reduce the IO overhead during enumeration. When
`enable_unique_key_merge_on_write` and `store_row_column` are enabled when
creating a Unique table, the query of the primary key will take a short path to
optimize SQL execution, and only one RPC is required to complete the query. The
following is an example of enabling the Merge-On-Write strategy under the
Unique model by combining the query and r [...]
-
-```sql
-CREATE TABLE `tbl_point_query` (
- `key` int(11) NULL,
- `v1` decimal(27, 9) NULL,
- `v2` varchar(30) NULL,
- `v3` varchar(30) NULL,
- `v4` date NULL,
- `v5` datetime NULL,
- `v6` float NULL,
- `v7` datev2 NULL
-) ENGINE=OLAP
-UNIQUE KEY(`key`)
-COMMENT 'OLAP'
-DISTRIBUTED BY HASH(`key)` BUCKETS 1
-PROPERTIES (
- "replication_allocation" = "tag.location.default: 1",
- "enable_unique_key_merge_on_write" = "true",
- "light_schema_change" = "true",
- "store_row_column" = "true"
-);
-```
-
-**Note:**
-1. `enable_unique_key_merge_on_write` should be enabled, since we need primary
key for quick point lookup in storage engine
-2. when condition only contains primary key like `select * from
tbl_point_query where key = 123`, such query will go through the short fast path
-3. `light_schema_change` should also been enabled since we rely on `column
unique id` of each column when doing a point query.
-
-## Using `PreparedStatement`
-
-In order to reduce CPU cost for parsing query SQL and SQL expressions, we
provide `PreparedStatement` feature in FE fully compatible with mysql protocol
(currently only support point queries like above mentioned).Enable it will pre
caculate PreparedStatement SQL and expresions and caches it in a session level
memory buffer and will be reused later on.We could improve 4x+ performance by
using `PreparedStatement` when CPU became hotspot doing such queries.Bellow is
an JDBC example of using [...]
-
-1. Setup JDBC url and enable server side prepared statement
-
-```
-url = jdbc:mysql://127.0.0.1:9030/ycsb?useServerPrepStmts=true
-```
-
-2. Using `PreparedStatement`
-
-```java
-// use `?` for placement holders, readStatement should be reused
-PreparedStatement readStatement = conn.prepareStatement("select * from
tbl_point_query where key = ?");
-...
-readStatement.setInt(1234);
-ResultSet resultSet = readStatement.executeQuery();
-...
-readStatement.setInt(1235);
-resultSet = readStatement.executeQuery();
-...
-```
-
-## Enable row cache
-Doris has a page-level cache that stores data for a specific column in each
page. Therefore, the page cache is a column-based cache. For the row storage
mentioned earlier, a row contains data for multiple columns, and the cache may
be evicted by large queries, which can reduce the hit rate. To increase the hit
rate of the row cache, a separate row cache is introduced, which reuses the LRU
cache mechanism in Doris to ensure memory usage. You can enable it by
specifying the following BE co [...]
-
-- `disable_storage_row_cache` : Whether to enable the row cache. It is not
enabled by default.
-- `row_cache_mem_limit` : Specifies the percentage of memory occupied by the
row cache. The default is 20% of memory.
-
diff --git a/versioned_docs/version-1.2/query-acceleration/nereids.md
b/versioned_docs/version-1.2/query-acceleration/nereids.md
deleted file mode 100644
index 68260aebcb4..00000000000
--- a/versioned_docs/version-1.2/query-acceleration/nereids.md
+++ /dev/null
@@ -1,87 +0,0 @@
----
-{
- "title": "Nereids-The Brand New Planner",
- "language": "en"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Nereids-The Brand New Planner
-
-<version since="dev"></version>
-
-## R&D background
-
-Modern query optimizers face challenges such as more complex query statements
and more diverse query scenarios. At the same time, users are more and more
eager to obtain query results as soon as possible. The outdated architecture of
the old optimizer is difficult to meet the needs of rapid iteration in the
future. Based on this, we set out to develop a new query optimizer for modern
architectures. While processing the query requests of the current Doris scene
more efficiently, it provid [...]
-
-## Advantages of the new optimizer
-
-### Smarter
-
-The new optimizer presents the optimization points of each RBO and CBO in the
form of rules. For each rule, the new optimizer provides a set of patterns used
to describe the shape of the query plan, which can exactly match the query plan
that can be optimized. Based on this, the new optimizer can better support more
complex query statements such as multi-level subquery nesting.
-
-At the same time, the CBO of the new optimizer is based on the advanced
cascades framework, uses richer data statistics, and applies a cost model with
more scientific dimensions. This makes the new optimizer more handy when faced
with multi-table join queries.
-
-TPC-H SF100 query speed comparison. The environment is 3BE, the new optimizer
uses the original SQL, and the statistical information is collected before
executing the SQL. Old optimizers use hand-tuned SQL. It can be seen that the
new optimizer does not need to manually optimize the query, and the overall
query time is similar to that of the old optimizer after manual optimization.
-
-
-
-### more robust
-
-All optimization rules of the new optimizer are completed on the logical
execution plan tree. After the query syntax and semantic analysis is completed,
it will be transformed into a tree structure. Compared with the internal data
structure of the old optimizer, it is more reasonable and unified. Taking
subquery processing as an example, the new optimizer is based on a new data
structure, which avoids separate processing of subqueries by many rules in the
old optimizer. In turn, the poss [...]
-
-### more flexible
-
-The architectural design of the new optimizer is more reasonable and modern.
Optimization rules and processing stages can be easily extended. Can more
quickly respond to user needs.
-
-## How to use
-
-Turn on Nereids
-
-```sql
-SET enable_nereids_planner=true;
-```
-
-Turn on auto fall back to legacy planner
-
-```sql
-SET enable_fallback_to_original_planner=true;
-```
-
-## Known issues and temporarily unsupported features
-
-### temporarily unsupported features
-
-> If automatic fallback is enabled, it will automatically fall back to the old
optimizer execution
-
-- Json、Array、Map and Struct types: The table in the query contains the above
types, or the expressions in the query outputs the above types
-- DML: All DML statements such as Insert Into Select, Create Table As Select,
Update, Delete, etc.
-- Function alias
-- Java UDF and HDFS UDF
-- High concurrent point query optimize
-- Inverted index
-
-### known issues
-
-- Cannot use query cache and partition cache to accelarate query
-- Not support MTMV
-- Not support MV created after version 2.0.0
-- Some unsupported subquery usage will produce an error result instead of an
error
diff --git
a/versioned_docs/version-1.2/query-acceleration/pipeline-execution-engine.md
b/versioned_docs/version-1.2/query-acceleration/pipeline-execution-engine.md
deleted file mode 100644
index ecebea78eea..00000000000
--- a/versioned_docs/version-1.2/query-acceleration/pipeline-execution-engine.md
+++ /dev/null
@@ -1,78 +0,0 @@
----
-{
- "title": "Pipeline execution engine",
- "language": "en",
- "toc_min_heading_level": 2,
- "toc_max_heading_level": 4
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Pipeline execution engine
-
-<version since="2.0.0"></version>
-
-Pipeline execution engine is an experimental feature added by Doris in version
2.0. The goal is to replace the current execution engine of Doris's volcano
model, fully release the computing power of multi-core CPUs, and limit the
number of Doris's query threads to solve the problem of Doris's execution
thread bloat.
-
-Its specific design, implementation and effects can be found in
[DSIP-027]([DSIP-027: Support Pipeline Exec Engine - DORIS - Apache Software
Foundation](https://cwiki.apache.org/confluence/display/DORIS/DSIP-027%3A+Support+Pipeline+Exec+Engine))。
-
-## Principle
-
-The current Doris SQL execution engine is designed based on the traditional
volcano model, which has the following problems in a single multi-core scenario：
-* Inability to take full advantage of multi-core computing power to improve
query performance,**most scenarios require manual setting of parallelism** for
performance tuning, which is almost difficult to set in production environments.
-
-* Each instance of a standalone query corresponds to one thread of the thread
pool, which introduces two additional problems.
- * Once the thread pool is hit full. **Doris' query engine will enter a
pseudo-deadlock** and will not respond to subsequent queries. **At the same
time there is a certain probability of entering a logical deadlock** situation:
for example, all threads are executing an instance's probe task.
- * Blocking arithmetic will take up thread resources,**blocking thread
resources can not be yielded to instances that can be scheduled**, the overall
resource utilization does not go up.
-
-* Blocking arithmetic relies on the OS thread scheduling mechanism, **thread
switching overhead (especially in the scenario of system mixing)）**
-
-The resulting set of problems drove Doris to implement an execution engine
adapted to the architecture of modern multi-core CPUs.
-
-And as shown in the figure below (quoted from[Push versus pull-based loop
fusion in query engines]([jfp_1800010a
(cambridge.org)](https://www.cambridge.org/core/services/aop-cambridge-core/content/view/D67AE4899E87F4B5102F859B0FC02045/S0956796818000102a.pdf/div-class-title-push-versus-pull-based-loop-fusion-in-query-engines-div.pdf))），The
resulting set of problems drove Doris to implement an execution engine adapted
to the architecture of modern multi-core CPUs.：
-
-
-
-1. Transformation of the traditional pull pull logic-driven execution process
into a data-driven execution engine for the push model
-2. Blocking operations are asynchronous, reducing the execution overhead
caused by thread switching and thread blocking and making more efficient use of
the CPU
-3. Controls the number of threads to be executed and reduces the resource
congestion of large queries on small queries in mixed load scenarios by
controlling time slice switching
-
-This improves the efficiency of CPU execution on mixed-load SQL and enhances
the performance of SQL queries.
-
-## Usage
-
-### Set session variable
-
-#### enable_pipeline_engine
-
-This improves the efficiency of CPU execution on mixed-load SQL and enhances
the performance of SQL queries
-
-```
-set enable_pipeline_engine = true;
-```
-
-#### parallel_fragment_exec_instance_num
-
-The default configuration of `parallel_fragment_exec_instance_num` represents
the number of instances that a SQL query will query concurrently; Doris
defaults to `1`, which affects the number of query threads in the non-Pipeline
execution engine, whereas in the Pipeline execution engine there is no thread
inflation This configuration affects the number of threads in the Non-Pipeline
execution engine. The recommended configuration here is `16`, but users can
actually adjust it to suit the [...]
-
-```
-set parallel_fragment_exec_instance_num = 16;
-```
diff --git a/versioned_sidebars/version-1.2-sidebars.json
b/versioned_sidebars/version-1.2-sidebars.json
index c543983e053..2b88321d73b 100644
--- a/versioned_sidebars/version-1.2-sidebars.json
+++ b/versioned_sidebars/version-1.2-sidebars.json
@@ -50,15 +50,12 @@
"data-table/basic-usage",
"data-table/hit-the-rollup",
"data-table/best-practice",
- "data-table/dynamic-schema-table",
{
"type": "category",
"label": "Index",
"items": [
"data-table/index/prefix-index",
- "data-table/index/inverted-index",
"data-table/index/bloomfilter",
- "data-table/index/ngram-bloomfilter-index",
"data-table/index/bitmap-index"
]
}
@@ -179,9 +176,6 @@
"type": "category",
"label": "Query Acceleration",
"items": [
- "query-acceleration/pipeline-execution-engine",
- "query-acceleration/nereids",
- "query-acceleration/hight-concurrent-point-query",
"query-acceleration/materialized-view",
{
"type": "category",
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
