This is an automated email from the ASF dual-hosted git repository.
kxiao pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new 72c925bafea [branch-2.0][catalog](paimon)paimon catalog supports
complex types #25364 #25834 (#26190)
72c925bafea is described below
commit 72c925bafeaa98aa8713c67006f28996a51dc6d8
Author: DongLiang-0 <[email protected]>
AuthorDate: Wed Nov 1 19:40:01 2023 +0800
[branch-2.0][catalog](paimon)paimon catalog supports complex types #25364
#25834 (#26190)
---
docs/en/docs/lakehouse/multi-catalog/paimon.md | 25 +-
docs/zh-CN/docs/lakehouse/multi-catalog/paimon.md | 25 +-
.../apache/doris/common/jni/vec/ColumnType.java | 5 +-
.../org/apache/doris/paimon/PaimonColumnValue.java | 34 +-
.../org/apache/doris/paimon/PaimonJniScanner.java | 1 -
.../org/apache/doris/paimon/PaimonTableCache.java | 2 -
.../org/apache/doris/paimon/PaimonTypeUtils.java | 28 +-
.../catalog/external/PaimonExternalTable.java | 10 +
.../paimon/test_paimon_catalog.out | 388 +++++++++++++++++++++
.../paimon/test_paimon_catalog.groovy | 157 +++++++++
10 files changed, 655 insertions(+), 20 deletions(-)
diff --git a/docs/en/docs/lakehouse/multi-catalog/paimon.md
b/docs/en/docs/lakehouse/multi-catalog/paimon.md
index 062565fba6f..29c6a364e38 100644
--- a/docs/en/docs/lakehouse/multi-catalog/paimon.md
+++ b/docs/en/docs/lakehouse/multi-catalog/paimon.md
@@ -33,7 +33,7 @@ under the License.
## Instructions for use
1. When data in hdfs,need to put core-site.xml, hdfs-site.xml and
hive-site.xml in the conf directory of FE and BE. First read the hadoop
configuration file in the conf directory, and then read the related to the
environment variable `HADOOP_CONF_DIR` configuration file.
-2. The currently adapted version of the payment is 0.4.0
+2. The currently adapted version of the payment is 0.5.0
## Create Catalog
@@ -64,7 +64,7 @@ CREATE CATALOG `paimon_hdfs` PROPERTIES (
> Note that.
>
-> user need download
[paimon-s3-0.4.0-incubating.jar](https://repo.maven.apache.org/maven2/org/apache/paimon/paimon-s3/0.4.0-incubating/paimon-s3-0.4.0-incubating.jar)
+> user need download
[paimon-s3-0.5.0-incubating.jar](https://repo.maven.apache.org/maven2/org/apache/paimon/paimon-s3/0.5.0-incubating/paimon-s3-0.5.0-incubating.jar)
>
> Place it in directory
> ${DORIS_HOME}/be/lib/java_extensions/preload-extensions and restart be
>
@@ -85,7 +85,7 @@ CREATE CATALOG `paimon_s3` PROPERTIES (
>Note that.
>
-> user need download
[paimon-oss-0.4.0-incubating.jar](https://repo.maven.apache.org/maven2/org/apache/paimon/paimon-oss/0.4.0-incubating/paimon-oss-0.4.0-incubating.jar)
+> user need download
[paimon-oss-0.5.0-incubating.jar](https://repo.maven.apache.org/maven2/org/apache/paimon/paimon-oss/0.5.0-incubating/paimon-oss-0.5.0-incubating.jar)
> Place it in directory
> ${DORIS_HOME}/be/lib/java_extensions/preload-extensions and restart be
@@ -120,4 +120,21 @@ CREATE CATALOG `paimon_hms` PROPERTIES (
## Column Type Mapping
-Same as that in Hive Catalogs. See the relevant section in [Hive](./hive.md).
+| Paimon Data Type | Doris Data Type | Comment
|
+|---------------------------------------|---------------------------|-----------|
+| BooleanType | Boolean |
|
+| TinyIntType | TinyInt |
|
+| SmallIntType | SmallInt |
|
+| IntType | Int |
|
+| FloatType | Float |
|
+| BigIntType | BigInt |
|
+| DoubleType | Double |
|
+| VarCharType | VarChar |
|
+| CharType | Char |
|
+| DecimalType(precision, scale) | Decimal(precision, scale) |
|
+| TimestampType,LocalZonedTimestampType | DateTime |
|
+| DateType | Date |
|
+| MapType | Map | Support
Map nesting |
+| ArrayType | Array | Support
Array nesting |
+| VarBinaryType, BinaryType | Binary |
|
+
diff --git a/docs/zh-CN/docs/lakehouse/multi-catalog/paimon.md
b/docs/zh-CN/docs/lakehouse/multi-catalog/paimon.md
index 90443fef6f9..ab11cc32c7b 100644
--- a/docs/zh-CN/docs/lakehouse/multi-catalog/paimon.md
+++ b/docs/zh-CN/docs/lakehouse/multi-catalog/paimon.md
@@ -33,7 +33,7 @@ under the License.
## 使用须知
1. 数据放在hdfs时，需要将 core-site.xml，hdfs-site.xml 和 hive-site.xml 放到 FE 和 BE 的
conf 目录下。优先读取 conf 目录下的 hadoop 配置文件，再读取环境变量 `HADOOP_CONF_DIR` 的相关配置文件。
-2. 当前适配的paimon版本为0.4.0
+2. 当前适配的paimon版本为0.5.0
## 创建 Catalog
@@ -64,7 +64,7 @@ CREATE CATALOG `paimon_hdfs` PROPERTIES (
> 注意：
>
->
用户需要手动下载[paimon-s3-0.4.0-incubating.jar](https://repo.maven.apache.org/maven2/org/apache/paimon/paimon-s3/0.4.0-incubating/paimon-s3-0.4.0-incubating.jar)
+>
用户需要手动下载[paimon-s3-0.5.0-incubating.jar](https://repo.maven.apache.org/maven2/org/apache/paimon/paimon-s3/0.5.0-incubating/paimon-s3-0.5.0-incubating.jar)
> 放在${DORIS_HOME}/be/lib/java_extensions/preload-extensions目录下并重启be。
>
@@ -85,7 +85,7 @@ CREATE CATALOG `paimon_s3` PROPERTIES (
>注意：
>
->
用户需要手动下载[paimon-oss-0.4.0-incubating.jar](https://repo.maven.apache.org/maven2/org/apache/paimon/paimon-oss/0.4.0-incubating/paimon-oss-0.4.0-incubating.jar)
+>
用户需要手动下载[paimon-oss-0.5.0-incubating.jar](https://repo.maven.apache.org/maven2/org/apache/paimon/paimon-oss/0.5.0-incubating/paimon-oss-0.5.0-incubating.jar)
> 放在${DORIS_HOME}/be/lib/java_extensions/preload-extensions目录下并重启be
```sql
@@ -119,5 +119,22 @@ CREATE CATALOG `paimon_hms` PROPERTIES (
## 列类型映射
-和 Hive Catalog 基本一致，可参阅 [Hive Catalog](./hive.md) 中 **列类型映射** 一节。
+| Paimon Data Type | Doris Data Type | Comment
|
+|---------------------------------------|---------------------------|-----------|
+| BooleanType | Boolean |
|
+| TinyIntType | TinyInt |
|
+| SmallIntType | SmallInt |
|
+| IntType | Int |
|
+| FloatType | Float |
|
+| BigIntType | BigInt |
|
+| DoubleType | Double |
|
+| VarCharType | VarChar |
|
+| CharType | Char |
|
+| DecimalType(precision, scale) | Decimal(precision, scale) |
|
+| TimestampType,LocalZonedTimestampType | DateTime |
|
+| DateType | Date |
|
+| MapType | Map | 支持Map嵌套
|
+| ArrayType | Array |
支持Array嵌套 |
+| VarBinaryType, BinaryType | Binary |
|
+
diff --git
a/fe/be-java-extensions/java-common/src/main/java/org/apache/doris/common/jni/vec/ColumnType.java
b/fe/be-java-extensions/java-common/src/main/java/org/apache/doris/common/jni/vec/ColumnType.java
index 1cf7e887d28..a88f9d04b9a 100644
---
a/fe/be-java-extensions/java-common/src/main/java/org/apache/doris/common/jni/vec/ColumnType.java
+++
b/fe/be-java-extensions/java-common/src/main/java/org/apache/doris/common/jni/vec/ColumnType.java
@@ -334,8 +334,9 @@ public class ColumnType {
String keyValue = lowerCaseType.substring(4,
lowerCaseType.length() - 1);
int index = findNextNestedField(keyValue);
if (index != keyValue.length() && index != 0) {
- ColumnType keyType = parseType("key",
keyValue.substring(0, index));
- ColumnType valueType = parseType("value",
keyValue.substring(index + 1));
+ ColumnType keyType = parseType("key",
keyValue.substring(0, index).trim());
+ ColumnType valueType =
+ parseType("value",
keyValue.substring(index + 1).trim());
ColumnType mapType = new ColumnType(columnName,
Type.MAP);
mapType.setChildTypes(Arrays.asList(keyType,
valueType));
return mapType;
diff --git
a/fe/be-java-extensions/paimon-scanner/src/main/java/org/apache/doris/paimon/PaimonColumnValue.java
b/fe/be-java-extensions/paimon-scanner/src/main/java/org/apache/doris/paimon/PaimonColumnValue.java
index a8783fe5294..c55b80b6e86 100644
---
a/fe/be-java-extensions/paimon-scanner/src/main/java/org/apache/doris/paimon/PaimonColumnValue.java
+++
b/fe/be-java-extensions/paimon-scanner/src/main/java/org/apache/doris/paimon/PaimonColumnValue.java
@@ -20,6 +20,9 @@ package org.apache.doris.paimon;
import org.apache.doris.common.jni.vec.ColumnType;
import org.apache.doris.common.jni.vec.ColumnValue;
+import org.apache.paimon.data.DataGetters;
+import org.apache.paimon.data.InternalArray;
+import org.apache.paimon.data.InternalMap;
import org.apache.paimon.data.InternalRow;
import java.math.BigDecimal;
@@ -32,12 +35,18 @@ import java.util.List;
public class PaimonColumnValue implements ColumnValue {
private int idx;
- private InternalRow record;
- ColumnType dorisType;
+ private DataGetters record;
+ private ColumnType dorisType;
public PaimonColumnValue() {
}
+ public PaimonColumnValue(DataGetters record, int idx, ColumnType
columnType) {
+ this.idx = idx;
+ this.record = record;
+ this.dorisType = columnType;
+ }
+
public void setIdx(int idx, ColumnType dorisType) {
this.idx = idx;
this.dorisType = dorisType;
@@ -130,12 +139,29 @@ public class PaimonColumnValue implements ColumnValue {
@Override
public void unpackArray(List<ColumnValue> values) {
-
+ InternalArray recordArray = record.getArray(idx);
+ for (int i = 0; i < recordArray.size(); i++) {
+ PaimonColumnValue arrayColumnValue = new
PaimonColumnValue((DataGetters) recordArray, i,
+ dorisType.getChildTypes().get(0));
+ values.add(arrayColumnValue);
+ }
}
@Override
public void unpackMap(List<ColumnValue> keys, List<ColumnValue> values) {
-
+ InternalMap map = record.getMap(idx);
+ InternalArray key = map.keyArray();
+ for (int i = 0; i < key.size(); i++) {
+ PaimonColumnValue keyColumnValue = new
PaimonColumnValue((DataGetters) key, i,
+ dorisType.getChildTypes().get(0));
+ keys.add(keyColumnValue);
+ }
+ InternalArray value = map.valueArray();
+ for (int i = 0; i < value.size(); i++) {
+ PaimonColumnValue valueColumnValue = new
PaimonColumnValue((DataGetters) value, i,
+ dorisType.getChildTypes().get(1));
+ values.add(valueColumnValue);
+ }
}
@Override
diff --git
a/fe/be-java-extensions/paimon-scanner/src/main/java/org/apache/doris/paimon/PaimonJniScanner.java
b/fe/be-java-extensions/paimon-scanner/src/main/java/org/apache/doris/paimon/PaimonJniScanner.java
index 4e3cda8222f..524f05edd23 100644
---
a/fe/be-java-extensions/paimon-scanner/src/main/java/org/apache/doris/paimon/PaimonJniScanner.java
+++
b/fe/be-java-extensions/paimon-scanner/src/main/java/org/apache/doris/paimon/PaimonJniScanner.java
@@ -40,7 +40,6 @@ import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
-
public class PaimonJniScanner extends JniScanner {
private static final Logger LOG =
LoggerFactory.getLogger(PaimonJniScanner.class);
private static final String PAIMON_OPTION_PREFIX = "paimon_option_prefix.";
diff --git
a/fe/be-java-extensions/paimon-scanner/src/main/java/org/apache/doris/paimon/PaimonTableCache.java
b/fe/be-java-extensions/paimon-scanner/src/main/java/org/apache/doris/paimon/PaimonTableCache.java
index caf1f156de5..f57ffeb5592 100644
---
a/fe/be-java-extensions/paimon-scanner/src/main/java/org/apache/doris/paimon/PaimonTableCache.java
+++
b/fe/be-java-extensions/paimon-scanner/src/main/java/org/apache/doris/paimon/PaimonTableCache.java
@@ -34,7 +34,6 @@ import java.util.Map;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;
-
public class PaimonTableCache {
private static final Logger LOG =
LoggerFactory.getLogger(PaimonTableCache.class);
// Max cache num of paimon table
@@ -83,7 +82,6 @@ public class PaimonTableCache {
}
}
-
public static class TableExt {
private Table table;
private long createTime;
diff --git
a/fe/be-java-extensions/paimon-scanner/src/main/java/org/apache/doris/paimon/PaimonTypeUtils.java
b/fe/be-java-extensions/paimon-scanner/src/main/java/org/apache/doris/paimon/PaimonTypeUtils.java
index b2a9450a6de..575809c5df9 100644
---
a/fe/be-java-extensions/paimon-scanner/src/main/java/org/apache/doris/paimon/PaimonTypeUtils.java
+++
b/fe/be-java-extensions/paimon-scanner/src/main/java/org/apache/doris/paimon/PaimonTypeUtils.java
@@ -45,6 +45,10 @@ import org.apache.paimon.types.VarCharType;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+
/**
* Convert paimon type to doris type.
*/
@@ -56,9 +60,11 @@ public class PaimonTypeUtils {
public static ColumnType fromPaimonType(String columnName, DataType type) {
PaimonColumnType paimonColumnType =
type.accept(PaimonToDorisTypeVisitor.INSTANCE);
- return new ColumnType(columnName, paimonColumnType.getType(),
paimonColumnType.getLength(),
+ ColumnType columnType = new ColumnType(columnName,
paimonColumnType.getType(), paimonColumnType.getLength(),
paimonColumnType.getPrecision(),
paimonColumnType.getScale());
+ columnType.setChildTypes(paimonColumnType.getChildTypes());
+ return columnType;
}
private static class PaimonToDorisTypeVisitor extends
DataTypeDefaultVisitor<PaimonColumnType> {
@@ -153,7 +159,10 @@ public class PaimonTypeUtils {
@Override
public PaimonColumnType visit(ArrayType arrayType) {
- return this.defaultMethod(arrayType);
+ PaimonColumnType paimonColumnType = new
PaimonColumnType(Type.ARRAY);
+ ColumnType elementColumnType = fromPaimonType("dummy-element",
arrayType.getElementType());
+
paimonColumnType.setChildTypes(Collections.singletonList(elementColumnType));
+ return paimonColumnType;
}
@Override
@@ -163,7 +172,11 @@ public class PaimonTypeUtils {
@Override
public PaimonColumnType visit(MapType mapType) {
- return this.defaultMethod(mapType);
+ PaimonColumnType paimonColumnType = new PaimonColumnType(Type.MAP);
+ ColumnType key = fromPaimonType("dummy-key", mapType.getKeyType());
+ ColumnType value = fromPaimonType("dummy-value",
mapType.getValueType());
+ paimonColumnType.setChildTypes(Arrays.asList(key, value));
+ return paimonColumnType;
}
@Override
@@ -184,6 +197,7 @@ public class PaimonTypeUtils {
private int length;
private int precision;
private int scale;
+ private List<ColumnType> childTypes;
public PaimonColumnType(Type type) {
this.type = type;
@@ -225,5 +239,13 @@ public class PaimonTypeUtils {
public void setPrecision(int precision) {
this.precision = precision;
}
+
+ public void setChildTypes(List<ColumnType> childTypes) {
+ this.childTypes = childTypes;
+ }
+
+ public List<ColumnType> getChildTypes() {
+ return childTypes;
+ }
}
}
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/catalog/external/PaimonExternalTable.java
b/fe/fe-core/src/main/java/org/apache/doris/catalog/external/PaimonExternalTable.java
index 2ad593b1b38..679a7842975 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/catalog/external/PaimonExternalTable.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/catalog/external/PaimonExternalTable.java
@@ -31,8 +31,10 @@ import org.apache.logging.log4j.Logger;
import org.apache.paimon.schema.TableSchema;
import org.apache.paimon.table.AbstractFileStoreTable;
import org.apache.paimon.table.Table;
+import org.apache.paimon.types.ArrayType;
import org.apache.paimon.types.DataField;
import org.apache.paimon.types.DecimalType;
+import org.apache.paimon.types.MapType;
import java.util.HashMap;
import java.util.List;
@@ -111,6 +113,14 @@ public class PaimonExternalTable extends ExternalTable {
case TIMESTAMP_WITHOUT_TIME_ZONE:
case TIMESTAMP_WITH_LOCAL_TIME_ZONE:
return
ScalarType.createDatetimeV2Type(PAIMON_DATETIME_SCALE_MS);
+ case ARRAY:
+ ArrayType arrayType = (ArrayType) dataType;
+ Type innerType =
paimonPrimitiveTypeToDorisType(arrayType.getElementType());
+ return org.apache.doris.catalog.ArrayType.create(innerType,
true);
+ case MAP:
+ MapType mapType = (MapType) dataType;
+ return new org.apache.doris.catalog.MapType(
+ paimonTypeToDorisType(mapType.getKeyType()),
paimonTypeToDorisType(mapType.getValueType()));
case TIME_WITHOUT_TIME_ZONE:
return Type.UNSUPPORTED;
default:
diff --git
a/regression-test/data/external_table_p0/paimon/test_paimon_catalog.out
b/regression-test/data/external_table_p0/paimon/test_paimon_catalog.out
index 16884b82e80..2c8a3d5cd26 100644
--- a/regression-test/data/external_table_p0/paimon/test_paimon_catalog.out
+++ b/regression-test/data/external_table_p0/paimon/test_paimon_catalog.out
@@ -66,3 +66,391 @@
-- !c22 --
+-- !c23 --
+1 [1111, 2222, 3333] {"a_test":1}
+2 [4444, 5555, 6666] {"b_test":0, "bbb":1}
+3 [7777, 8888, 9999] {"c_test":1, "ccc":0}
+
+-- !c24 --
+1 [1111, 2222, 3333] {"a_test":1}
+
+-- !c25 --
+true \N \N \N
+\N false true \N
+\N \N \N false
+
+-- !c26 --
+3333
+6666
+9999
+
+-- !c27 --
+\N [1, NULL, NULL] [1, NULL, NULL, NULL] [4, NULL, NULL] [NULL, NULL]
[NULL, 12, 13] [NULL, 15] [NULL, NULL] [18, 19, 20] [21.11, 22, 22]
[NULL] [NULL] [NULL, NULL, 2023-10-24] ["13str", "arr_string"]
["14varchar", "arr_varcha"] [NULL, NULL] [NULL, NULL] [NULL, NULL,
NULL] [NULL, NULL] {1:NULL, 0:NULL} {1:NULL, 0:NULL}
{1:NULL, 2:NULL} {1:6, 0:NULL} {1:7} {0:NULL, 1:20} {1:NULL}
{1:NULL} {0:21.11, 1:22.22} {1:23.333} {0:24.44, 1:NULL}
{1:NULL, 4:2023-02-03} {0:"byte_string"} {0:"aa", 1:" [...]
+1 [1, 0] [1, 2, 3] [4, 5, 6] [7, 8, 9] [10, 11, 12,
13] [14, 15] [16, 17] [18, 19, 20] [21.11, 22, 22]
[23.333] [24.44] [2020-02-02, 2023-10-24] ["13str", "arr_string"]
["14varchar", "arr_varcha"] ["a", "b"] [1, 0, 1] ["aaaa", "bbb"]
[2023-10-24 16:10:40.529, 2023-10-24 16:10:40.529] {1:0, 0:1} {0:2,
1:3} {1:4, 0:5} {1:6, 0:9} {1:7} {0:19, 1:20} {1:21}
{1:22} {0:21.11, 1:22.22} {1:23.333} {0:24.44, 1:0.00}
{1:2021-02-03, 4:2023-02-03} {0:"byte_string"} {0:"aa", 1:"bb"}
{1:"o", 0:"k"} {1: [...]
+2 [1, NULL, 0] [1, NULL, 2, 3] [4, 5, 6] [NULL, NULL] [NULL,
12, 13] [14, 15] [NULL, NULL] [18, 19, 20] [21.11, 22, 22]
[23.333] [NULL, 0.00] [2020-02-02, NULL, 2023-10-24] ["13str",
"arr_string"] ["14varchar", "arr_varcha"] ["a", "b"] [1, 0, 1]
["aaaa", "bbb", NULL, NULL] [2023-10-24 18:19:51.670, NULL, 2023-10-24
18:19:51.670] {1:NULL, 0:NULL} {1:NULL, 0:NULL} {1:4,
2:NULL} {1:6, 0:9} {1:7} {0:19, 1:20} {1:21} {1:22} {0:21.11,
1:22.22} {1:23.333} {0:25.55, 1:0.00} {1:NULL, 4:2023-02-03
[...]
+
+-- !c28 --
+0
+0
+1
+
+-- !c29 --
+1
+1
+1
+
+-- !c30 --
+4
+4
+4
+
+-- !c31 --
+7
+\N
+\N
+
+-- !c32 --
+10
+12
+12
+
+-- !c33 --
+14
+14
+15
+
+-- !c34 --
+16
+\N
+\N
+
+-- !c35 --
+18
+18
+18
+
+-- !c36 --
+21.11
+21.11
+21.11
+
+-- !c37 --
+23.333
+23.333
+\N
+
+-- !c38 --
+24.44
+0.00
+\N
+
+-- !c39 --
+2023-10-24
+2023-10-24
+2023-10-24
+
+-- !c40 --
+2
+2
+2
+
+-- !c41 --
+2
+2
+2
+
+-- !c42 --
+2
+2
+2
+
+-- !c43 --
+true
+true
+\N
+
+-- !c44 --
+2
+4
+3
+
+-- !c45 --
+2023-10-24T16:10:40.529
+2023-10-24T18:19:51.670
+\N
+
+-- !c46 --
+1
+\N
+\N
+
+-- !c47 --
+2
+\N
+\N
+
+-- !c48 --
+5
+\N
+\N
+
+-- !c49 --
+9
+9
+\N
+
+-- !c50 --
+7
+7
+7
+
+-- !c51 --
+19
+19
+\N
+
+-- !c52 --
+\N
+\N
+\N
+
+-- !c53 --
+\N
+\N
+\N
+
+-- !c54 --
+21.11
+21.11
+21.11
+
+-- !c55 --
+\N
+\N
+\N
+
+-- !c56 --
+24.44
+25.55
+24.44
+
+-- !c57 --
+\N
+\N
+\N
+
+-- !c58 --
+byte_string
+byte_string
+byte_string
+
+-- !c59 --
+aa
+aa
+aa
+
+-- !c60 --
+k
+k
+k
+
+-- !c61 --
+true
+true
+true
+
+-- !c62 --
+bbb
+bbb
+bbb
+
+-- !c63 --
+2023-10-24T16:10:40.532
+2023-10-24T18:19:51.672
+2023-10-24T18:20:16.600
+
+-- !c64 --
+1
+\N
+\N
+
+-- !c65 --
+3
+3
+3
+
+-- !c66 --
+4
+4
+4
+
+-- !c67 --
+6
+\N
+\N
+
+-- !c68 --
+7
+\N
+\N
+
+-- !c69 --
+19
+19
+19
+
+-- !c70 --
+21
+21
+21
+
+-- !c71 --
+22
+22
+\N
+
+-- !c72 --
+21.11
+21.11
+21.11
+
+-- !c73 --
+23.333
+23.333
+23.333
+
+-- !c74 --
+24.44
+25.55
+24.44
+
+-- !c75 --
+2021-02-03
+2021-02-03
+2021-02-03
+
+-- !c76 --
+byte_string
+byte_string
+byte_string
+
+-- !c77 --
+aa
+aa
+aa
+
+-- !c78 --
+o
+o
+\N
+
+-- !c79 --
+true
+\N
+\N
+
+-- !c80 --
+bbb
+bbb
+bbb
+
+-- !c80 --
+2023-10-24T16:10:40.533
+2023-10-24T18:19:51.673
+\N
+
+-- !c80 --
+1
+1
+1
+
+-- !c80 --
+3
+3
+3
+
+-- !c80 --
+4
+4
+4
+
+-- !c80 --
+6
+6
+6
+
+-- !c80 --
+7
+7
+7
+
+-- !c80 --
+19
+19
+19
+
+-- !c80 --
+21
+\N
+\N
+
+-- !c80 --
+22
+22
+22
+
+-- !c90 --
+21.11
+21.11
+21.11
+
+-- !c91 --
+23.333
+23.333
+23.333
+
+-- !c92 --
+24.44
+25.55
+24.44
+
+-- !c93 --
+2021-02-03
+2021-02-03
+2021-02-03
+
+-- !c94 --
+byte_string
+byte_string
+byte_string
+
+-- !c95 --
+aa
+aa
+aa
+
+-- !c96 --
+o
+o
+o
+
+-- !c97 --
+true
+true
+true
+
+-- !c98 --
+bbb
+bbb
+bbb
+
+-- !c99 --
+2023-10-24T16:10:40.533
+2023-10-24T18:19:51.673
+\N
+
+-- !c100 --
+1 [[0, 1, 1], [0, 1, 1], [1, 1, 1]] [["3", "7", "a"], ["0", "e",
"2"], ["b", "4", "5"]] [["77", "83", "1c"], ["af", "f0", "0b"], ["be",
"d2", "7b"]]
[["4a3aee3c345936f7a6eeb5307fc79d5f9ee3ae3a5e58edcfd4d40ec3d27353bf7b15a2eb6b6011e9ddf2c2cd1d6d50d6cc20",
"00cb283c1a2bb2f3f244f89eafb79effd8aea55d5e68ce9d707250a3b47b65c0ea2b591b7145a56c801b9e6bda853e2f0581",
"ae8fbe21e20f32a486757b5254faea09906bf451e096cc7d33b5a6fb56995c1601e7469a674f5e8475a2b86d2a69e02f9438"],
["6827f8e65869d476a9e2fdfa03 [...]
+2 [[0, 0, 0], [1, 0, 0], [0, 1, 1]] [["a", "5", "f"], ["7", "2",
"3"], ["1", "b", "f"]] [["11", "a8", "e3"], ["f4", "ee", "c3"], ["0f",
"c6", "05"]]
[["b723869515b24e9fbb54503f8a7584f083479998766213b784a9c530cbc0376bd5035054c657437251b85fa3dd41a0483776",
"ee780d81d3e9faa36aad06522a09cf9b18e00614c991d2c079243dcc7190f3dd6559e75e2c1992270272d9a9c01e950c7bd4",
"6e825a52cdae65786801caae53182956c80f88a48fa258a90d2c93302023c78b83f1dce758615a74731c9eef993a8c1dc4ad"],
["2ae97869372970f7ad7d0007a1 [...]
+3 [[1, 0, 0], [1, 0, 1], [1, 0, 1]] [["c", "6", "4"], ["1", "e",
"7"], ["7", "1", "d"]] [["50", "a9", "b0"], ["02", "bf", "3a"], ["0a",
"1d", "9a"]]
[["2df05ba6ce8661f0fb9bc88386a1ba67188e3e99a4142a0703e1cd8bdf041fbc20131e50bea2a9891498c638ebac842d3d46",
"f6fecf3a4263ed8d4c5b63e4b3d9f4084b83835d9fba2046bf48d0a8068f2044c48271a1e9726741a09badea72c37cf18de3",
"d23af3266db4eda12673f5c451d36343ba1cea00fbfeeff2165de40e834778eb96a1199cb523dd394b4f08824f6af2a7d894"],
["dc00e9b27b9540e170caf93805 [...]
+
diff --git
a/regression-test/suites/external_table_p0/paimon/test_paimon_catalog.groovy
b/regression-test/suites/external_table_p0/paimon/test_paimon_catalog.groovy
index 8c4e257f1f1..03603fda0de 100644
--- a/regression-test/suites/external_table_p0/paimon/test_paimon_catalog.groovy
+++ b/regression-test/suites/external_table_p0/paimon/test_paimon_catalog.groovy
@@ -81,6 +81,85 @@ suite("test_paimon_catalog",
"p0,external,doris,external_docker,external_docker_
def c26 = """select array_max(c2) from complex_tab"""
def c25 = """select c3['a_test'], c3['b_test'], c3['bbb'],
c3['ccc'] from complex_tab"""
+ def c27 = """select * from complex_all order by c1;"""
+ def c28 = """select array_min(c2) from complex_all"""
+ def c29 = """select array_min(c3) from complex_all"""
+ def c30= """select array_min(c4) from complex_all"""
+ def c31= """select array_min(c5) from complex_all"""
+ def c32= """select array_min(c6) from complex_all"""
+ def c33= """select array_min(c7) from complex_all"""
+ def c34= """select array_min(c8) from complex_all"""
+ def c35= """select array_min(c9) from complex_all"""
+ def c36= """select array_min(c10) from complex_all"""
+ def c37= """select array_max(c11) from complex_all"""
+ def c38= """select array_max(c12) from complex_all"""
+ def c39= """select array_max(c13) from complex_all"""
+ def c40= """select array_size(c14) from complex_all"""
+ def c41= """select array_size(c15) from complex_all"""
+ def c42= """select array_size(c16) from complex_all"""
+ def c43= """select array_max(c17) from complex_all"""
+ def c44= """select array_size(c18) from complex_all"""
+ def c45= """select array_max(c19) from complex_all"""
+
+ def c46= """select c20[0] from complex_all"""
+ def c47= """select c21[0] from complex_all"""
+ def c48= """select c22[0] from complex_all"""
+ def c49= """select c23[0] from complex_all"""
+ def c50= """select c24[1] from complex_all"""
+ def c51= """select c25[0] from complex_all"""
+ def c52= """select c26[0] from complex_all"""
+ def c53= """select c27[0] from complex_all"""
+ def c54= """select c28[0] from complex_all"""
+ def c55= """select c29[0] from complex_all"""
+ def c56= """select c30[0] from complex_all"""
+ def c57= """select c31[0] from complex_all"""
+ def c58= """select c32[0] from complex_all"""
+ def c59= """select c33[0] from complex_all"""
+ def c60= """select c34[0] from complex_all"""
+ def c61= """select c35[0] from complex_all"""
+ def c62= """select c36[0] from complex_all"""
+ def c63= """select c37[0] from complex_all"""
+
+ def c64= """select c38[2] from complex_all"""
+ def c65= """select c39[4] from complex_all"""
+ def c66= """select c40[5] from complex_all;"""
+ def c67= """select c41[7] from complex_all;"""
+ def c68= """select c42[9] from complex_all"""
+ def c69= """select c43[10] from complex_all"""
+ def c70= """select c44[12] from complex_all"""
+ def c71= """select c45[13] from complex_all"""
+ def c72= """select c46[14] from complex_all;"""
+ def c73= """select c47[16] from complex_all;"""
+ def c74= """select c48[17] from complex_all;"""
+ def c75= """select c49[19] from complex_all;"""
+ def c76= """select c50[21] from complex_all;"""
+ def c77= """select c51[22] from complex_all;"""
+ def c78= """select c52[25] from complex_all;"""
+ def c79= """select c53[27] from complex_all;"""
+ def c80= """select c54[29] from complex_all;"""
+ def c81= """select c55[30] from complex_all;"""
+
+ def c82= """select c56[2] from complex_all"""
+ def c83= """select c57[4] from complex_all"""
+ def c84= """select c58[5] from complex_all;"""
+ def c85= """select c59[7] from complex_all;"""
+ def c86= """select c60[9] from complex_all"""
+ def c87= """select c61[10] from complex_all"""
+ def c88= """select c62[12] from complex_all"""
+ def c89= """select c63[13] from complex_all"""
+ def c90= """select c64[14] from complex_all;"""
+ def c91= """select c65[16] from complex_all;"""
+ def c92= """select c66[17] from complex_all;"""
+ def c93= """select c67[19] from complex_all;"""
+ def c94= """select c68[21] from complex_all;"""
+ def c95= """select c69[22] from complex_all;"""
+ def c96= """select c70[25] from complex_all;"""
+ def c97= """select c71[27] from complex_all;"""
+ def c98= """select c72[29] from complex_all;"""
+ def c99= """select c73[30] from complex_all;"""
+
+ def c100= """select * from array_nested order by c1;"""
+
String hdfs_port = context.config.otherConfigs.get("hdfs_port")
String catalog_name = "paimon1"
String externalEnvIp =
context.config.otherConfigs.get("externalEnvIp")
@@ -115,5 +194,83 @@ suite("test_paimon_catalog",
"p0,external,doris,external_docker,external_docker_
qt_c20 c20
qt_c21 c21
qt_c22 c22
+ qt_c23 c23
+ qt_c24 c24
+ qt_c25 c25
+ qt_c26 c26
+ qt_c27 c27
+ qt_c28 c28
+ qt_c29 c29
+ qt_c30 c30
+ qt_c31 c31
+ qt_c32 c32
+ qt_c33 c33
+ qt_c34 c34
+ qt_c35 c35
+ qt_c36 c36
+ qt_c37 c37
+ qt_c38 c38
+ qt_c39 c39
+ qt_c40 c40
+ qt_c41 c41
+ qt_c42 c42
+ qt_c43 c43
+ qt_c44 c44
+ qt_c45 c45
+ qt_c46 c46
+ qt_c47 c47
+ qt_c48 c48
+ qt_c49 c49
+ qt_c50 c50
+ qt_c51 c51
+ qt_c52 c52
+ qt_c53 c53
+ qt_c54 c54
+ qt_c55 c55
+ qt_c56 c56
+ qt_c57 c57
+ qt_c58 c58
+ qt_c59 c59
+ qt_c60 c60
+ qt_c61 c61
+ qt_c62 c62
+ qt_c63 c63
+ qt_c64 c64
+ qt_c65 c65
+ qt_c66 c66
+ qt_c67 c67
+ qt_c68 c68
+ qt_c69 c69
+ qt_c70 c70
+ qt_c71 c71
+ qt_c72 c72
+ qt_c73 c73
+ qt_c74 c74
+ qt_c75 c75
+ qt_c76 c76
+ qt_c77 c77
+ qt_c78 c78
+ qt_c79 c79
+ qt_c80 c80
+ qt_c80 c81
+ qt_c80 c82
+ qt_c80 c83
+ qt_c80 c84
+ qt_c80 c85
+ qt_c80 c86
+ qt_c80 c87
+ qt_c80 c88
+ qt_c80 c89
+ qt_c90 c90
+ qt_c91 c91
+ qt_c92 c92
+ qt_c93 c93
+ qt_c94 c94
+ qt_c95 c95
+ qt_c96 c96
+ qt_c97 c97
+ qt_c98 c98
+ qt_c99 c99
+ qt_c100 c100
}
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
