xinyiZzz opened a new pull request, #48028:
URL: https://github.com/apache/doris/pull/48028
### What problem does this PR solve?
```
[==========] Running 3 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 3 tests from DataTypeSerDeArrowTest
[ RUN ] DataTypeSerDeArrowTest.DataTypeScalaSerDeTest
value:
block data:
+---------------+-------------------+---------------+------------------+---------------+---------------+-----------------+------------------+---------------+---------------+---------------+
|k1(Int32) |k7(Nullable(Int32))|k2(String) |k3(Decimal(27,
9))|k11(DateTime) |k4(UInt8) |k5(Decimal(9, 2))|k6(Decimal(18,
6))|k12(DateTimeV2)|k8(IPv4) |k9(IPv6) |
+---------------+-------------------+---------------+------------------+---------------+---------------+-----------------+------------------+---------------+---------------+---------------+
| 0| NULL| 0|
0.000000000|2021-05-01 0...| 0| 0.00|
0.000000| | 0.0.0.0| ::|
| 1| 1| 1|
1.100000000|2021-05-01 0...| 1|
-1234567.56|123456789012.12...| | 0.0.0.1| ::1|
| 2| NULL| 2|
2.200000000|2021-05-01 0...| 0|
1234567.56|-123456789012.1...| | 0.0.0.2| ::2|
| 3| 3| 3|
3.300000000|2021-05-01 0...| 1|
-1234567.56|123456789012.12...| | 0.0.0.3| ::3|
| 4| NULL| 4|
4.400000000|2021-05-01 0...| 0|
1234567.56|-123456789012.1...| | 0.0.0.4| ::4|
| 5| 5| 5|
5.500000000|2021-05-01 0...| 1|
-1234567.56|123456789012.12...| | 0.0.0.5| ::5|
| 6| NULL| 6|
6.600000000|2021-05-01 0...| 0|
1234567.56|-123456789012.1...| | 0.0.0.6| ::6|
+---------------+-------------------+---------------+------------------+---------------+---------------+-----------------+------------------+---------------+---------------+---------------+
_arrow_schema: k1: int32 not null
k7: int32
k2: string not null
k3: decimal128(27, 9) not null
k11: string not null
k4: bool not null
k5: decimal128(9, 2) not null
k6: decimal128(18, 6) not null
k12: timestamp[us, tz=UTC] not null
k8: int32 not null
k9: string not null
result: k1: [
0,
1,
2,
3,
4,
5,
6
]
k7: [
null,
1,
null,
3,
null,
5,
null
]
k2: [
"0",
"1",
"2",
"3",
"4",
"5",
"6"
]
k3: [
0.E-9,
1.100000000,
2.200000000,
3.300000000,
4.400000000,
5.500000000,
6.600000000
]
k11: [
"2021-05-01 08:09:10",
"2021-05-01 08:09:10",
"2021-05-01 08:09:10",
"2021-05-01 08:09:10",
"2021-05-01 08:09:10",
"2021-05-01 08:09:10",
"2021-05-01 08:09:10"
]
k4: [
false,
true,
false,
true,
false,
true,
false
]
k5: [
0.00,
-1234567.56,
1234567.56,
-1234567.56,
1234567.56,
-1234567.56,
1234567.56
]
k6: [
0.000000,
123456789012.123456,
-123456789012.123456,
123456789012.123456,
-123456789012.123456,
123456789012.123456,
-123456789012.123456
]
k12: [
1969-12-31 06:43:50.016000Z,
1969-12-31 06:43:50.016000Z,
1969-12-31 06:43:50.016000Z,
1969-12-31 06:43:50.016000Z,
1969-12-31 06:43:50.016000Z,
1969-12-31 06:43:50.016000Z,
1969-12-31 06:43:50.016000Z
]
k8: [
0,
1,
2,
3,
4,
5,
6
]
k9: [
"::",
"::1",
"::2",
"::3",
"::4",
"::5",
"::6"
]
+---------------+-------------------+---------------+------------------+---------------+---------------+-----------------+------------------+---------------+---------------+
|k1(Int32) |k7(Nullable(Int32))|k2(String) |k3(Decimal(27,
9))|k11(DateTime) |k4(UInt8) |k5(Decimal(9, 2))|k6(Decimal(18,
6))|k8(IPv4) |k9(IPv6) |
+---------------+-------------------+---------------+------------------+---------------+---------------+-----------------+------------------+---------------+---------------+
| 0| NULL| 0|
0.000000000|2021-05-01 0...| 0| 0.00|
0.000000| 0.0.0.0| ::|
| 1| 1| 1|
1.100000000|2021-05-01 0...| 1|
-1234567.56|123456789012.12...| 0.0.0.1| ::1|
| 2| NULL| 2|
2.200000000|2021-05-01 0...| 0|
1234567.56|-123456789012.1...| 0.0.0.2| ::2|
| 3| 3| 3|
3.300000000|2021-05-01 0...| 1|
-1234567.56|123456789012.12...| 0.0.0.3| ::3|
| 4| NULL| 4|
4.400000000|2021-05-01 0...| 0|
1234567.56|-123456789012.1...| 0.0.0.4| ::4|
| 5| 5| 5|
5.500000000|2021-05-01 0...| 1|
-1234567.56|123456789012.12...| 0.0.0.5| ::5|
| 6| NULL| 6|
6.600000000|2021-05-01 0...| 0|
1234567.56|-123456789012.1...| 0.0.0.6| ::6|
+---------------+-------------------+---------------+------------------+---------------+---------------+-----------------+------------------+---------------+---------------+
+---------------+-------------------+---------------+------------------+---------------+---------------+-----------------+------------------+---------------+---------------+
|k1(Int32) |k7(Nullable(Int32))|k2(String) |k3(Decimal(27,
9))|k11(DateTime) |k4(UInt8) |k5(Decimal(9, 2))|k6(Decimal(18,
6))|k8(IPv4) |k9(IPv6) |
+---------------+-------------------+---------------+------------------+---------------+---------------+-----------------+------------------+---------------+---------------+
| 0| NULL| 0|
0.000000000|2021-05-01 0...| 0| 0.00|
0.000000| 0.0.0.0| ::|
| 1| 1| 1|
1.100000000|2021-05-01 0...| 1|
-1234567.56|123456789012.12...| 0.0.0.1| ::1|
| 2| NULL| 2|
2.200000000|2021-05-01 0...| 0|
1234567.56|-123456789012.1...| 0.0.0.2| ::2|
| 3| 3| 3|
3.300000000|2021-05-01 0...| 1|
-1234567.56|123456789012.12...| 0.0.0.3| ::3|
| 4| NULL| 4|
4.400000000|2021-05-01 0...| 0|
1234567.56|-123456789012.1...| 0.0.0.4| ::4|
| 5| 5| 5|
5.500000000|2021-05-01 0...| 1|
-1234567.56|123456789012.12...| 0.0.0.5| ::5|
| 6| NULL| 6|
6.600000000|2021-05-01 0...| 0|
1234567.56|-123456789012.1...| 0.0.0.6| ::6|
+---------------+-------------------+---------------+------------------+---------------+---------------+-----------------+------------------+---------------+---------------+
[ OK ] DataTypeSerDeArrowTest.DataTypeScalaSerDeTest (4 ms)
[ RUN ] DataTypeSerDeArrowTest.DataTypeCollectionSerDeTest
block data:
+--------------------------+------------------------------------------+--------------------------------------------------------------------+
|a(Array(Nullable(String)))|m(Map(Nullable(String),
Nullable(String)))|s(Struct(1:Nullable(String), 2:Nullable(Int128),
3:Nullable(UInt8)))|
+--------------------------+------------------------------------------+--------------------------------------------------------------------+
|['sss', 'NULL', 'clever...|{"null":"ss", "doris":null, "clever amo...|
{amory cute, 37, 1}|
|['hello amory', 'NULL',...|{"hello amory":"s", "NULL":"0", "cute a...|
{null, 26, 0}|
+--------------------------+------------------------------------------+--------------------------------------------------------------------+
_arrow_schema: a: list<item: string> not null
m: map<string, string> not null
s: struct<1: string, 2: string, 3: bool> not null
result: a: [
[
"sss",
null,
"clever amory"
],
[
"hello amory",
null,
"cute amory",
"sf"
]
]
m: [
keys:
[
"null",
"doris",
"clever amory"
]
values:
[
"ss",
null,
"NULL"
],
keys:
[
"hello amory",
"NULL",
"cute amory",
"doris"
]
values:
[
"s",
"0",
"sf",
null
]
]
s: -- is_valid: all not null
-- child 0 type: string
[
"amory cute",
"null"
]
-- child 1 type: string
[
"37",
"26"
]
-- child 2 type: bool
[
true,
false
]
+--------------------------+------------------------------------------+--------------------------------------------------------------------+
|a(Array(Nullable(String)))|m(Map(Nullable(String),
Nullable(String)))|s(Struct(1:Nullable(String), 2:Nullable(Int128),
3:Nullable(UInt8)))|
+--------------------------+------------------------------------------+--------------------------------------------------------------------+
|['sss', 'NULL', 'clever...|{"null":"ss", "doris":null, "clever amo...|
{amory cute, 37, 1}|
|['hello amory', 'NULL',...|{"hello amory":"s", "NULL":"0", "cute a...|
{null, 26, 0}|
+--------------------------+------------------------------------------+--------------------------------------------------------------------+
+--------------------------+------------------------------------------+--------------------------------------------------------------------+
|a(Array(Nullable(String)))|m(Map(Nullable(String),
Nullable(String)))|s(Struct(1:Nullable(String), 2:Nullable(Int128),
3:Nullable(UInt8)))|
+--------------------------+------------------------------------------+--------------------------------------------------------------------+
|['sss', 'NULL', 'clever...|{"null":"ss", "doris":null, "clever amo...|
{amory cute, 37, 1}|
|['hello amory', 'NULL',...|{"hello amory":"s", "NULL":"0", "cute a...|
{null, 26, 0}|
+--------------------------+------------------------------------------+--------------------------------------------------------------------+
[ OK ] DataTypeSerDeArrowTest.DataTypeCollectionSerDeTest (3 ms)
[ RUN ] DataTypeSerDeArrowTest.DataTypeMapNullKeySerDeTest
block structure: map_null_key Map(Nullable(String), Nullable(Int32))
Map(Nullable(String), Nullable(Int32))(size = 3, Nullable(String)(size = 7,
String(size = 7), UInt8(size = 7)), Nullable(Int32)(size = 7, Int32(size = 7),
UInt8(size = 7)), UInt64(size = 3))(use_count=1)
_arrow_schema: map_null_key: map<string, int32> not null
result: map_null_key: [
keys:
[
"doris",
"clever amory"
]
values:
[
null,
30
],
keys:
[
"hello amory",
"NULL",
"cute amory",
"doris"
]
values:
[
26,
null,
6,
7
],
keys:
[
"test"
]
values:
[
11
]
]
+----------------------------------------------------+
|map_null_key(Map(Nullable(String), Nullable(Int32)))|
+----------------------------------------------------+
| {"doris":null, "clever amory":30}|
|{"hello amory":26, "NULL":null, "cute amory":6, "...|
| {"test":11}|
+----------------------------------------------------+
+----------------------------------------------------+
|map_null_key(Map(Nullable(String), Nullable(Int32)))|
+----------------------------------------------------+
| {"doris":null, "clever amory":30}|
|{"hello amory":26, "NULL":null, "cute amory":6, "...|
| {"test":11}|
+----------------------------------------------------+
[ OK ] DataTypeSerDeArrowTest.DataTypeMapNullKeySerDeTest (1 ms)
[----------] 3 tests from DataTypeSerDeArrowTest (9 ms total)
[----------] Global test environment tear-down
[==========] 3 tests from 1 test suite ran. (9 ms total)
[ PASSED ] 3 tests.
```
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should
merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
