yinzhijian opened a new pull request, #9433:
URL: https://github.com/apache/incubator-doris/pull/9433
# Proposed changes
Issue Number: close #xxx
## Problem Summary:
## Optimization Todo list：
1. FE generates the corresponding src slot desc and Expr through the parquet
schema
2. BE supports direct conversion of arrow type into dest primitive type of
similar type. For example, arrow type is INT32, and dest type is TYPE_BIGINT
(int64), INT32=>TYPE_BIGINT. Instead of the current way: INT32=> TYPE_INT =>
TYPE_BIGINT
## Performance Testing:
load parquet file in vec version almost 1x faster than rowset version.
rows num:300k
test table schema:
CREATE TABLE `parquet` (
`id` int(11) NOT NULL COMMENT "",
`email` varchar(26) NOT NULL COMMENT "",
`c_date32` DATE NOT NULL COMMENT "",
`c_date64` DATETIME NOT NULL COMMENT "",
`c_timestamp` DATETIME NOT NULL COMMENT "",
`c_decimal128` DECIMAL(27, 9) NULL COMMENT "",
`c_bool` BOOLEAN NULL COMMENT "",
`c_float` FLOAT NULL COMMENT "",
`c_double` DOUBLE NULL COMMENT "",
`c_fixed_size_binary` CHAR(20) NULL COMMENT "",
`c_binary` VARCHAR(32) NULL COMMENT "",
`c_uint64` BIGINT NULL COMMENT ""
)
DISTRIBUTED BY HASH(`id`) BUCKETS 1
PROPERTIES (
"replication_num" = "1"
);
## Checklist(Required)
1. Does it affect the original behavior: (No)
2. Has unit tests been added: (No)
4. Has document been added or modified: (No Need)
5. Does it need to update dependencies: (No)
6. Are there any changes that cannot be rolled back: (No)
## Further comments
If this is a relatively large or complex change, kick off the discussion at
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you
chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
