Re: [PR] [typo](docs) Fix doc links to 404 (#24417) [doris]
TimCoder commented on PR #24418: URL: https://github.com/apache/doris/pull/24418#issuecomment-1751628093 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [branch-2.0-pick](merge-on-write) picks #22782, #24011 and #24877 [doris]
bobhan1 opened a new pull request, #25071: URL: https://github.com/apache/doris/pull/25071 ## Proposed changes picks #22782, #24011 and #24877 ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [branch-2.0-pick](merge-on-write) picks #22782, #24011 and #24877 [doris]
bobhan1 commented on PR #25071: URL: https://github.com/apache/doris/pull/25071#issuecomment-1751628891 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [doc](fix)fix misspell [doris]
LemonLiTree opened a new pull request, #25072: URL: https://github.com/apache/doris/pull/25072 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [doc](fix)fix misspell [doris]
LemonLiTree commented on PR #25072: URL: https://github.com/apache/doris/pull/25072#issuecomment-1751629458 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Nerids) fix error when the view has lambda functions [doris]
doris-robot commented on PR #25067: URL: https://github.com/apache/doris/pull/25067#issuecomment-1751629546 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.53 seconds stream load tsv: 563 seconds loaded 74807831229 Bytes, about 126 MB/s stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 29.0 seconds inserted 1000 Rows, about 344K ops/s storage size: 17162162686 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](nereids)align node id in explain with nereids node id [doris]
doris-robot commented on PR #25068: URL: https://github.com/apache/doris/pull/25068#issuecomment-1751629610 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.85 seconds stream load tsv: 568 seconds loaded 74807831229 Bytes, about 125 MB/s stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 28.8 seconds inserted 1000 Rows, about 347K ops/s storage size: 17162470154 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [doc](fix)fix misspell [doris]
github-actions[bot] commented on PR #25072: URL: https://github.com/apache/doris/pull/25072#issuecomment-1751629820 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [doc](fix)fix misspell [doris]
github-actions[bot] commented on PR #25072: URL: https://github.com/apache/doris/pull/25072#issuecomment-1751629830 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](k8s) specify namespace using downward api [doris]
murong00 commented on PR #24961: URL: https://github.com/apache/doris/pull/24961#issuecomment-1751630124 > Hi, thanks for your contribution to doris community. Directly use statefulset to deploy doris on k8s is not a recommendation. Now selectdb provide a more common mode to deploy doris on k8s, the operator mode. directly mode will not supported, we recommend you see into it. the address is that https://github.com/selectdb/doris-operator/tree/master Although selectdb operator is recommended, some users may expect using doris in an native mode for some reason. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Opt](performance) Optimize timeround with minute / second [doris]
Mryange opened a new pull request, #25073: URL: https://github.com/apache/doris/pull/25073 ## Proposed changes https://github.com/apache/doris/pull/25026 ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](Nereids): new Date/Datetime parser [doris]
jackwener commented on PR #24523: URL: https://github.com/apache/doris/pull/24523#issuecomment-1751630230 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Opt](performance) Optimize timeround with minute / second [doris]
Mryange commented on PR #25073: URL: https://github.com/apache/doris/pull/25073#issuecomment-1751630440 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [branch-2.0-pick](merge-on-write) picks #22782, #24011 and #24877 [doris]
github-actions[bot] commented on PR #25071: URL: https://github.com/apache/doris/pull/25071#issuecomment-1751630642 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] (Fix)(RoutineLoad)Query the transaction status NPE when the task has not yet started scheduling [doris]
CalvinKirs opened a new pull request, #25074: URL: https://github.com/apache/doris/pull/25074 ### error log When the task has not yet started scheduling, there is no transaction status at this time. Therefore the query will cause an NPE exception ``` 2023-10-07 13:52:16,883 WARN (mysql-nio-pool-3|1310) [StmtExecutor.executeByLegacy():807] execute Exception. stmt[1368, fc075cfd62aa4459-a8014ca6b5f574fa] java.lang.NullPointerException: null at org.apache.doris.load.routineload.RoutineLoadJob.lambda$getTasksShowInfo$6(RoutineLoadJob.java:1452) ~[classes/:?] at java.util.ArrayList.forEach(ArrayList.java:1259) ~[?:1.8.0_362] at org.apache.doris.load.routineload.RoutineLoadJob.getTasksShowInfo(RoutineLoadJob.java:1450) ~[classes/:?] at org.apache.doris.qe.ShowExecutor.handleShowRoutineLoadTask(ShowExecutor.java:1605) ~[classes/:?] at org.apache.doris.qe.ShowExecutor.execute(ShowExecutor.java:316) ~[classes/:?] at org.apache.doris.qe.StmtExecutor.handleShow(StmtExecutor.java:2208) ~[classes/:?] at org.apache.doris.qe.StmtExecutor.executeByLegacy(StmtExecutor.java:775) ~[classes/:?] at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:468) ~[classes/:?] at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:438) ~[classes/:?] at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:353) ~[classes/:?] at org.apache.doris.qe.ConnectProcessor.dispatch(ConnectProcessor.java:501) ~[classes/:?] at org.apache.doris.qe.ConnectProcessor.processOnce(ConnectProcessor.java:752) ~[classes/:?] at org.apache.doris.mysql.ReadListener.lambda$handleEvent$0(ReadListener.java:52) ~[classes/:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_362] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_362] ``` ### Changes When the task has not yet started scheduling, that is, when txn_id is -1, the transaction status is set to null ### Test https://github.com/apache/doris/assets/16631152/55d6d97f-c8df-4b8b-82ea-a9661387aefe";> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](Variant) support variant load [doris]
doris-robot commented on PR #24554: URL: https://github.com/apache/doris/pull/24554#issuecomment-1751631794 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.49 seconds stream load tsv: 578 seconds loaded 74807831229 Bytes, about 123 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 29.0 seconds inserted 1000 Rows, about 344K ops/s storage size: 17162544698 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](RoutineLoad)Query the transaction status NPE when the task has not yet started scheduling [doris]
CalvinKirs commented on PR #25074: URL: https://github.com/apache/doris/pull/25074#issuecomment-1751631940 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Opt](performance) Optimize timeround with minute / second [doris]
github-actions[bot] commented on PR #25073: URL: https://github.com/apache/doris/pull/25073#issuecomment-1751632040 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [monitor][doc]modify incorrect status for doris_be_engine_requests_total [doris]
xingyingone opened a new pull request, #25075: URL: https://github.com/apache/doris/pull/25075 ## Proposed changes [monitor][doc]modify incorrect status for doris_be_engine_requests_total Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [chore](unused) Remove unused variable CPU_HARD_LIMIT in task_group.cc [doris]
JackDrogon opened a new pull request, #25076: URL: https://github.com/apache/doris/pull/25076 ## Proposed changes Remove unused variable CPU_HARD_LIMIT in task_group.cc ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](unused) Remove unused variable CPU_HARD_LIMIT in task_group.cc [doris]
github-actions[bot] commented on PR #25076: URL: https://github.com/apache/doris/pull/25076#issuecomment-1751633183 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](unused) Remove unused variable CPU_HARD_LIMIT in task_group.cc [doris]
github-actions[bot] commented on PR #25076: URL: https://github.com/apache/doris/pull/25076#issuecomment-1751633192 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](streamload) fix http_stream retry mechanism [doris]
yiguolei commented on code in PR #24978: URL: https://github.com/apache/doris/pull/24978#discussion_r1349478506 ## be/src/vec/exec/format/json/new_json_reader.cpp: ## @@ -382,7 +382,8 @@ Status NewJsonReader::_open_file_reader() { _current_offset = start_offset; if (_params.file_type == TFileType::FILE_STREAM) { -RETURN_IF_ERROR(FileFactory::create_pipe_reader(_range.load_id, &_file_reader, _state)); Review Comment: If it is reading schema, why not set read schema to true??? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](streamload) fix http_stream retry mechanism [doris]
yiguolei commented on code in PR #24978: URL: https://github.com/apache/doris/pull/24978#discussion_r1349478525 ## be/src/vec/exec/format/csv/csv_reader.cpp: ## @@ -824,7 +825,8 @@ Status CsvReader::_prepare_parse(size_t* read_line, bool* is_parse_name) { io::FileReaderOptions reader_options = FileFactory::get_reader_options(_state, _file_description); if (_params.file_type == TFileType::FILE_STREAM) { -RETURN_IF_ERROR(FileFactory::create_pipe_reader(_params.load_id, &_file_reader, _state)); Review Comment: add comment here, to explain why set the parameter to true ## be/src/vec/exec/format/csv/csv_reader.cpp: ## @@ -294,7 +294,8 @@ Status CsvReader::init_reader(bool is_load) { } if (_params.file_type == TFileType::FILE_STREAM) { -RETURN_IF_ERROR(FileFactory::create_pipe_reader(_range.load_id, &_file_reader, _state)); Review Comment: add comment here, to explain why set the parameter to false -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](streamload) fix http_stream retry mechanism [doris]
yiguolei commented on code in PR #24978: URL: https://github.com/apache/doris/pull/24978#discussion_r1349478591 ## be/src/io/file_factory.cpp: ## @@ -141,20 +141,19 @@ Status FileFactory::create_file_reader(const io::FileSystemProperties& system_pr // file scan node/stream load pipe Status FileFactory::create_pipe_reader(const TUniqueId& load_id, io::FileReaderSPtr* file_reader, - RuntimeState* runtime_state) { + RuntimeState* runtime_state, bool need_schema) { auto stream_load_ctx = ExecEnv::GetInstance()->new_load_stream_mgr()->get(load_id); if (!stream_load_ctx) { return Status::InternalError("unknown stream load id: {}", UniqueId(load_id).to_string()); } -if (stream_load_ctx->need_schema == true) { +if (need_schema == true) { Review Comment: if (need_schema) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [bug](schema change)fix schema change cause load failed due to err -215 [doris]
doris-robot commented on PR #23836: URL: https://github.com/apache/doris/pull/23836#issuecomment-1751633370 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.26 seconds stream load tsv: 561 seconds loaded 74807831229 Bytes, about 127 MB/s stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s insert into select: 28.9 seconds inserted 1000 Rows, about 346K ops/s storage size: 17162644398 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](streamload) fix http_stream retry mechanism [doris]
yiguolei commented on code in PR #24978: URL: https://github.com/apache/doris/pull/24978#discussion_r1349478632 ## be/src/http/action/http_stream.cpp: ## @@ -245,7 +245,6 @@ void HttpStreamAction::on_chunk_data(HttpRequest* req) { if (ctx->schema_buffer->pos + remove_bytes < config::stream_tvf_buffer_size) { ctx->schema_buffer->put_bytes(bb->ptr, remove_bytes); } else { -ctx->need_schema = true; ctx->is_read_schema = false; ctx->status = _process_put(req, ctx); Review Comment: if process_put return error status, should handle it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](Outfile) Use data_type_serde to export data to csv file format [doris]
BePPPower commented on PR #24721: URL: https://github.com/apache/doris/pull/24721#issuecomment-1751636181 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](unused) Remove unused variable CPU_HARD_LIMIT in task_group.cc [doris]
JackDrogon commented on PR #25076: URL: https://github.com/apache/doris/pull/25076#issuecomment-1751636283 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](unused) Remove unused variable CPU_HARD_LIMIT in task_group.cc [doris]
github-actions[bot] commented on PR #25076: URL: https://github.com/apache/doris/pull/25076#issuecomment-1751636415 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](unused) Remove unused variable CPU_HARD_LIMIT in task_group.cc [doris]
github-actions[bot] commented on PR #25076: URL: https://github.com/apache/doris/pull/25076#issuecomment-1751636840 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](streamload) fix http_stream retry mechanism [doris]
yiguolei commented on code in PR #24978: URL: https://github.com/apache/doris/pull/24978#discussion_r1349479879 ## be/src/io/file_factory.cpp: ## @@ -141,20 +141,19 @@ Status FileFactory::create_file_reader(const io::FileSystemProperties& system_pr // file scan node/stream load pipe Status FileFactory::create_pipe_reader(const TUniqueId& load_id, io::FileReaderSPtr* file_reader, - RuntimeState* runtime_state) { + RuntimeState* runtime_state, bool need_schema) { auto stream_load_ctx = ExecEnv::GetInstance()->new_load_stream_mgr()->get(load_id); if (!stream_load_ctx) { return Status::InternalError("unknown stream load id: {}", UniqueId(load_id).to_string()); } -if (stream_load_ctx->need_schema == true) { +if (need_schema == true) { Review Comment: add comment -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] 01/01: [feature](autobucekt) Add support autobucket min buckets config Add support autobucket (#24920)
This is an automated email from the ASF dual-hosted git repository. eldenmoon pushed a commit to branch branch-2.0-var in repository https://gitbox.apache.org/repos/asf/doris.git commit 39b83792c78cb421a00ff7f5df1c99cba672294b Author: Jack Drogon AuthorDate: Wed Sep 27 08:13:53 2023 +0800 [feature](autobucekt) Add support autobucket min buckets config Add support autobucket (#24920) Signed-off-by: Jack Drogon --- docs/en/docs/advanced/autobucket.md| 1 + docs/zh-CN/docs/advanced/autobucket.md | 1 + .../main/java/org/apache/doris/common/Config.java | 6 ++ .../org/apache/doris/analysis/CreateTableStmt.java | 2 +- .../doris/clone/DynamicPartitionScheduler.java | 2 +- .../apache/doris/common/util/AutoBucketUtils.java | 5 + .../suites/autobucket/test_autobucket.groovy | 23 ++ 7 files changed, 38 insertions(+), 2 deletions(-) diff --git a/docs/en/docs/advanced/autobucket.md b/docs/en/docs/advanced/autobucket.md index bc05e3e72f6..e9fdfcdf63b 100644 --- a/docs/en/docs/advanced/autobucket.md +++ b/docs/en/docs/advanced/autobucket.md @@ -72,6 +72,7 @@ First, use the value of estimate_partition_size divided by 5 (calculated as a 5- 3. Calculation logic to get the final number of buckets. First calculate an intermediate value x = min(M, N, 128). If x < N and x < the number of BE nodes, the final bucket is y, the number of BE nodes; otherwise, the final bucket is x. +4. x = max(x, autobucket_min_buckets), 这里autobucket_min_buckets是在Config中配置的,默认是1 The pseudo-code representation of the above process is as follows diff --git a/docs/zh-CN/docs/advanced/autobucket.md b/docs/zh-CN/docs/advanced/autobucket.md index 43a1124db77..ccbceaf633d 100644 --- a/docs/zh-CN/docs/advanced/autobucket.md +++ b/docs/zh-CN/docs/advanced/autobucket.md @@ -73,6 +73,7 @@ properties("estimate_partition_size" = "100G") 3. 得到最终的分桶个数计算逻辑: 先计算一个中间值 x = min(M, N, 128), 如果 x < N并且x < BE节点个数,则最终分桶为 y 即 BE 节点个数;否则最终分桶数为 x +4. x = max(x, autobucket_min_buckets), 这里autobucket_min_buckets是在Config中配置的,默认是1 上述过程伪代码表现形式为: diff --git a/fe/fe-common/src/main/java/org/apache/doris/common/Config.java b/fe/fe-common/src/main/java/org/apache/doris/common/Config.java index 5f2173af7ca..3c5bb36094d 100644 --- a/fe/fe-common/src/main/java/org/apache/doris/common/Config.java +++ b/fe/fe-common/src/main/java/org/apache/doris/common/Config.java @@ -2125,4 +2125,10 @@ public class Config extends ConfigBase { + "The larger the value, the more uniform the distribution of the hash algorithm, " + "but it will increase the memory overhead."}) public static int virtual_node_number = 2048; + +@ConfField(description = { +"Auto Buckets中最小的buckets数目", +"min buckets of auto bucket" +}) +public static int autobucket_min_buckets = 1; } diff --git a/fe/fe-core/src/main/java/org/apache/doris/analysis/CreateTableStmt.java b/fe/fe-core/src/main/java/org/apache/doris/analysis/CreateTableStmt.java index b1df8498b8b..fa305793f56 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/analysis/CreateTableStmt.java +++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/CreateTableStmt.java @@ -122,7 +122,7 @@ public class CreateTableStmt extends DdlStmt { } else { long partitionSize = ParseUtil .analyzeDataVolumn(newProperties.get(PropertyAnalyzer.PROPERTIES_ESTIMATE_PARTITION_SIZE)); - distributionDesc.setBuckets(AutoBucketUtils.getBucketsNum(partitionSize)); + distributionDesc.setBuckets(AutoBucketUtils.getBucketsNum(partitionSize, Config.autobucket_min_buckets)); } return newProperties; diff --git a/fe/fe-core/src/main/java/org/apache/doris/clone/DynamicPartitionScheduler.java b/fe/fe-core/src/main/java/org/apache/doris/clone/DynamicPartitionScheduler.java index 53f44070d93..dc03ecf8233 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/clone/DynamicPartitionScheduler.java +++ b/fe/fe-core/src/main/java/org/apache/doris/clone/DynamicPartitionScheduler.java @@ -221,7 +221,7 @@ public class DynamicPartitionScheduler extends MasterDaemon { // plus 5 for uncompressed data long uncompressedPartitionSize = getNextPartitionSize(partitionSizeArray) * 5; -return AutoBucketUtils.getBucketsNum(uncompressedPartitionSize); +return AutoBucketUtils.getBucketsNum(uncompressedPartitionSize, Config.autobucket_min_buckets); } private ArrayList getAddPartitionClause(Database db, OlapTable olapTable, diff --git a/fe/fe-core/src/main/java/org/apache/doris/common/util/AutoBucketUtils.java b/fe/fe-core/src/main/java/org/apache/doris/common/util/AutoBucketUtils.java index ca935ab20e7..294250fd213 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/common/util/AutoBucketUtils.java +++ b/fe/fe-core/src/main/java/org/apache/doris/common/util/AutoBucketUtils
Re: [PR] [typo](doc)Add be's enable_java_support configuration document [doris]
DongLiang-0 commented on PR #25069: URL: https://github.com/apache/doris/pull/25069#issuecomment-1751637711 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](unused) Remove unused variable CPU_HARD_LIMIT in task_group.cc [doris]
github-actions[bot] commented on PR #25076: URL: https://github.com/apache/doris/pull/25076#issuecomment-1751637846 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [prune](partition)support prune partition when is auto partition with function call [doris]
zhangstar333 commented on PR #24747: URL: https://github.com/apache/doris/pull/24747#issuecomment-1751638167 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](Outfile) Use data_type_serde to export data to csv file format [doris]
github-actions[bot] commented on PR #24721: URL: https://github.com/apache/doris/pull/24721#issuecomment-1751638318 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [typo](doc)Add be's enable_java_support configuration document [doris]
github-actions[bot] commented on PR #25069: URL: https://github.com/apache/doris/pull/25069#issuecomment-1751638403 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [typo](doc)Add be's enable_java_support configuration document [doris]
github-actions[bot] commented on PR #25069: URL: https://github.com/apache/doris/pull/25069#issuecomment-1751638411 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](RoutineLoad)Query the transaction status NPE when the task has not yet started scheduling [doris]
github-actions[bot] commented on PR #25074: URL: https://github.com/apache/doris/pull/25074#issuecomment-1751638426 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [branch-2.0-pick](merge-on-write) picks #22782, #24011 and #24877 [doris]
hello-stephen commented on PR #25071: URL: https://github.com/apache/doris/pull/25071#issuecomment-1751638522 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 48.94 seconds stream load tsv: 568 seconds loaded 74807831229 Bytes, about 125 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 29.4 seconds inserted 1000 Rows, about 340K ops/s storage size: 17162736395 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [typo](doc)Add be's enable_java_support configuration document [doris]
DongLiang-0 commented on PR #25069: URL: https://github.com/apache/doris/pull/25069#issuecomment-1751638662 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](Nereids): new Date/Datetime parser [doris]
hello-stephen commented on PR #24523: URL: https://github.com/apache/doris/pull/24523#issuecomment-1751638941 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 47.09 seconds stream load tsv: 578 seconds loaded 74807831229 Bytes, about 123 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s insert into select: 29.8 seconds inserted 1000 Rows, about 335K ops/s storage size: 17162546942 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [optimize](chinese) optimize stopwords [doris-thirdparty]
zzzxl1993 opened a new pull request, #128: URL: https://github.com/apache/doris-thirdparty/pull/128 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Opt](performance) Optimize timeround with minute / second [doris]
doris-robot commented on PR #25073: URL: https://github.com/apache/doris/pull/25073#issuecomment-1751639720 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.48 seconds stream load tsv: 562 seconds loaded 74807831229 Bytes, about 126 MB/s stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 28.7 seconds inserted 1000 Rows, about 348K ops/s storage size: 17162129047 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](mysql result format) use new serde framework to tuple convert [doris]
zhiqiang- commented on PR #25006: URL: https://github.com/apache/doris/pull/25006#issuecomment-1751640018 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improvement](hash) refactor of hash map context [doris]
BiteThet commented on PR #24966: URL: https://github.com/apache/doris/pull/24966#issuecomment-1751640213 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [monitor][doc]modify incorrect status for doris_be_engine_requests_total [doris]
yiguolei commented on PR #25075: URL: https://github.com/apache/doris/pull/25075#issuecomment-1751640252 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [monitor][doc]modify incorrect status for doris_be_engine_requests_total [doris]
github-actions[bot] commented on PR #25075: URL: https://github.com/apache/doris/pull/25075#issuecomment-1751640494 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [monitor][doc]modify incorrect status for doris_be_engine_requests_total [doris]
github-actions[bot] commented on PR #25075: URL: https://github.com/apache/doris/pull/25075#issuecomment-1751640480 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](code-style)add spotless plugin and remove checkstyle [doris]
liugddx commented on PR #25033: URL: https://github.com/apache/doris/pull/25033#issuecomment-1751640545 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](RoutineLoad)Query the transaction status NPE when the task has not yet started scheduling [doris]
doris-robot commented on PR #25074: URL: https://github.com/apache/doris/pull/25074#issuecomment-1751640569 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.63 seconds stream load tsv: 566 seconds loaded 74807831229 Bytes, about 126 MB/s stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s insert into select: 28.8 seconds inserted 1000 Rows, about 347K ops/s storage size: 17162339591 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](pipelinex) fix multi cast sink with out init [doris]
Gabriel39 merged PR #25066: URL: https://github.com/apache/doris/pull/25066 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (3c9ff7af399 -> 335804bb253)
This is an automated email from the ASF dual-hosted git repository. gabriellee pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 3c9ff7af399 [feature](Nereids): push down topN through join (#24720) add 335804bb253 [fix](pipelinex) fix multi cast sink without init (#25066) No new revisions were added by this update. Summary of changes: .../pipeline/exec/multi_cast_data_stream_sink.cpp | 8 be/src/pipeline/exec/multi_cast_data_stream_sink.h | 7 .../pipeline/exec/multi_cast_data_stream_source.h | 4 +- be/src/pipeline/pipeline_x/operator.cpp| 47 +++--- .../pipeline_x/pipeline_x_fragment_context.cpp | 27 - 5 files changed, 54 insertions(+), 39 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] 02/06: [fix](planner) having clause analyze bug #24288
This is an automated email from the ASF dual-hosted git repository. morningman pushed a commit to branch branch-1.2-lts in repository https://gitbox.apache.org/repos/asf/doris.git commit 887dcf17aca3cb25b0aa32349998babcd9f41126 Author: minghong AuthorDate: Thu Sep 14 09:54:09 2023 +0800 [fix](planner) having clause analyze bug #24288 --- .../java/org/apache/doris/analysis/SelectStmt.java | 78 +++--- .../suites/query_p0/having/having.groovy | 49 ++ 2 files changed, 89 insertions(+), 38 deletions(-) diff --git a/fe/fe-core/src/main/java/org/apache/doris/analysis/SelectStmt.java b/fe/fe-core/src/main/java/org/apache/doris/analysis/SelectStmt.java index 16bb1ecf2fd..4b5836074f0 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/analysis/SelectStmt.java +++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/SelectStmt.java @@ -109,7 +109,7 @@ public class SelectStmt extends QueryStmt { // having clause which has been analyzed // For example: select k1, sum(k2) a from t group by k1 having a>1; // this parameter: sum(t.k2) > 1 -private Expr havingClauseAfterAnaylzed; +private Expr havingClauseAfterAnalyzed; // END: Members that need to be reset() // /// @@ -167,8 +167,8 @@ public class SelectStmt extends QueryStmt { whereClause = (other.whereClause != null) ? other.whereClause.clone() : null; groupByClause = (other.groupByClause != null) ? other.groupByClause.clone() : null; havingClause = (other.havingClause != null) ? other.havingClause.clone() : null; -havingClauseAfterAnaylzed = -other.havingClauseAfterAnaylzed != null ? other.havingClauseAfterAnaylzed.clone() : null; +havingClauseAfterAnalyzed = +other.havingClauseAfterAnalyzed != null ? other.havingClauseAfterAnalyzed.clone() : null; colLabels = Lists.newArrayList(other.colLabels); aggInfo = (other.aggInfo != null) ? other.aggInfo.clone() : null; @@ -193,7 +193,7 @@ public class SelectStmt extends QueryStmt { if (havingClause != null) { havingClause.reset(); } -havingClauseAfterAnaylzed = null; +havingClauseAfterAnalyzed = null; havingPred = null; aggInfo = null; analyticInfo = null; @@ -240,7 +240,7 @@ public class SelectStmt extends QueryStmt { } public Expr getHavingClauseAfterAnaylzed() { -return havingClauseAfterAnaylzed; +return havingClauseAfterAnalyzed; } public List getTableRefs() { @@ -639,8 +639,8 @@ public class SelectStmt extends QueryStmt { if (whereClause != null) { whereClause.getIds(result, null); } -if (havingClauseAfterAnaylzed != null) { -havingClauseAfterAnaylzed.getIds(result, null); +if (havingClauseAfterAnalyzed != null) { +havingClauseAfterAnalyzed.getIds(result, null); } return result; } @@ -1055,48 +1055,48 @@ public class SelectStmt extends QueryStmt { // according to case3, column name do not exist, keep alias name inside alias map } } -havingClauseAfterAnaylzed = havingClause.substitute(excludeAliasSMap, analyzer, false); +havingClauseAfterAnalyzed = havingClause.substitute(excludeAliasSMap, analyzer, false); } else { // If user set force using alias, then having clauses prefer using alias rather than column name -havingClauseAfterAnaylzed = havingClause.substitute(aliasSMap, analyzer, false); +havingClauseAfterAnalyzed = havingClause.substitute(aliasSMap, analyzer, false); } } else { // according to mysql // if there is no group by clause, the having clause should use alias -havingClauseAfterAnaylzed = havingClause.substitute(aliasSMap, analyzer, false); +havingClauseAfterAnalyzed = havingClause.substitute(aliasSMap, analyzer, false); } -havingClauseAfterAnaylzed = rewriteQueryExprByMvColumnExpr(havingClauseAfterAnaylzed, analyzer); -havingClauseAfterAnaylzed.checkReturnsBool("HAVING clause", true); +havingClauseAfterAnalyzed = rewriteQueryExprByMvColumnExpr(havingClauseAfterAnalyzed, analyzer); +havingClauseAfterAnalyzed.checkReturnsBool("HAVING clause", true); if (groupingInfo != null) { - groupingInfo.substituteGroupingFn(Arrays.asList(havingClauseAfterAnaylzed), analyzer); + groupingInfo.substituteGroupingFn(Arrays.asList(havingClauseAfterAnalyzed), analyzer); } // can't contain analytic exprs -Expr analyticExpr = havingClauseAfterAnaylzed.findFirstOf(Analy
[doris] 01/06: [fix](planner)cast string to float like type should return NULL literal if it fails (#24222)
This is an automated email from the ASF dual-hosted git repository. morningman pushed a commit to branch branch-1.2-lts in repository https://gitbox.apache.org/repos/asf/doris.git commit 81a7b69b1fa72cd09bbd366bf1af437438f76003 Author: starocean999 <40539150+starocean...@users.noreply.github.com> AuthorDate: Thu Sep 14 15:59:20 2023 +0800 [fix](planner)cast string to float like type should return NULL literal if it fails (#24222) --- .../org/apache/doris/analysis/StringLiteral.java | 6 +-- .../data/correctness_p0/test_cast_null.out | 4 ++ .../suites/correctness_p0/test_cast_null.groovy| 46 ++ 3 files changed, 52 insertions(+), 4 deletions(-) diff --git a/fe/fe-core/src/main/java/org/apache/doris/analysis/StringLiteral.java b/fe/fe-core/src/main/java/org/apache/doris/analysis/StringLiteral.java index 1804730663d..743b074f909 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/analysis/StringLiteral.java +++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/StringLiteral.java @@ -24,8 +24,6 @@ import org.apache.doris.catalog.PrimitiveType; import org.apache.doris.catalog.Type; import org.apache.doris.common.AnalysisException; import org.apache.doris.common.DdlException; -import org.apache.doris.common.ErrorCode; -import org.apache.doris.common.ErrorReport; import org.apache.doris.common.io.Text; import org.apache.doris.qe.VariableVarConverters; import org.apache.doris.thrift.TExprNode; @@ -249,9 +247,9 @@ public class StringLiteral extends LiteralExpr { try { return new FloatLiteral(Double.valueOf(value), targetType); } catch (NumberFormatException e) { - ErrorReport.reportAnalysisException(ErrorCode.ERR_BAD_NUMBER, value); +// consistent with CastExpr's getResultValue() method +return new NullLiteral(); } -break; case DECIMALV2: case DECIMAL32: case DECIMAL64: diff --git a/regression-test/data/correctness_p0/test_cast_null.out b/regression-test/data/correctness_p0/test_cast_null.out new file mode 100644 index 000..aca783cc93d --- /dev/null +++ b/regression-test/data/correctness_p0/test_cast_null.out @@ -0,0 +1,4 @@ +-- This file is automatically generated. You should know what you did if you want to edit this +-- !sql5 -- +\N + diff --git a/regression-test/suites/correctness_p0/test_cast_null.groovy b/regression-test/suites/correctness_p0/test_cast_null.groovy new file mode 100644 index 000..f6761c8b90b --- /dev/null +++ b/regression-test/suites/correctness_p0/test_cast_null.groovy @@ -0,0 +1,46 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +suite("test_cast_null") { +sql """ +drop table if exists test_table_t1; +""" + +sql """ +CREATE TABLE `test_table_t1` ( +`k1` DECIMAL(12, 5) NULL +) ENGINE=OLAP +DUPLICATE KEY(`k1`) +COMMENT 'OLAP' +DISTRIBUTED BY HASH(`k1`) BUCKETS 1 +PROPERTIES ( +"replication_allocation" = "tag.location.default: 1", +"is_being_synced" = "false", +"storage_format" = "V2", +"light_schema_change" = "true", +"disable_auto_compaction" = "false", +"enable_single_replica_compaction" = "false" +); +""" +sql """insert into test_table_t1 values(1.0);""" + +qt_sql5 """select k1 <> '' from test_table_t1;""" + +sql """ +drop table if exists test_table_t1; +""" +} - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] 05/06: [fix](planner)SelectStmt's constructor should initialize originSelectList member (#24755)
This is an automated email from the ASF dual-hosted git repository. morningman pushed a commit to branch branch-1.2-lts in repository https://gitbox.apache.org/repos/asf/doris.git commit 5a7e8495e636576c17ea7f60c9a8d9685c7e0950 Author: starocean999 <40539150+starocean...@users.noreply.github.com> AuthorDate: Fri Sep 22 19:31:29 2023 +0800 [fix](planner)SelectStmt's constructor should initialize originSelectList member (#24755) resetSelectList method will use originSelectList to recover the origin select list. If the originSelectList is lost in constructor, the resetSelectList will fail to recover and make the analyze process fail. --- .../java/org/apache/doris/analysis/SelectStmt.java | 1 + .../test_inlineview_with_window_function.out | 4 ++ .../test_inlineview_with_window_function.groovy| 81 ++ 3 files changed, 86 insertions(+) diff --git a/fe/fe-core/src/main/java/org/apache/doris/analysis/SelectStmt.java b/fe/fe-core/src/main/java/org/apache/doris/analysis/SelectStmt.java index 1ed0960411c..6cc606b204b 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/analysis/SelectStmt.java +++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/SelectStmt.java @@ -164,6 +164,7 @@ public class SelectStmt extends QueryStmt { this.id = other.id; selectList = other.selectList.clone(); fromClause = other.fromClause.clone(); +originSelectList = other.originSelectList != null ? other.originSelectList.clone() : null; whereClause = (other.whereClause != null) ? other.whereClause.clone() : null; groupByClause = (other.groupByClause != null) ? other.groupByClause.clone() : null; havingClause = (other.havingClause != null) ? other.havingClause.clone() : null; diff --git a/regression-test/data/correctness_p0/test_inlineview_with_window_function.out b/regression-test/data/correctness_p0/test_inlineview_with_window_function.out new file mode 100644 index 000..281bf5dd1b2 --- /dev/null +++ b/regression-test/data/correctness_p0/test_inlineview_with_window_function.out @@ -0,0 +1,4 @@ +-- This file is automatically generated. You should know what you did if you want to edit this +-- !order -- +2023-06-10 cib2205045_1_1s 0. 168939.00. 0.0 0. 0.0 0. day + diff --git a/regression-test/suites/correctness_p0/test_inlineview_with_window_function.groovy b/regression-test/suites/correctness_p0/test_inlineview_with_window_function.groovy index 0da26ff8f37..cdeb65e9ec5 100644 --- a/regression-test/suites/correctness_p0/test_inlineview_with_window_function.groovy +++ b/regression-test/suites/correctness_p0/test_inlineview_with_window_function.groovy @@ -70,4 +70,85 @@ suite("test_inlineview_with_window_function") { sql """ drop table if exists test_table_bb; """ + +sql """drop table if exists test_table_aaa;""" +sql """drop table if exists test_table_bbb;""" +sql """CREATE TABLE `test_table_aaa` ( +`ordernum` varchar(65533) NOT NULL , +`dnt` datetime NOT NULL , +`data` json NULL +) ENGINE=OLAP +DUPLICATE KEY(`ordernum`, `dnt`) +COMMENT 'OLAP' +DISTRIBUTED BY HASH(`ordernum`) BUCKETS 3 +PROPERTIES ( +"replication_allocation" = "tag.location.default: 1" +);""" + +sql """CREATE TABLE `test_table_bbb` ( +`dnt` varchar(200) NULL, +`ordernum` varchar(200) NULL, +`type` varchar(20) NULL, +`powers` double SUM NULL, +`p0` double REPLACE NULL, +`heatj` double SUM NULL, +`j0` double REPLACE NULL, +`heatg` double SUM NULL, +`g0` double REPLACE NULL, +`solar` double SUM NULL +) ENGINE=OLAP +AGGREGATE KEY(`dnt`, `ordernum`, `type`) +COMMENT 'OLAP' +DISTRIBUTED BY HASH(`ordernum`) BUCKETS 1 +PROPERTIES ( +"replication_allocation" = "tag.location.default: 1" +); """ + +sql """DROP MATERIALIZED VIEW IF EXISTS ods_zn_dnt_max1 ON test_table_aaa;""" +sql """create materialized view ods_zn_dnt_max1 as +select ordernum,max(dnt) as dnt from test_table_aaa +group by ordernum +ORDER BY ordernum;""" + +sql """insert into test_table_aaa values('cib2205045_1_1s','2023/6/10 3:55:33','{"DB1":168939,"DNT":"2023-06-10 03:55:33"}');""" +sql """insert into test_table_aaa values('cib2205045_1_1s','2023/6/10 3:56:33','{"DB1":168939,"DNT":"2023-06-10 03:56:33"}');""" +sql """insert into test_table_aaa values('cib2205045_1_1s','2023/6/10 3:57:33','{"DB1":168939,"DNT":"2023-06-10 03:57:33"}');""" +sql """insert into test_table_aaa values('cib2205045_1_1s','2023/6/10 3:58:33','{"DB1":168939,"DNT":"2023-06-10 03:58:33"}');""" + +qt_order """select +'2023-06-10', +
[doris] branch branch-1.2-lts updated (01d4aa726bd -> b40fc3e6115)
This is an automated email from the ASF dual-hosted git repository. morningman pushed a change to branch branch-1.2-lts in repository https://gitbox.apache.org/repos/asf/doris.git from 01d4aa726bd [Fix](Parquet Reader)Branch 1.2 lts bugfix (#24094) new 81a7b69b1fa [fix](planner)cast string to float like type should return NULL literal if it fails (#24222) new 887dcf17aca [fix](planner) having clause analyze bug #24288 new faf0588e009 [fix](planner) statement run successful but log error msg in audit log (#24628) new 9ef74b5f7ca [fix](planner) do not support UDF without paramter (#24730) new 5a7e8495e63 [fix](planner)SelectStmt's constructor should initialize originSelectList member (#24755) new b40fc3e6115 [Fix](Planner) disable bitmap type in compare expression (#24792) The 6 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: fe/fe-core/src/main/cup/sql_parser.cup | 20 - .../org/apache/doris/analysis/BinaryPredicate.java | 1 + .../java/org/apache/doris/analysis/CaseExpr.java | 6 ++ .../java/org/apache/doris/analysis/CastExpr.java | 3 + .../org/apache/doris/analysis/DateLiteral.java | 4 + .../apache/doris/analysis/DefaultValueExprDef.java | 4 + .../main/java/org/apache/doris/analysis/Expr.java | 11 +++ .../apache/doris/analysis/ExpressionFunctions.java | 9 +++ .../org/apache/doris/analysis/GroupByClause.java | 4 + .../org/apache/doris/analysis/InPredicate.java | 1 + .../java/org/apache/doris/analysis/IntLiteral.java | 4 + .../java/org/apache/doris/analysis/QueryStmt.java | 4 + .../java/org/apache/doris/analysis/SelectStmt.java | 88 - .../org/apache/doris/analysis/StringLiteral.java | 6 +- .../apache/doris/datasource/InternalCatalog.java | 3 + .../apache/doris/rewrite/FoldConstantsRule.java| 6 ++ .../doris/rewrite/RewriteBinaryPredicatesRule.java | 4 + .../doris/rewrite/RewriteDateLiteralRule.java | 4 + .../doris/rewrite/RewriteInPredicateRule.java | 7 ++ .../doris/rewrite/mvrewrite/MVExprEquivalent.java | 1 + .../org/apache/doris/planner/QueryPlanTest.java| 2 +- .../test_cast_null.out}| 2 +- .../test_inlineview_with_window_function.out} | 4 +- .../data/datatype_p0/bitmap/test_bitmap_int.out| Bin 315 -> 351 bytes ...rouping_window.groovy => test_cast_null.groovy} | 31 .../test_inlineview_with_window_function.groovy| 81 +++ .../datatype_p0/bitmap/test_bitmap_int.groovy | 33 .../suites/query_p0/having/having.groovy | 49 .../sql_functions/test_alias_function.groovy | 25 +- 29 files changed, 352 insertions(+), 65 deletions(-) copy regression-test/data/{query_p0/aggregate/nullablity_consistency.out => correctness_p0/test_cast_null.out} (89%) copy regression-test/data/{brown_p2/sql/dataTransferRate.out => correctness_p0/test_inlineview_with_window_function.out} (51%) copy regression-test/suites/correctness_p0/{test_no_grouping_window.groovy => test_cast_null.groovy} (60%) create mode 100644 regression-test/suites/query_p0/having/having.groovy - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] 04/06: [fix](planner) do not support UDF without paramter (#24730)
This is an automated email from the ASF dual-hosted git repository. morningman pushed a commit to branch branch-1.2-lts in repository https://gitbox.apache.org/repos/asf/doris.git commit 9ef74b5f7ca7ed4b160ac32e3e175f5ac2e0fc10 Author: morrySnow <101034200+morrys...@users.noreply.github.com> AuthorDate: Thu Sep 21 22:23:21 2023 +0800 [fix](planner) do not support UDF without paramter (#24730) for example: CREATE ALIAS FUNCTION f() WITH PARAMETERS() AS now(); --- fe/fe-core/src/main/cup/sql_parser.cup | 20 ++--- .../apache/doris/analysis/ExpressionFunctions.java | 3 +++ .../sql_functions/test_alias_function.groovy | 25 +- 3 files changed, 44 insertions(+), 4 deletions(-) diff --git a/fe/fe-core/src/main/cup/sql_parser.cup b/fe/fe-core/src/main/cup/sql_parser.cup index a15c09a0d06..dcf6fd394ab 100644 --- a/fe/fe-core/src/main/cup/sql_parser.cup +++ b/fe/fe-core/src/main/cup/sql_parser.cup @@ -683,7 +683,7 @@ nonterminal ArrayList expr_list, values, row_value, opt_values; nonterminal ArrayList func_arg_list; nonterminal ArrayList expr_pipe_list; nonterminal String select_alias, opt_table_alias, lock_alias, opt_alias; -nonterminal ArrayList ident_list; +nonterminal ArrayList ident_list, opt_ident_list; nonterminal PartitionNames opt_partition_names, partition_names; nonterminal ArrayList opt_tablet_list, tablet_list; nonterminal TableSample opt_table_sample, table_sample; @@ -1737,7 +1737,7 @@ create_stmt ::= RESULT = new CreateFunctionStmt(ifNotExists, isAggregate, functionName, args, returnType, intermediateType, properties); :} | KW_CREATE KW_ALIAS KW_FUNCTION opt_if_not_exists:ifNotExists function_name:functionName LPAREN func_args_def:args RPAREN -KW_WITH KW_PARAMETER LPAREN ident_list:parameters RPAREN KW_AS expr:func +KW_WITH KW_PARAMETER LPAREN opt_ident_list:parameters RPAREN KW_AS expr:func {: RESULT = new CreateFunctionStmt(ifNotExists, functionName, args, parameters, func); :} @@ -5378,6 +5378,16 @@ ident_list ::= :} ; +opt_ident_list ::= +{: +RESULT = Lists.newArrayList(); +:} +| ident_list:list +{: +RESULT = list; +:} +; + expr_list ::= expr:e {: @@ -5675,7 +5685,11 @@ type_def_list ::= ; func_args_def ::= - type_def_list:argTypes + /* empty */ + {: +RESULT = new FunctionArgsDef(Lists.newArrayList(), false); + :} + | type_def_list:argTypes {: RESULT = new FunctionArgsDef(argTypes, false); :} diff --git a/fe/fe-core/src/main/java/org/apache/doris/analysis/ExpressionFunctions.java b/fe/fe-core/src/main/java/org/apache/doris/analysis/ExpressionFunctions.java index 4357c35011b..826d3e7a7ae 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/analysis/ExpressionFunctions.java +++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/ExpressionFunctions.java @@ -67,6 +67,9 @@ public enum ExpressionFunctions { || constExpr instanceof FunctionCallExpr || constExpr instanceof TimestampArithmeticExpr) { Function fn = constExpr.getFn(); +if (fn == null) { +return constExpr; +} Preconditions.checkNotNull(fn, "Expr's fn can't be null."); diff --git a/regression-test/suites/query_p0/sql_functions/test_alias_function.groovy b/regression-test/suites/query_p0/sql_functions/test_alias_function.groovy index 5cd25fd694d..c53e2e89b55 100644 --- a/regression-test/suites/query_p0/sql_functions/test_alias_function.groovy +++ b/regression-test/suites/query_p0/sql_functions/test_alias_function.groovy @@ -16,7 +16,6 @@ // under the License. suite('test_alias_function') { -sql "use test_query_db" sql ''' CREATE ALIAS FUNCTION IF NOT EXISTS f1(DATETIMEV2(3), INT) with PARAMETER (datetime1, int1) as date_trunc(days_sub(datetime1, int1), 'day')''' @@ -31,4 +30,28 @@ suite('test_alias_function') { sql '''select f2(f1('2023-03-29', 2), 3)''' result([['20230327:01']]) } + +sql "set enable_nereids_planner=false" + +sql ''' +DROP FUNCTION IF EXISTS legacy_f4() +''' + +sql ''' +CREATE ALIAS FUNCTION legacy_f4() WITH PARAMETER() AS now() +''' + +sql ''' +SELECT legacy_f4(), now() +''' + +sql "set enable_nereids_planner=true" + +sql ''' +SELECT legacy_f4(), now() +''' + +sql ''' +DROP FUNCTION IF EXISTS legacy_f4() +''' } - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] 03/06: [fix](planner) statement run successful but log error msg in audit log (#24628)
This is an automated email from the ASF dual-hosted git repository. morningman pushed a commit to branch branch-1.2-lts in repository https://gitbox.apache.org/repos/asf/doris.git commit faf0588e009836284a55281643a9d1d4ad1f47f0 Author: morrySnow <101034200+morrys...@users.noreply.github.com> AuthorDate: Wed Sep 20 11:32:47 2023 +0800 [fix](planner) statement run successful but log error msg in audit log (#24628) legacy planner will set error msg when throw AnalysisException. However, in some place, we catch these exception and muted them. So, we should reset back error msg and error code. --- fe/fe-core/src/main/java/org/apache/doris/analysis/CastExpr.java | 3 +++ .../src/main/java/org/apache/doris/analysis/DateLiteral.java | 4 .../main/java/org/apache/doris/analysis/DefaultValueExprDef.java | 4 .../main/java/org/apache/doris/analysis/ExpressionFunctions.java | 6 ++ .../src/main/java/org/apache/doris/analysis/GroupByClause.java | 4 .../src/main/java/org/apache/doris/analysis/IntLiteral.java | 4 .../src/main/java/org/apache/doris/analysis/QueryStmt.java | 4 .../src/main/java/org/apache/doris/analysis/SelectStmt.java | 9 + .../main/java/org/apache/doris/datasource/InternalCatalog.java | 3 +++ .../main/java/org/apache/doris/rewrite/FoldConstantsRule.java| 6 ++ .../org/apache/doris/rewrite/RewriteBinaryPredicatesRule.java| 4 .../java/org/apache/doris/rewrite/RewriteDateLiteralRule.java| 4 .../java/org/apache/doris/rewrite/RewriteInPredicateRule.java| 7 +++ .../org/apache/doris/rewrite/mvrewrite/MVExprEquivalent.java | 1 + 14 files changed, 63 insertions(+) diff --git a/fe/fe-core/src/main/java/org/apache/doris/analysis/CastExpr.java b/fe/fe-core/src/main/java/org/apache/doris/analysis/CastExpr.java index e364de5279a..51c8f5a257f 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/analysis/CastExpr.java +++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/CastExpr.java @@ -408,6 +408,9 @@ public class CastExpr extends Expr { try { targetExpr = castTo((LiteralExpr) value); } catch (AnalysisException ae) { +if (ConnectContext.get() != null) { +ConnectContext.get().getState().reset(); +} targetExpr = this; } catch (NumberFormatException nfe) { targetExpr = new NullLiteral(); diff --git a/fe/fe-core/src/main/java/org/apache/doris/analysis/DateLiteral.java b/fe/fe-core/src/main/java/org/apache/doris/analysis/DateLiteral.java index 6e07904869f..f1716ad830c 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/analysis/DateLiteral.java +++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/DateLiteral.java @@ -26,6 +26,7 @@ import org.apache.doris.catalog.Type; import org.apache.doris.common.AnalysisException; import org.apache.doris.common.Config; import org.apache.doris.common.InvalidFormatException; +import org.apache.doris.qe.ConnectContext; import org.apache.doris.thrift.TDateLiteral; import org.apache.doris.thrift.TExprNode; import org.apache.doris.thrift.TExprNodeType; @@ -648,6 +649,9 @@ public class DateLiteral extends LiteralExpr { try { checkValueValid(); } catch (AnalysisException e) { +if (ConnectContext.get() != null) { +ConnectContext.get().getState().reset(); +} // If date value is invalid, set this to null msg.node_type = TExprNodeType.NULL_LITERAL; msg.setIsNullable(true); diff --git a/fe/fe-core/src/main/java/org/apache/doris/analysis/DefaultValueExprDef.java b/fe/fe-core/src/main/java/org/apache/doris/analysis/DefaultValueExprDef.java index adccf078c6f..d5b2d0548f3 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/analysis/DefaultValueExprDef.java +++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/DefaultValueExprDef.java @@ -22,6 +22,7 @@ package org.apache.doris.analysis; import org.apache.doris.catalog.Type; import org.apache.doris.common.AnalysisException; +import org.apache.doris.qe.ConnectContext; import com.google.gson.annotations.SerializedName; import org.apache.log4j.LogManager; @@ -48,6 +49,9 @@ public class DefaultValueExprDef { try { expr.analyzeImplForDefaultValue(type); } catch (AnalysisException e) { +if (ConnectContext.get() != null) { +ConnectContext.get().getState().reset(); +} LOG.warn("analyzeImplForDefaultValue fail: {}", e); } return expr; diff --git a/fe/fe-core/src/main/java/org/apache/doris/analysis/ExpressionFunctions.java b/fe/fe-core/src/main/java/org/apache/doris/analysis/ExpressionFunctions.java index 8a3fa649d3d..4357c35011b 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/analysis/ExpressionFunctions.java +++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/Expr
[doris] 06/06: [Fix](Planner) disable bitmap type in compare expression (#24792)
This is an automated email from the ASF dual-hosted git repository. morningman pushed a commit to branch branch-1.2-lts in repository https://gitbox.apache.org/repos/asf/doris.git commit b40fc3e61154760dc0db28d0f153b86d917da93d Author: LiBinfeng <46676950+libinfeng...@users.noreply.github.com> AuthorDate: Wed Sep 27 16:57:06 2023 +0800 [Fix](Planner) disable bitmap type in compare expression (#24792) Problem: be core because of bitmap calculation. Reason: when be check failed, it would core directly. Example: SELECT id_bitmap FROM test_bitmap WHERE id_bitmap IN (NULL) LIMIT 20; Solved: Forbidden this kind of expression in fe when analyze. And also forbid bitmap type comparing in other unsupported expressions. --- .../org/apache/doris/analysis/BinaryPredicate.java | 1 + .../java/org/apache/doris/analysis/CaseExpr.java | 6 .../main/java/org/apache/doris/analysis/Expr.java | 11 +++ .../org/apache/doris/analysis/InPredicate.java | 1 + .../org/apache/doris/planner/QueryPlanTest.java| 2 +- .../data/datatype_p0/bitmap/test_bitmap_int.out| Bin 315 -> 351 bytes .../datatype_p0/bitmap/test_bitmap_int.groovy | 33 + 7 files changed, 53 insertions(+), 1 deletion(-) diff --git a/fe/fe-core/src/main/java/org/apache/doris/analysis/BinaryPredicate.java b/fe/fe-core/src/main/java/org/apache/doris/analysis/BinaryPredicate.java index 11504d9571c..adf579aa99a 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/analysis/BinaryPredicate.java +++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/BinaryPredicate.java @@ -412,6 +412,7 @@ public class BinaryPredicate extends Predicate implements Writable { @Override public void analyzeImpl(Analyzer analyzer) throws AnalysisException { super.analyzeImpl(analyzer); +this.checkIncludeBitmap(); for (Expr expr : children) { if (expr instanceof Subquery) { diff --git a/fe/fe-core/src/main/java/org/apache/doris/analysis/CaseExpr.java b/fe/fe-core/src/main/java/org/apache/doris/analysis/CaseExpr.java index 83fb77c120b..177c937c0d5 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/analysis/CaseExpr.java +++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/CaseExpr.java @@ -187,6 +187,9 @@ public class CaseExpr extends Expr { if (caseExpr instanceof Subquery && !caseExpr.getType().isScalarType()) { throw new AnalysisException("Subquery in case-when must return scala type"); } +if (caseExpr.getType().isBitmapType()) { +throw new AnalysisException("Unsupported bitmap type in expression: " + toSql()); +} whenType = caseExpr.getType(); lastCompatibleWhenExpr = children.get(0); } else { @@ -221,6 +224,9 @@ public class CaseExpr extends Expr { && !((hasCaseExpr() && whenExpr instanceof Subquery || !checkSubquery(whenExpr { throw new AnalysisException("Only support subquery in binary predicate in case statement."); } +if (whenExpr.getType().isBitmapType()) { +throw new AnalysisException("Unsupported bitmap type in expression: " + toSql()); +} // Determine maximum compatible type of the then exprs seen so far. // We will add casts to them at the very end. Expr thenExpr = children.get(i + 1); diff --git a/fe/fe-core/src/main/java/org/apache/doris/analysis/Expr.java b/fe/fe-core/src/main/java/org/apache/doris/analysis/Expr.java index 8615d3d8037..cdc662502b5 100755 --- a/fe/fe-core/src/main/java/org/apache/doris/analysis/Expr.java +++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/Expr.java @@ -1334,6 +1334,17 @@ public abstract class Expr extends TreeNode implements ParseNode, Cloneabl } } +/** + * Checks whether comparing predicates' children include bitmap type. + */ +public void checkIncludeBitmap() throws AnalysisException { +for (int i = 0; i < children.size(); ++i) { +if (children.get(i).getType().isBitmapType()) { +throw new AnalysisException("Unsupported bitmap type in expression: " + toSql()); +} +} +} + public Expr checkTypeCompatibility(Type targetType) throws AnalysisException { if (targetType.getPrimitiveType() != PrimitiveType.ARRAY && targetType.getPrimitiveType() == type.getPrimitiveType()) { diff --git a/fe/fe-core/src/main/java/org/apache/doris/analysis/InPredicate.java b/fe/fe-core/src/main/java/org/apache/doris/analysis/InPredicate.java index e041549ab71..349c761f9bd 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/analysis/InPredicate.java +++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/InPredicate.java @@ -157,6 +157,7 @@ public class InPredicate extends Predicate { @Override
Re: [PR] [chore](unused) Remove unused variable CPU_HARD_LIMIT in task_group.cc [doris]
doris-robot commented on PR #25076: URL: https://github.com/apache/doris/pull/25076#issuecomment-1751641373 TeamCity be ut coverage result: Function Coverage: 36.28% (8150/22462) Line Coverage: 28.42% (65188/229392) Region Coverage: 27.36% (33761/123414) Branch Coverage: 24.02% (17220/71680) Coverage Report: http://coverage.selectdb-in.cc/coverage/fef1c0130b0924c51512d2b5651dcc0406331dae_fef1c0130b0924c51512d2b5651dcc0406331dae/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](nereids)align node id in explain with nereids node id [doris]
englefly commented on PR #25068: URL: https://github.com/apache/doris/pull/25068#issuecomment-1751643236 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [minor](be) set fd number check to 60000 for BE start script [doris]
morningman opened a new pull request, #25078: URL: https://github.com/apache/doris/pull/25078 ## Proposed changes Modify the BE fd number check to 6, because the default fd number value of some system is 65535, which is smaller than previous threshold 65536, so reduce to 6 to let Doris start normally in most of system. ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](nereids) unify withSel/updateRowCountOnly/withRowCount [doris]
github-actions[bot] commented on PR #24997: URL: https://github.com/apache/doris/pull/24997#issuecomment-1751644882 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [tools](tpc)make tpch-tools and tpcds-tools default scale factor 100 [doris]
github-actions[bot] commented on PR #25002: URL: https://github.com/apache/doris/pull/25002#issuecomment-1751645143 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](stack trace) Optimize stack trace output [doris]
github-actions[bot] commented on PR #24933: URL: https://github.com/apache/doris/pull/24933#issuecomment-1751645444 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](catalog)fix when modifying comments in property, it will modify the comments in the catalog [doris]
zddr commented on PR #24857: URL: https://github.com/apache/doris/pull/24857#issuecomment-1751645270 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch branch-1.2-lts updated: [branch-1.2] fix code format after merge
This is an automated email from the ASF dual-hosted git repository. morningman pushed a commit to branch branch-1.2-lts in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/branch-1.2-lts by this push: new 44a896fa112 [branch-1.2] fix code format after merge 44a896fa112 is described below commit 44a896fa11203b25c032afe4f7ae090e08416676 Author: morningman AuthorDate: Sat Oct 7 16:09:02 2023 +0800 [branch-1.2] fix code format after merge --- be/src/vec/exec/format/parquet/parquet_thrift_util.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/be/src/vec/exec/format/parquet/parquet_thrift_util.h b/be/src/vec/exec/format/parquet/parquet_thrift_util.h index 5642bbff040..fb55f16d45b 100644 --- a/be/src/vec/exec/format/parquet/parquet_thrift_util.h +++ b/be/src/vec/exec/format/parquet/parquet_thrift_util.h @@ -51,8 +51,8 @@ static Status parse_thrift_footer(FileReader* file, std::shared_ptr file_size - PARQUET_FOOTER_SIZE) { return Status::Corruption("Parquet file size is ", file_size, - " bytes, smaller than the size reported by footer's (", metadata_size, - "bytes)"); + " bytes, smaller than the size reported by footer's (", + metadata_size, "bytes)"); } tparquet::FileMetaData t_metadata; // deserialize footer - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Nerids) fix error when the view has lambda functions [doris]
XieJiann commented on PR #25067: URL: https://github.com/apache/doris/pull/25067#issuecomment-1751645745 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improvement](hash) refactor of hash map context [doris]
doris-robot commented on PR #24966: URL: https://github.com/apache/doris/pull/24966#issuecomment-1751646126 TeamCity be ut coverage result: Function Coverage: 36.35% (8139/22390) Line Coverage: 28.50% (65160/228671) Region Coverage: 27.40% (33739/123155) Branch Coverage: 24.05% (17220/71592) Coverage Report: http://coverage.selectdb-in.cc/coverage/6647dbdcde7e6d8d8d847ced0474d95504b7e4f0_6647dbdcde7e6d8d8d847ced0474d95504b7e4f0/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](code-style)add spotless plugin and remove checkstyle [doris]
liugddx commented on PR #25033: URL: https://github.com/apache/doris/pull/25033#issuecomment-1751646118 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](unused) Remove unused variable CPU_HARD_LIMIT in task_group.cc [doris]
doris-robot commented on PR #25076: URL: https://github.com/apache/doris/pull/25076#issuecomment-1751646379 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.58 seconds stream load tsv: 574 seconds loaded 74807831229 Bytes, about 124 MB/s stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 29.0 seconds inserted 1000 Rows, about 344K ops/s storage size: 17162266941 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] Dev 0925 2 [doris]
BiteThet closed pull request #24856: Dev 0925 2 URL: https://github.com/apache/doris/pull/24856 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [feat](optimizer) Scale sample stats with ratio to make it more precise [doris]
Kikyou1997 opened a new pull request, #25079: URL: https://github.com/apache/doris/pull/25079 ## Proposed changes Since Doris support query specific tablet only, so we don't depend on tableSample to do sample, instead use grammar: TABLET(id) to do so. In OlapAnalyzeTask, we calculate which tablets would be hit and set theirs id in it, so we could get how many rows actually queried and furthur we could get the scale up ratio here ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](es-catalog) add `include_hidden_index`in order to get the hidden index. [doris]
liugddx commented on PR #24826: URL: https://github.com/apache/doris/pull/24826#issuecomment-1751647216 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](code-style)add spotless plugin and remove checkstyle [doris]
doris-robot commented on PR #25033: URL: https://github.com/apache/doris/pull/25033#issuecomment-1751647307 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.98 seconds stream load tsv: 562 seconds loaded 74807831229 Bytes, about 126 MB/s stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s insert into select: 28.9 seconds inserted 1000 Rows, about 346K ops/s storage size: 17162279212 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Nerids) fix error when the view has lambda functions [doris]
XieJiann commented on PR #25067: URL: https://github.com/apache/doris/pull/25067#issuecomment-1751647906 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (335804bb253 -> 1405f1efd24)
This is an automated email from the ASF dual-hosted git repository. englefly pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 335804bb253 [fix](pipelinex) fix multi cast sink without init (#25066) add 1405f1efd24 [refactor](nereids) unify withSel/updateRowCountOnly/withRowCount (#24997) No new revisions were added by this update. Summary of changes: .../nereids/jobs/cascades/DeriveStatsJob.java | 2 +- .../doris/nereids/stats/ExpressionEstimation.java | 2 +- .../doris/nereids/stats/FilterEstimation.java | 36 +- .../apache/doris/nereids/stats/JoinEstimation.java | 14 ++-- .../doris/nereids/stats/StatsCalculator.java | 16 ++--- .../doris/nereids/types/coercion/DateLikeType.java | 13 ++-- .../apache/doris/statistics/ColumnStatistic.java | 8 +-- .../org/apache/doris/statistics/Statistics.java| 76 ++ .../doris/statistics/util/StatisticsUtil.java | 4 +- 9 files changed, 53 insertions(+), 118 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](optimizer) Scale sample stats with ratio to make it more precise [doris]
Kikyou1997 commented on PR #25079: URL: https://github.com/apache/doris/pull/25079#issuecomment-1751648620 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](nereids) unify withSel/updateRowCountOnly/withRowCount [doris]
englefly merged PR #24997: URL: https://github.com/apache/doris/pull/24997 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](stream load) stearm load record use valid txn info when two txn with same label [doris]
github-actions[bot] commented on PR #24320: URL: https://github.com/apache/doris/pull/24320#issuecomment-1751648903 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](stream load) stearm load record use valid txn info when two txn with same label [doris]
github-actions[bot] commented on PR #24320: URL: https://github.com/apache/doris/pull/24320#issuecomment-1751648915 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [minor](be) set fd number check to 60000 for BE start script [doris]
morningman commented on PR #25078: URL: https://github.com/apache/doris/pull/25078#issuecomment-1751649901 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
yujun777 commented on PR #21118: URL: https://github.com/apache/doris/pull/21118#issuecomment-1751649939 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [optimize](chinese) optimize stopwords [doris-thirdparty]
qidaye merged PR #128: URL: https://github.com/apache/doris-thirdparty/pull/128 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-thirdparty] branch clucene updated: [optimize](chinese) optimize stopwords (#128)
This is an automated email from the ASF dual-hosted git repository. jianliangqi pushed a commit to branch clucene in repository https://gitbox.apache.org/repos/asf/doris-thirdparty.git The following commit(s) were added to refs/heads/clucene by this push: new a950c0ab [optimize](chinese) optimize stopwords (#128) a950c0ab is described below commit a950c0ab370f210b92a70139217fe1ee2511bda2 Author: zzzxl <33418555+zzzxl1...@users.noreply.github.com> AuthorDate: Sat Oct 7 16:29:32 2023 +0800 [optimize](chinese) optimize stopwords (#128) --- .../CLucene/analysis/jieba/dict/stop_words.utf8| 739 + 1 file changed, 17 insertions(+), 722 deletions(-) diff --git a/src/contribs-lib/CLucene/analysis/jieba/dict/stop_words.utf8 b/src/contribs-lib/CLucene/analysis/jieba/dict/stop_words.utf8 index 32ac9e67..6f04 100644 --- a/src/contribs-lib/CLucene/analysis/jieba/dict/stop_words.utf8 +++ b/src/contribs-lib/CLucene/analysis/jieba/dict/stop_words.utf8 @@ -127,6 +127,23 @@ _ ︼ 的 了 +和 +是 +就 +都 +而 +及 +兴 +着 +或 +一个 +没有 +我们 +你们 +妳们 +他们 +她们 +是否 the a an @@ -153,728 +170,6 @@ _ 。 《 》 -一 -一些 -一何 -一切 -一则 -一方面 -一旦 -一来 -一样 -一般 -一转眼 -万一 -上 -上下 -下 -不 -不仅 -不但 -不光 -不单 -不只 -不外乎 -不如 -不妨 -不尽 -不尽然 -不得 -不怕 -不惟 -不成 -不拘 -不料 -不是 -不比 -不然 -不特 -不独 -不管 -不至于 -不若 -不论 -不过 -不问 -与 -与其 -与其说 -与否 -与此同时 -且 -且不说 -且说 -两者 -个 -个别 -临 -为 -为了 -为什么 -为何 -为止 -为此 -为着 -乃 -乃至 -乃至于 -么 -之 -之一 -之所以 -之类 -乌乎 -乎 -乘 -也 -也好 -也罢 -了 -二来 -于 -于是 -于是乎 -云云 -云尔 -些 -亦 -人 -人们 -人家 -什么 -什么样 -今 -介于 -仍 -仍旧 -从 -从此 -从而 -他 -他人 -他们 -以 -以上 -以为 -以便 -以免 -以及 -以故 -以期 -以来 -以至 -以至于 -以致 -们 -任 -任何 -任凭 -似的 -但 -但凡 -但是 -何 -何以 -何况 -何处 -何时 -余外 -作为 -你 -你们 -使 -使得 -例如 -依 -依据 -依照 -便于 -俺 -俺们 -倘 -倘使 -倘或 -倘然 -倘若 -借 -假使 -假如 -假若 -傥然 -像 -儿 -先不先 -光是 -全体 -全部 -兮 -关于 -其 -其一 -其中 -其二 -其他 -其余 -其它 -其次 -具体地说 -具体说来 -兼之 -内 -再 -再其次 -再则 -再有 -再者 -再者说 -再说 -冒 -冲 -况且 -几 -几时 -凡 -凡是 -凭 -凭借 -出于 -出来 -分别 -则 -则甚 -别 -别人 -别处 -别是 -别的 -别管 -别说 -到 -前后 -前此 -前者 -加之 -加以 -即 -即令 -即使 -即便 -即如 -即或 -即若 -却 -去 -又 -又及 -及 -及其 -及至 -反之 -反而 -反过来 -反过来说 -受到 -另 -另一方面 -另外 -另悉 -只 -只当 -只怕 -只是 -只有 -只消 -只要 -只限 -叫 -叮咚 -可 -可以 -可是 -可见 -各 -各个 -各位 -各种 -各自 -同 -同时 -后 -后者 -向 -向使 -向着 -吓 -吗 -否则 -吧 -吧哒 -吱 -呀 -呃 -呕 -呗 -呜 -呜呼 -呢 -呵 -呵呵 -呸 -呼哧 -咋 -和 -咚 -咦 -咧 -咱 -咱们 -咳 -哇 -哈 -哈哈 -哉 -哎 -哎呀 -哎哟 -哗 -哟 -哦 -哩 -哪 -哪个 -哪些 -哪儿 -哪天 -哪年 -哪怕 -哪样 -哪边 -哪里 -哼 -哼唷 -唉 -唯有 -啊 -啐 -啥 -啦 -啪达 -啷当 -喂 -喏 -喔唷 -喽 -嗡 -嗡嗡 -嗬 -嗯 -嗳 -嘎 -嘎登 -嘘 -嘛 -嘻 -嘿 -嘿嘿 -因 -因为 -因了 -因此 -因着 -因而 -固然 -在 -在下 -在于 -地 -基于 -处在 -多 -多么 -多少 -大 -大家 -她 -她们 -好 -如 -如上 -如上所述 -如下 -如何 -如其 -如同 -如是 -如果 -如此 -如若 -始而 -孰料 -孰知 -宁 -宁可 -宁愿 -宁肯 -它 -它们 -对 -对于 -对待 -对方 -对比 -将 -小 -尔 -尔后 -尔尔 -尚且 -就 -就是 -就是了 -就是说 -就算 -就要 -尽 -尽管 -尽管如此 -岂但 -己 -已 -已矣 -巴 -巴巴 -并 -并且 -并非 -庶乎 -庶几 -开外 -开始 -归 -归齐 -当 -当地 -当然 -当着 -彼 -彼时 -彼此 -往 -待 -很 -得 -得了 -怎 -怎么 -怎么办 -怎么样 -怎奈 -怎样 -总之 -总的来看 -总的来说 -总的说来 -总而言之 -恰恰相反 -您 -惟其 -慢说 -我 -我们 -或 -或则 -或是 -或曰 -或者 -截至 -所 -所以 -所在 -所幸 -所有 -才 -才能 -打 -打从 -把 -抑或 -拿 -按 -按照 -换句话说 -换言之 -据 -据此 -接着 -故 -故此 -故而 -旁人 -无 -无宁 -无论 -既 -既往 -既是 -既然 -时候 -是 -是以 -是的 -曾 -替 -替代 -最 -有 -有些 -有关 -有及 -有时 -有的 -望 -朝 -朝着 -本 -本人 -本地 -本着 -本身 -来 -来着 -来自 -来说 -极了 -果然 -果真 -某 -某个 -某些 -某某 -根据 -欤 -正值 -正如 -正巧 -正是 -此 -此地 -此处 -此外 -此时 -此次 -此间 -毋宁 -每 -每当 -比 -比及 -比如 -比方 -没奈何 -沿 -沿着 -漫说 -焉 -然则 -然后 -然而 -照 -照着 -犹且 -犹自 -甚且 -甚么 -甚或 -甚而 -甚至 -甚至于 -用 -用来 -由 -由于 -由是 -由此 -由此可见 -的 -的确 -的话 -直到 -相对而言 -省得 -看 -眨眼 -着 -着呢 -矣 -矣乎 -矣哉 -离 -竟而 -第 -等 -等到 -等等 -简言之 -管 -类如 -紧接着 -纵 -纵令 -纵使 -纵然 -经 -经过 -结果 -给 -继之 -继后 -继而 -综上所述 -罢了 -者 -而 -而且 -而况 -而后 -而外 -而已 -而是 -而言 -能 -能否 -腾 -自 -自个儿 -自从 -自各儿 -自后 -自家 -自己 -自打 -自身 -至 -至于 -至今 -至若 -致 -般的 -若 -若夫 -若是 -若果 -若非 -莫不然 -莫如 -莫若 -虽 -虽则 -虽然 -虽说 -被 -要 -要不 -要不是 -要不然 -要么 -要是 -譬喻 -譬如 -让 -许多 -论 -设使 -设或 -设若 -诚如 -诚然 -该 -说来 -诸 -诸位 -诸如 -谁 -谁人 -谁料 -谁知 -贼死 -赖以 -赶 -起 -起见 -趁 -趁着 -越是 -距 -跟 -较 -较之 -边 -过 -还 -还是 -还有 -还要 -这 -这一来 -这个 -这么 -这么些 -这么样 -这么点儿 -这些 -这会儿 -这儿 -这就是说 -这时 -这样 -这次 -这般 -这边 -这里 -进而 -连 -连同 -逐步 -通过 -遵循 -遵照 -那 -那个 -那么 -那么些 -那么样 -那些 -那会儿 -那儿 -那时 -那样 -那般 -那边 -那里 -都 -鄙人 -鉴于 -针对 -阿 -除 -除了 -除外 -除开 -除此之外 -除非 -随 -随后 -随时 -随着 -难道说 -非但 -非徒 -非特 -非独 -靠 -顺 -顺着 -首先 ! , : - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [minor](be) set fd number check to 60000 for BE start script [doris]
github-actions[bot] commented on PR #25078: URL: https://github.com/apache/doris/pull/25078#issuecomment-1751651025 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [minor](be) set fd number check to 60000 for BE start script [doris]
github-actions[bot] commented on PR #25078: URL: https://github.com/apache/doris/pull/25078#issuecomment-1751651035 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](x-load) support config min replica num for loading data [doris]
yujun777 commented on PR #21118: URL: https://github.com/apache/doris/pull/21118#issuecomment-1751651360 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](Nereids): new Date/Datetime parser [doris]
jackwener commented on PR #24523: URL: https://github.com/apache/doris/pull/24523#issuecomment-1751651748 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](unused) Remove unused variable CPU_HARD_LIMIT in task_group.cc [doris]
zy-kkk merged PR #25076: URL: https://github.com/apache/doris/pull/25076 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (1405f1efd24 -> 59261174d55)
This is an automated email from the ASF dual-hosted git repository. zykkk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 1405f1efd24 [refactor](nereids) unify withSel/updateRowCountOnly/withRowCount (#24997) add 59261174d55 [chore](unused) Remove unused variable CPU_HARD_LIMIT in task_group.cc (#25076) No new revisions were added by this update. Summary of changes: be/src/runtime/task_group/task_group.cpp | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [log](load) PUBLISH_TIMEOUT should not print stacktrace [doris]
zhannngchen opened a new pull request, #25080: URL: https://github.com/apache/doris/pull/25080 ## Proposed changes Issue Number: close #xxx PUBLISH_TIMEOUT is a very comment error code, should not print stack trace ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](catalog)compatible with paimon 0.5 [doris]
zddr commented on PR #24985: URL: https://github.com/apache/doris/pull/24985#issuecomment-1751652486 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](nereids)align node id in explain with nereids node id [doris]
doris-robot commented on PR #25068: URL: https://github.com/apache/doris/pull/25068#issuecomment-1751652856 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.48 seconds stream load tsv: 560 seconds loaded 74807831229 Bytes, about 127 MB/s stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 28.9 seconds inserted 1000 Rows, about 346K ops/s storage size: 17162148481 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](catalog)fix when modifying comments in property, it will modify the comments in the catalog [doris]
doris-robot commented on PR #24857: URL: https://github.com/apache/doris/pull/24857#issuecomment-1751653052 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.22 seconds stream load tsv: 561 seconds loaded 74807831229 Bytes, about 127 MB/s stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 29.2 seconds inserted 1000 Rows, about 342K ops/s storage size: 17162293610 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](stream load) stearm load record use valid txn info when two txn with same label [doris]
yiguolei merged PR #24320: URL: https://github.com/apache/doris/pull/24320 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [Fix](stream load) stearm load record use valid txn info when two txn with same label #24320
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/master by this push: new 976335e236a [Fix](stream load) stearm load record use valid txn info when two txn with same label #24320 976335e236a is described below commit 976335e236ad91b1036c6578e79b29ab406671c9 Author: wangqt AuthorDate: Sat Oct 7 16:42:45 2023 +0800 [Fix](stream load) stearm load record use valid txn info when two txn with same label #24320 Co-authored-by: wangqingtao6 --- fe/fe-core/src/main/java/org/apache/doris/load/StreamLoadRecordMgr.java | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fe/fe-core/src/main/java/org/apache/doris/load/StreamLoadRecordMgr.java b/fe/fe-core/src/main/java/org/apache/doris/load/StreamLoadRecordMgr.java index 149d5c0a921..2848311f895 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/load/StreamLoadRecordMgr.java +++ b/fe/fe-core/src/main/java/org/apache/doris/load/StreamLoadRecordMgr.java @@ -142,6 +142,8 @@ public class StreamLoadRecordMgr extends MasterDaemon { Map labelToStreamLoadRecord = dbIdToLabelToStreamLoadRecord.get(dbId); if (!labelToStreamLoadRecord.containsKey(label)) { labelToStreamLoadRecord.put(label, streamLoadRecord); +} else if (labelToStreamLoadRecord.get(label).getFinishTime().compareTo(streamLoadRecord.getFinishTime()) < 0) { +labelToStreamLoadRecord.put(label, streamLoadRecord); } writeUnlock(); } - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[I] [Bug] where,某些情况下多个条件无法被正确解释? [doris]
ynzzxc opened a new issue, #25081: URL: https://github.com/apache/doris/issues/25081 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version 2.0 ### What's Wrong? 示例如下: 条件1  返回了正确结果; 条件2  返回了正确结果; 条件1 and 条件2  返回了错误结果; ### What You Expected? 1 ### How to Reproduce? _No response_ ### Anything Else? _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org