Re: [PR] add -Wno-incompatible-pointer-types for ext [doris-thirdparty]

2025-07-17 Thread via GitHub
BiteThet commented on PR #336: URL: https://github.com/apache/doris-thirdparty/pull/336#issuecomment-3086680826 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

Re: [PR] [fix](inverted index) Add position compression limit [doris-thirdparty]

2025-07-03 Thread via GitHub
zzzxl1993 commented on PR #324: URL: https://github.com/apache/doris-thirdparty/pull/324#issuecomment-3034697953 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[I] 导入数据只导入了一部分 [doris-streamloader]

2025-06-03 Thread via GitHub
HelloPtr opened a new issue, #30: URL: https://github.com/apache/doris-streamloader/issues/30 ./doris-streamloader --source_file=save_doris_tmp.txt --url="http://xxx"; --header="column_separator:&?format:csv?max_filter_ratio:0" --db="xxx" --table=xxx --u="ad_data" --p="x@x" --auto_retry_int

Re: [PR] [improve] update config default value and add combine flush mode [doris-kafka-connector]

2025-05-11 Thread via GitHub
JNSimba commented on code in PR #73: URL: https://github.com/apache/doris-kafka-connector/pull/73#discussion_r2083726760 ## src/main/java/org/apache/doris/kafka/connector/service/DorisCombinedSinkService.java: ## @@ -0,0 +1,143 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] [improve] update config default value and add combine flush mode [doris-kafka-connector]

2025-05-09 Thread via GitHub
JNSimba commented on code in PR #73: URL: https://github.com/apache/doris-kafka-connector/pull/73#discussion_r2081417047 ## src/main/java/org/apache/doris/kafka/connector/service/DorisCombinedSinkService.java: ## @@ -0,0 +1,143 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] [improve] update config default value and add combine flush mode [doris-kafka-connector]

2025-05-09 Thread via GitHub
chuang-wang-pre commented on code in PR #73: URL: https://github.com/apache/doris-kafka-connector/pull/73#discussion_r2081428633 ## src/main/java/org/apache/doris/kafka/connector/service/DorisCombinedSinkService.java: ## @@ -0,0 +1,143 @@ +/* + * Licensed to the Apache Software

Re: [PR] [improve] update config default value and add combine flush mode [doris-kafka-connector]

2025-05-09 Thread via GitHub
chuang-wang-pre commented on code in PR #73: URL: https://github.com/apache/doris-kafka-connector/pull/73#discussion_r2081403240 ## src/main/java/org/apache/doris/kafka/connector/service/DorisCombinedSinkService.java: ## @@ -0,0 +1,143 @@ +/* + * Licensed to the Apache Software

Re: [PR] [improve] update config default value and add combine flush mode [doris-kafka-connector]

2025-05-09 Thread via GitHub
JNSimba commented on code in PR #73: URL: https://github.com/apache/doris-kafka-connector/pull/73#discussion_r2081304171 ## src/main/java/org/apache/doris/kafka/connector/service/DorisCombinedSinkService.java: ## @@ -0,0 +1,143 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] [improve] update config default value and add combine flush mode [doris-kafka-connector]

2025-05-08 Thread via GitHub
chuang-wang-pre commented on code in PR #73: URL: https://github.com/apache/doris-kafka-connector/pull/73#discussion_r2080993360 ## src/main/java/org/apache/doris/kafka/connector/service/DorisCombinedSinkService.java: ## @@ -0,0 +1,143 @@ +/* + * Licensed to the Apache Software

Re: [PR] [improve] update config default value and add combine flush mode [doris-kafka-connector]

2025-05-08 Thread via GitHub
JNSimba commented on code in PR #73: URL: https://github.com/apache/doris-kafka-connector/pull/73#discussion_r2080824130 ## src/main/java/org/apache/doris/kafka/connector/service/DorisCombinedSinkService.java: ## @@ -0,0 +1,143 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] [improve] update config default value and add combine flush mode [doris-kafka-connector]

2025-05-08 Thread via GitHub
JNSimba commented on code in PR #73: URL: https://github.com/apache/doris-kafka-connector/pull/73#discussion_r2080766838 ## src/main/java/org/apache/doris/kafka/connector/service/DorisCombinedSinkService.java: ## @@ -0,0 +1,143 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] [improve] update config default value and add combine flush mode [doris-kafka-connector]

2025-05-08 Thread via GitHub
DongLiang-0 commented on code in PR #73: URL: https://github.com/apache/doris-kafka-connector/pull/73#discussion_r2080756707 ## src/main/java/org/apache/doris/kafka/connector/service/DorisCombinedSinkService.java: ## @@ -0,0 +1,143 @@ +/* + * Licensed to the Apache Software Fou

Re: [PR] [improve] update config default value and add combine flush mode [doris-kafka-connector]

2025-05-08 Thread via GitHub
JNSimba commented on code in PR #73: URL: https://github.com/apache/doris-kafka-connector/pull/73#discussion_r2079706137 ## src/main/java/org/apache/doris/kafka/connector/service/DorisCombinedSinkService.java: ## @@ -0,0 +1,276 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] [improve] update config default value and add combine flush mode [doris-kafka-connector]

2025-05-08 Thread via GitHub
DongLiang-0 commented on code in PR #73: URL: https://github.com/apache/doris-kafka-connector/pull/73#discussion_r2079663191 ## src/main/java/org/apache/doris/kafka/connector/service/DorisCombinedSinkService.java: ## @@ -0,0 +1,276 @@ +/* + * Licensed to the Apache Software Fou

Re: [PR] [Chore] update version to 25.0.0 [doris-kafka-connector]

2025-04-28 Thread via GitHub
JNSimba commented on PR #71: URL: https://github.com/apache/doris-kafka-connector/pull/71#issuecomment-2837366099 > @JNSimba hi~ I noticed that the doc on doris-website has not been updated since version 24.0.0 , Can I try it ? Of course, thanks ! -- This is an automated message f

Re: [PR] [Chore] update version to 25.0.0 [doris-kafka-connector]

2025-04-28 Thread via GitHub
chuang-wang-pre commented on PR #71: URL: https://github.com/apache/doris-kafka-connector/pull/71#issuecomment-2837352587 @JNSimba hi~ I noticed that the doc on doris-website has not been updated since version 24.0.0 , Can I try it ? -- This is an automated message from the Apache Git Se

Re: [PR] [fix] Fix complex type late materialization. [doris-thirdparty]

2025-04-27 Thread via GitHub
Copilot commented on code in PR #311: URL: https://github.com/apache/doris-thirdparty/pull/311#discussion_r2062812917 ## c++/src/Reader.cc: ## @@ -1341,6 +1348,7 @@ namespace orc { prepareFollowReaders(footer->rowindexstride(), currentRowInStripe, followRowInStri

Re: [PR] [Improve] add behavior on null values [doris-kafka-connector]

2025-04-25 Thread via GitHub
chuang-wang-pre commented on PR #69: URL: https://github.com/apache/doris-kafka-connector/pull/69#issuecomment-2829872771 > Thanks for your contribution. Can you add an itcase with null value? You can refer to this StringMsgE2ETest @JNSimba Thank you for the reminder, I have already

Re: [PR] [Improve] check topic mutating SMTs [doris-kafka-connector]

2025-04-24 Thread via GitHub
chuang-wang-pre commented on code in PR #68: URL: https://github.com/apache/doris-kafka-connector/pull/68#discussion_r2057737144 ## src/test/java/org/apache/doris/kafka/connector/service/TestDorisSinkService.java: ## @@ -53,7 +60,11 @@ public void init() throws IOException {

Re: [PR] [Improve] check topic mutating SMTs [doris-kafka-connector]

2025-04-24 Thread via GitHub
JNSimba commented on PR #68: URL: https://github.com/apache/doris-kafka-connector/pull/68#issuecomment-2826601626 > > Thank you for your contribution. -> Because in `org.apache.kafka.connect.sink.SinkTask#preCommit`, the topic corresponds to the original topic before any transformations ar

Re: [PR] [Improve] check topic mutating SMTs [doris-kafka-connector]

2025-04-24 Thread via GitHub
JNSimba commented on code in PR #68: URL: https://github.com/apache/doris-kafka-connector/pull/68#discussion_r2057698669 ## src/test/java/org/apache/doris/kafka/connector/service/TestDorisSinkService.java: ## @@ -53,7 +60,11 @@ public void init() throws IOException { p

Re: [PR] [Improve] check topic mutating SMTs [doris-kafka-connector]

2025-04-23 Thread via GitHub
chuang-wang-pre commented on PR #68: URL: https://github.com/apache/doris-kafka-connector/pull/68#issuecomment-2826254870 > Thank you for your contribution. -> Because in `org.apache.kafka.connect.sink.SinkTask#preCommit`, the topic corresponds to the original topic before any transformati

Re: [PR] Revert "[fix](tgt)hadoop kerberos support renew TGT by keytab (#173)" [doris-thirdparty]

2025-04-23 Thread via GitHub
morningman commented on PR #293: URL: https://github.com/apache/doris-thirdparty/pull/293#issuecomment-2825871957 already merged in #298 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] [Improve] check topic mutating SMTs [doris-kafka-connector]

2025-04-23 Thread via GitHub
JNSimba commented on PR #68: URL: https://github.com/apache/doris-kafka-connector/pull/68#issuecomment-2824409149 Thank you for your contribution. -> Because in `org.apache.kafka.connect.sink.SinkTask#preCommit`, the topic corresponds to the original topic before any transformations are

Re: [PR] [Improve] add behavior on null values [doris-kafka-connector]

2025-04-23 Thread via GitHub
JNSimba commented on PR #69: URL: https://github.com/apache/doris-kafka-connector/pull/69#issuecomment-2824336269 Thanks for your contribution. Can you add an itcase with null value? You can refer to this StringMsgE2ETest -- This is an automated message from the Apache Git Service. To re

Re: [PR] [Improve] add retry strategy [doris-kafka-connector]

2025-04-10 Thread via GitHub
JNSimba commented on code in PR #67: URL: https://github.com/apache/doris-kafka-connector/pull/67#discussion_r2038702350 ## src/main/java/org/apache/doris/kafka/connector/cfg/DorisSinkConnectorConfig.java: ## @@ -89,6 +89,12 @@ public class DorisSinkConnectorConfig { publi

Re: [PR] [Improve] add retry strategy [doris-kafka-connector]

2025-04-10 Thread via GitHub
Copilot commented on code in PR #67: URL: https://github.com/apache/doris-kafka-connector/pull/67#discussion_r2036885245 ## src/test/java/org/apache/doris/kafka/connector/e2e/doris/DorisContainerServiceImpl.java: ## @@ -69,14 +70,23 @@ public GenericContainer createDorisContain

Re: [PR] [Improve] add retry strategy [doris-kafka-connector]

2025-04-10 Thread via GitHub
JNSimba commented on code in PR #67: URL: https://github.com/apache/doris-kafka-connector/pull/67#discussion_r2037090722 ## src/main/java/org/apache/doris/kafka/connector/cfg/DorisSinkConnectorConfig.java: ## @@ -89,6 +89,12 @@ public class DorisSinkConnectorConfig { publi

Re: [PR] [Improve] add retry strategy [doris-kafka-connector]

2025-04-10 Thread via GitHub
JNSimba commented on code in PR #67: URL: https://github.com/apache/doris-kafka-connector/pull/67#discussion_r2037090722 ## src/main/java/org/apache/doris/kafka/connector/cfg/DorisSinkConnectorConfig.java: ## @@ -89,6 +89,12 @@ public class DorisSinkConnectorConfig { publi

Re: [PR] [Improve] add retry strategy [doris-kafka-connector]

2025-04-10 Thread via GitHub
DongLiang-0 commented on code in PR #67: URL: https://github.com/apache/doris-kafka-connector/pull/67#discussion_r2037040321 ## src/main/java/org/apache/doris/kafka/connector/cfg/DorisSinkConnectorConfig.java: ## @@ -89,6 +89,12 @@ public class DorisSinkConnectorConfig { p

Re: [PR] [Improve] add retry strategy [doris-kafka-connector]

2025-04-10 Thread via GitHub
DongLiang-0 commented on code in PR #67: URL: https://github.com/apache/doris-kafka-connector/pull/67#discussion_r2037040321 ## src/main/java/org/apache/doris/kafka/connector/cfg/DorisSinkConnectorConfig.java: ## @@ -89,6 +89,12 @@ public class DorisSinkConnectorConfig { p

Re: [I] 1.0.2 Release Notes [doris-streamloader]

2025-04-06 Thread via GitHub
sollhui closed issue #21: 1.0.2 Release Notes URL: https://github.com/apache/doris-streamloader/issues/21 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail

Re: [PR] [opt](inverted index) Enhance I/O statistics collection for the inverted index in file cache scenarios [doris-thirdparty]

2025-03-16 Thread via GitHub
airborne12 commented on code in PR #295: URL: https://github.com/apache/doris-thirdparty/pull/295#discussion_r1997861820 ## src/core/CLucene/index/MultiReader.cpp: ## @@ -253,14 +253,18 @@ void MultiReader::doSetNorm(int32_t n, const TCHAR* field, uint8_t value){ (*subR

Re: [PR] [opt](inverted index) Enhance I/O statistics collection for the inverted index in file cache scenarios [doris-thirdparty]

2025-03-16 Thread via GitHub
airborne12 commented on code in PR #295: URL: https://github.com/apache/doris-thirdparty/pull/295#discussion_r1997861820 ## src/core/CLucene/index/MultiReader.cpp: ## @@ -253,14 +253,18 @@ void MultiReader::doSetNorm(int32_t n, const TCHAR* field, uint8_t value){ (*subR

Re: [PR] [opt](inverted index) Refactoring Phrase Query Implementation Algorithms with Inverted Index Optimization [doris-thirdparty]

2025-03-02 Thread via GitHub
zzzxl1993 commented on PR #287: URL: https://github.com/apache/doris-thirdparty/pull/287#issuecomment-2693172019 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] [Feature]: Support for Custom Line Break Characters [doris-streamloader]

2025-02-26 Thread via GitHub
cfbber commented on PR #27: URL: https://github.com/apache/doris-streamloader/pull/27#issuecomment-2684246016 @sollhui The conflicts have been resolved, please review the changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

Re: [PR] [Feature]: Support for Custom Line Break Characters [doris-streamloader]

2025-02-23 Thread via GitHub
sollhui commented on PR #27: URL: https://github.com/apache/doris-streamloader/pull/27#issuecomment-2676863165 @cfbber There are some conflicts that must be resolved. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [I] 导入数据到doris2.1.7中出现Status:Success,但LoadedRows:0的问题 [doris-streamloader]

2025-02-16 Thread via GitHub
sollhui closed issue #28: 导入数据到doris2.1.7中出现Status:Success,但LoadedRows:0的问题 URL: https://github.com/apache/doris-streamloader/issues/28 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [PR] [chore](build) Update build.yml [doris-thirdparty]

2025-01-23 Thread via GitHub
airborne12 commented on PR #279: URL: https://github.com/apache/doris-thirdparty/pull/279#issuecomment-2609205114 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [PR] [chore](build) Update build.yml [doris-thirdparty]

2025-01-23 Thread via GitHub
airborne12 commented on PR #279: URL: https://github.com/apache/doris-thirdparty/pull/279#issuecomment-2609173029 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [PR] [fix](bkd) make ram_bytes_used for bkd reader public (#273) [doris-thirdparty]

2025-01-23 Thread via GitHub
airborne12 commented on PR #278: URL: https://github.com/apache/doris-thirdparty/pull/278#issuecomment-2609162911 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [PR] [Chore](clucene) reorder pr [doris-thirdparty]

2025-01-20 Thread via GitHub
airborne12 commented on PR #274: URL: https://github.com/apache/doris-thirdparty/pull/274#issuecomment-2603608293 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [PR] [fix](bkd) make ram_bytes_used for bkd reader public [doris-thirdparty]

2025-01-20 Thread via GitHub
airborne12 commented on PR #273: URL: https://github.com/apache/doris-thirdparty/pull/273#issuecomment-2603583523 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [PR] [feature](analysis) add new chinese tokenizer IK [doris-thirdparty]

2025-01-17 Thread via GitHub
zzzxl1993 commented on code in PR #269: URL: https://github.com/apache/doris-thirdparty/pull/269#discussion_r1903616931 ## src/contribs-lib/CLucene/analysis/LanguageBasedAnalyzer.h: ## @@ -7,14 +7,18 @@ #ifndef _lucene_analysis_languagebasedanalyzer_ #define _lucene_analysis_l

Re: [I] 导入数据到doris2.1.7中出现Status:Success,但LoadedRows:0的问题 [doris-streamloader]

2025-01-02 Thread via GitHub
sollhui commented on issue #28: URL: https://github.com/apache/doris-streamloader/issues/28#issuecomment-2567808583 This 1.0.2 version has a bug and requires setting concurrency, please add `-- workers = n` -- This is an automated message from the Apache Git Service. To respond to the m

[I] 导入数据到doris2.1.7中出现Status:Success,但LoadedRows:0的问题 [doris-streamloader]

2025-01-02 Thread via GitHub
18859108815 opened a new issue, #28: URL: https://github.com/apache/doris-streamloader/issues/28 doris-streamloader版本1.0.2在导入数据到doris2.1.7中出现Status:Success,但LoadedRows:0的问题,这个工具的release版本是2024年8月22日,doris2.1.7版本的release时间是2024年11月10号,工具和doris不同版本是否存在兼容性问题。 [](./doris-streamloader --source

Re: [PR] [improve]support string type record in one topic to multiple tables [doris-kafka-connector]

2024-12-29 Thread via GitHub
DongLiang-0 commented on PR #59: URL: https://github.com/apache/doris-kafka-connector/pull/59#issuecomment-2565103032 Hi @chuang-wang-pre , I noticed that string type records are not supported and need to be strongly bound to the converter of `org.apache.kafka.connect.json.JsonConverter`,

Re: [PR] [improve] Support writing data from one topic to multiple tables [doris-kafka-connector]

2024-12-26 Thread via GitHub
DongLiang-0 commented on PR #55: URL: https://github.com/apache/doris-kafka-connector/pull/55#issuecomment-2562285705 > > https://github.com/apache/doris-kafka-connector?tab=readme-ov-file#code-formatting > > @DongLiang-0 I noticed this and just ran it again without any changes to t

Re: [PR] [improve] Support writing data from one topic to multiple tables [doris-kafka-connector]

2024-12-24 Thread via GitHub
chuang-wang-pre commented on PR #55: URL: https://github.com/apache/doris-kafka-connector/pull/55#issuecomment-2561574640 > https://github.com/apache/doris-kafka-connector?tab=readme-ov-file#code-formatting @DongLiang-0 I noticed this and just ran it again without any changes to the

Re: [PR] [improve] Support writing data from one topic to multiple tables [doris-kafka-connector]

2024-12-24 Thread via GitHub
DongLiang-0 commented on PR #55: URL: https://github.com/apache/doris-kafka-connector/pull/55#issuecomment-2561571176 It looks like a code needs to be formatted. refer to: https://github.com/apache/doris-kafka-connector?tab=readme-ov-file#code-formatting -- This is an automated messa

Re: [PR] [improve] Support writing data from one topic to multiple tables by c… [doris-kafka-connector]

2024-12-23 Thread via GitHub
chuang-wang-pre commented on code in PR #55: URL: https://github.com/apache/doris-kafka-connector/pull/55#discussion_r1895707494 ## src/main/java/org/apache/doris/kafka/connector/service/DorisDefaultSinkService.java: ## @@ -161,12 +177,66 @@ public void commit(Map offsets) {

Re: [PR] [improve] Support writing data from one topic to multiple tables by c… [doris-kafka-connector]

2024-12-23 Thread via GitHub
DongLiang-0 commented on code in PR #55: URL: https://github.com/apache/doris-kafka-connector/pull/55#discussion_r1895578856 ## src/main/java/org/apache/doris/kafka/connector/cfg/DorisSinkConnectorConfig.java: ## @@ -52,6 +52,7 @@ public class DorisSinkConnectorConfig { pu

Re: [PR] [improve] Support writing data from one topic to multiple tables by c… [doris-kafka-connector]

2024-12-23 Thread via GitHub
DongLiang-0 commented on code in PR #55: URL: https://github.com/apache/doris-kafka-connector/pull/55#discussion_r1895527089 ## src/main/java/org/apache/doris/kafka/connector/service/DorisDefaultSinkService.java: ## @@ -161,12 +177,66 @@ public void commit(Map offsets) {

Re: [PR] [improve] Support writing data from one topic to multiple tables by c… [doris-kafka-connector]

2024-12-23 Thread via GitHub
chuang-wang-pre commented on PR #55: URL: https://github.com/apache/doris-kafka-connector/pull/55#issuecomment-2559134920 > Please add an e2e test case, please refer to `org.apache.doris.kafka.connector.e2e.sink.stringconverter.StringMsgE2ETest` @DongLiang-0 Thank you for the reminde

Re: [PR] [improve] Support writing data from one topic to multiple tables by c… [doris-kafka-connector]

2024-12-20 Thread via GitHub
chuang-wang-pre commented on PR #55: URL: https://github.com/apache/doris-kafka-connector/pull/55#issuecomment-2556784611 > @chuang-wang-pre Thank you for your contribution. But I would like to ask, in an actual production environment, under what circumstances would there be multiple table

Re: [PR] [improve] Support writing data from one topic to multiple tables by c… [doris-kafka-connector]

2024-12-20 Thread via GitHub
DongLiang-0 commented on PR #55: URL: https://github.com/apache/doris-kafka-connector/pull/55#issuecomment-2556550651 Thank you for your contribution. But I would like to ask, in an actual production environment, under what circumstances would there be multiple tables in the same topic?

Re: [PR] [fix](inverted index) Fix the issue with incorrect seek results in DICT_COMPRESS [doris-thirdparty]

2024-12-16 Thread via GitHub
zzzxl1993 commented on PR #262: URL: https://github.com/apache/doris-thirdparty/pull/262#issuecomment-2545190443 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Bump io.debezium:debezium-core from 1.9.8.Final to 2.3.0.Final [doris-kafka-connector]

2024-11-20 Thread via GitHub
dependabot[bot] commented on PR #52: URL: https://github.com/apache/doris-kafka-connector/pull/52#issuecomment-2489912285 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor v

Re: [PR] Bump commons-io:commons-io from 2.3 to 2.14.0 [doris-kafka-connector]

2024-11-20 Thread via GitHub
dependabot[bot] commented on PR #48: URL: https://github.com/apache/doris-kafka-connector/pull/48#issuecomment-2489912499 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor v

Re: [PR] Bump org.apache.avro:avro from 1.11.3 to 1.11.4 [doris-kafka-connector]

2024-11-20 Thread via GitHub
dependabot[bot] commented on PR #49: URL: https://github.com/apache/doris-kafka-connector/pull/49#issuecomment-2489912106 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor v

Re: [PR] [opt](inverted index) add performance profiling for remote io access in inverted index [doris-thirdparty]

2024-11-11 Thread via GitHub
zzzxl1993 commented on code in PR #250: URL: https://github.com/apache/doris-thirdparty/pull/250#discussion_r1837417453 ## src/core/CLucene/index/SegmentTermPositions.cpp.rej: ## @@ -0,0 +1,9 @@ +diff a/src/core/CLucene/index/SegmentTermPositions.cpp b/src/core/CLucene/index/Se

Re: [PR] [opt](inverted index) add performance profiling for remote io access in inverted index [doris-thirdparty]

2024-11-11 Thread via GitHub
airborne12 commented on code in PR #250: URL: https://github.com/apache/doris-thirdparty/pull/250#discussion_r1837393472 ## src/core/CLucene/index/SegmentTermPositions.cpp.rej: ## @@ -0,0 +1,9 @@ +diff a/src/core/CLucene/index/SegmentTermPositions.cpp b/src/core/CLucene/index/S

Re: [PR] [opt](inverted index) Inverted Index Dictionary Compression [doris-thirdparty]

2024-11-05 Thread via GitHub
zzzxl1993 commented on code in PR #245: URL: https://github.com/apache/doris-thirdparty/pull/245#discussion_r1830493671 ## src/core/CLucene/index/IndexWriter.cpp: ## @@ -1879,7 +1884,8 @@ void IndexWriter::mergeTerms(bool hasProx, IndexVersion indexVersion) { TermI

Re: [PR] [opt](inverted index) Inverted Index Dictionary Compression [doris-thirdparty]

2024-11-05 Thread via GitHub
zzzxl1993 commented on code in PR #245: URL: https://github.com/apache/doris-thirdparty/pull/245#discussion_r1829009376 ## src/test/index/TestIndexCompressV3.cpp: ## @@ -0,0 +1,325 @@ +#include // IWYU pragma: keep +#include +#include +#include + +#include +#include +#incl

Re: [PR] [opt](inverted index) Inverted Index Dictionary Compression [doris-thirdparty]

2024-11-05 Thread via GitHub
airborne12 commented on code in PR #245: URL: https://github.com/apache/doris-thirdparty/pull/245#discussion_r1828970052 ## src/core/CLucene/index/IndexWriter.cpp: ## @@ -1879,7 +1884,8 @@ void IndexWriter::mergeTerms(bool hasProx, IndexVersion indexVersion) { Term

Re: [PR] [opt](inverted index) Inverted Index Dictionary Compression [doris-thirdparty]

2024-11-05 Thread via GitHub
airborne12 commented on code in PR #245: URL: https://github.com/apache/doris-thirdparty/pull/245#discussion_r1828964829 ## src/core/CLucene/index/FieldInfos.cpp: ## @@ -114,6 +118,17 @@ IndexVersion FieldInfos::getIndexVersion() { return IndexVersion::kV0; } +uint32_

Re: [PR] [opt](inverted index) Inverted Index Dictionary Compression [doris-thirdparty]

2024-11-04 Thread via GitHub
zzzxl1993 commented on code in PR #245: URL: https://github.com/apache/doris-thirdparty/pull/245#discussion_r1827675233 ## src/core/CLucene/store/v2/GrowableByteArrayDataOutput.h: ## @@ -0,0 +1,108 @@ +#pragma once + +#include +#include +#include +#include +#include + +#inc

Re: [PR] [opt](inverted index) Inverted Index Dictionary Compression [doris-thirdparty]

2024-11-04 Thread via GitHub
zzzxl1993 commented on code in PR #245: URL: https://github.com/apache/doris-thirdparty/pull/245#discussion_r1827657172 ## src/core/CLucene/store/v2/ByteArrayDataInput.h: ## @@ -0,0 +1,126 @@ +#pragma once + +#include + +#include +#include +#include +#include + +#include "C

Re: [PR] [opt](inverted index) Inverted Index Dictionary Compression [doris-thirdparty]

2024-11-04 Thread via GitHub
zzzxl1993 commented on code in PR #245: URL: https://github.com/apache/doris-thirdparty/pull/245#discussion_r1827644180 ## CMakeLists.txt: ## @@ -17,7 +17,7 @@ MATH(EXPR CLUCENE_INT_VERSION "(${CLUCENE_VERSION_MAJOR} * 100) + (${CLUCENE SET(CLUCENE_VERSION "${CLUCENE_VERS

Re: [PR] [opt](inverted index) Inverted Index Dictionary Compression [doris-thirdparty]

2024-11-04 Thread via GitHub
zzzxl1993 commented on code in PR #245: URL: https://github.com/apache/doris-thirdparty/pull/245#discussion_r1827643630 ## CMakeLists.txt: ## @@ -17,7 +17,7 @@ MATH(EXPR CLUCENE_INT_VERSION "(${CLUCENE_VERSION_MAJOR} * 100) + (${CLUCENE SET(CLUCENE_VERSION "${CLUCENE_VERS

Re: [PR] [opt](inverted index) Inverted Index Dictionary Compression [doris-thirdparty]

2024-11-04 Thread via GitHub
zzzxl1993 commented on code in PR #245: URL: https://github.com/apache/doris-thirdparty/pull/245#discussion_r1827641782 ## src/core/CLucene/store/v2/ByteArrayDataInput.h: ## @@ -0,0 +1,126 @@ +#pragma once + +#include + +#include +#include +#include +#include + +#include "C

Re: [PR] [opt](inverted index) Inverted Index Dictionary Compression [doris-thirdparty]

2024-11-04 Thread via GitHub
zzzxl1993 commented on code in PR #245: URL: https://github.com/apache/doris-thirdparty/pull/245#discussion_r1827639636 ## src/core/CLucene/store/v2/GrowableByteArrayDataOutput.h: ## @@ -0,0 +1,108 @@ +#pragma once + +#include +#include +#include +#include +#include + +#inc

Re: [PR] [opt](inverted index) Inverted Index Dictionary Compression [doris-thirdparty]

2024-11-04 Thread via GitHub
xiaokang commented on code in PR #245: URL: https://github.com/apache/doris-thirdparty/pull/245#discussion_r1827601054 ## src/core/CLucene/store/v2/GrowableByteArrayDataOutput.h: ## @@ -0,0 +1,108 @@ +#pragma once + +#include +#include +#include +#include +#include + +#incl

Re: [PR] [opt](inverted index) Inverted Index Dictionary Compression [doris-thirdparty]

2024-11-04 Thread via GitHub
zzzxl1993 commented on code in PR #245: URL: https://github.com/apache/doris-thirdparty/pull/245#discussion_r1827566925 ## src/core/CLucene/index/FieldInfos.cpp: ## @@ -257,6 +274,9 @@ void FieldInfos::write(IndexOutput* output) const{ if (fi->getIndexVersion() >

Re: [PR] [opt](inverted index) Inverted Index Dictionary Compression [doris-thirdparty]

2024-11-04 Thread via GitHub
zzzxl1993 commented on code in PR #245: URL: https://github.com/apache/doris-thirdparty/pull/245#discussion_r1827565459 ## src/core/CLucene/index/FieldInfos.cpp: ## @@ -114,6 +118,17 @@ IndexVersion FieldInfos::getIndexVersion() { return IndexVersion::kV0; } +uint32_t

Re: [PR] [opt](inverted index) Inverted Index Dictionary Compression [doris-thirdparty]

2024-11-04 Thread via GitHub
zzzxl1993 commented on code in PR #245: URL: https://github.com/apache/doris-thirdparty/pull/245#discussion_r1827554420 ## src/core/CLucene/index/FieldConfig.h: ## @@ -0,0 +1,11 @@ +#pragma once + +#include + +enum class FlagBits : uint32_t { +DICT_COMPRESS = 1 << 0, // 000

Re: [PR] [opt](inverted index) Inverted Index Dictionary Compression [doris-thirdparty]

2024-11-04 Thread via GitHub
zzzxl1993 commented on code in PR #245: URL: https://github.com/apache/doris-thirdparty/pull/245#discussion_r1827547272 ## src/core/CLucene/index/IndexWriter.cpp: ## @@ -1879,7 +1884,8 @@ void IndexWriter::mergeTerms(bool hasProx, IndexVersion indexVersion) { TermI

Re: [PR] [opt](inverted index) Inverted Index Dictionary Compression [doris-thirdparty]

2024-11-04 Thread via GitHub
zzzxl1993 commented on code in PR #245: URL: https://github.com/apache/doris-thirdparty/pull/245#discussion_r1827544734 ## src/core/CLucene/index/TermInfosWriter.cpp: ## @@ -85,6 +87,16 @@ void STermInfosWriter::add(STerm *term, TermInfo *ti) { add(fieldInfos->fieldNumber(t

Re: [PR] [opt](inverted index) Inverted Index Dictionary Compression [doris-thirdparty]

2024-11-04 Thread via GitHub
zzzxl1993 commented on code in PR #245: URL: https://github.com/apache/doris-thirdparty/pull/245#discussion_r1827539630 ## cmake/FindZstd.cmake: ## @@ -0,0 +1,68 @@ +# FindZstd.cmake +# Try to find or build the zstd library + +if(NOT TARGET zstd) + message(STATUS "Zstd target n

Re: [PR] [opt](inverted index) Inverted Index Dictionary Compression [doris-thirdparty]

2024-11-04 Thread via GitHub
xiaokang commented on code in PR #245: URL: https://github.com/apache/doris-thirdparty/pull/245#discussion_r1827420549 ## src/core/CLucene/index/FieldConfig.h: ## @@ -0,0 +1,11 @@ +#pragma once + +#include + +enum class FlagBits : uint32_t { +DICT_COMPRESS = 1 << 0, //

Re: [PR] [opt](inverted index) Inverted Index Dictionary Compression [doris-thirdparty]

2024-11-03 Thread via GitHub
airborne12 commented on code in PR #245: URL: https://github.com/apache/doris-thirdparty/pull/245#discussion_r1822156436 ## cmake/FindZstd.cmake: ## @@ -0,0 +1,68 @@ +# FindZstd.cmake +# Try to find or build the zstd library + +if(NOT TARGET zstd) + message(STATUS "Zstd target

Re: [PR] [fix](inverted index) Fix incorrect exception handling (#243) [doris-thirdparty]

2024-10-24 Thread via GitHub
zzzxl1993 commented on PR #246: URL: https://github.com/apache/doris-thirdparty/pull/246#issuecomment-2436668626 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] [opt](inverted index) Inverted Index Dictionary Compression [doris-thirdparty]

2024-10-21 Thread via GitHub
zzzxl1993 commented on PR #245: URL: https://github.com/apache/doris-thirdparty/pull/245#issuecomment-2428183782 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [PR] [opt](inverted index) Optimize the compression of inverted index position information [doris-thirdparty]

2024-10-17 Thread via GitHub
zzzxl1993 commented on PR #242: URL: https://github.com/apache/doris-thirdparty/pull/242#issuecomment-2419371701 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] [fix]avoid label conflict when setting labels in headers. [doris-streamloader]

2024-10-16 Thread via GitHub
sollhui commented on PR #25: URL: https://github.com/apache/doris-streamloader/pull/25#issuecomment-2415896298 @vinlee19 Please add some description about what you fix. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] [opt](inverted index) Optimize the compression of inverted index position information [doris-thirdparty]

2024-10-14 Thread via GitHub
zzzxl1993 commented on code in PR #242: URL: https://github.com/apache/doris-thirdparty/pull/242#discussion_r1799010998 ## src/core/CLucene/index/IndexWriter.cpp: ## @@ -1308,20 +1309,36 @@ void IndexWriter::indexCompaction(std::vector &src_d } assert(readers.size() =

Re: [PR] [opt](inverted index) Optimize the compression of inverted index position information [doris-thirdparty]

2024-10-14 Thread via GitHub
zzzxl1993 commented on code in PR #242: URL: https://github.com/apache/doris-thirdparty/pull/242#discussion_r1799008204 ## src/core/CLucene/util/PFORUtil.h: ## @@ -18,9 +18,22 @@ #include #include +#include + +#include "CLucene.h" +#include "CLucene/store/IndexOutput.h" +

Re: [PR] [opt](inverted index) Optimize the compression of inverted index position information [doris-thirdparty]

2024-10-14 Thread via GitHub
zzzxl1993 commented on PR #242: URL: https://github.com/apache/doris-thirdparty/pull/242#issuecomment-2410476697 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] [fix](inverted index) Fix incorrect exception handling [doris-thirdparty]

2024-10-13 Thread via GitHub
zzzxl1993 commented on PR #243: URL: https://github.com/apache/doris-thirdparty/pull/243#issuecomment-2410193399 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] [opt](inverted index) Optimize the compression of inverted index position information [doris-thirdparty]

2024-10-12 Thread via GitHub
airborne12 commented on code in PR #242: URL: https://github.com/apache/doris-thirdparty/pull/242#discussion_r1797694674 ## src/core/CLucene/index/_SegmentHeader.h: ## @@ -86,6 +87,62 @@ class TermDocsBuffer { IndexVersion indexVersion_ = IndexVersion::kV0; }; +class Term

Re: [PR] [opt](inverted index) Optimize the compression of inverted index position information [doris-thirdparty]

2024-10-12 Thread via GitHub
zzzxl1993 commented on PR #242: URL: https://github.com/apache/doris-thirdparty/pull/242#issuecomment-2408459618 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] [Feature] Remote URL Support [doris-streamloader]

2024-10-11 Thread via GitHub
bokket commented on issue #26: URL: https://github.com/apache/doris-streamloader/issues/26#issuecomment-2408331344 Sorry for the ambiguity:Remote Local Files.Another location is optional,OpenDAL may be useful.But I'm not sure if their GO package is available.If I have time, I will do it.

Re: [I] [Feature] Remote URL Support [doris-streamloader]

2024-10-11 Thread via GitHub
liaoxin01 commented on issue #26: URL: https://github.com/apache/doris-streamloader/issues/26#issuecomment-2408303166 Where is the remote file? If it's on S3 or HDFS, broker Load can be used directly. If it's in another location, you are welcome to contribute to this feature. -- This is

[I] [Feature] Remote URL Support [doris-streamloader]

2024-10-10 Thread via GitHub
bokket opened a new issue, #26: URL: https://github.com/apache/doris-streamloader/issues/26 Hi.I didn't find any parameter about remote host file transfer, this requirement is somewhat helpful for my work, is there any plan for development? -- This is an automated message from the Apache

Re: [PR] [opt](inverted index) optimize the space usage of the inverted index dictionary file and position information [doris-thirdparty]

2024-09-21 Thread via GitHub
zzzxl1993 commented on PR #238: URL: https://github.com/apache/doris-thirdparty/pull/238#issuecomment-2365192205 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] fix CLStream memcpy read 1 extra byte [doris-thirdparty]

2024-09-14 Thread via GitHub
xiaokang commented on code in PR #240: URL: https://github.com/apache/doris-thirdparty/pull/240#discussion_r1759963263 ## src/core/CLucene/util/CLStreams.h: ## @@ -207,7 +207,8 @@ class CLUCENE_EXPORT SStringReader : public Reader { tmp = (T *) realloc(tmp, size

Re: [PR] fix CLStream memcpy read 1 extra byte [doris-thirdparty]

2024-09-14 Thread via GitHub
xiaokang commented on code in PR #240: URL: https://github.com/apache/doris-thirdparty/pull/240#discussion_r1759963263 ## src/core/CLucene/util/CLStreams.h: ## @@ -207,7 +207,8 @@ class CLUCENE_EXPORT SStringReader : public Reader { tmp = (T *) realloc(tmp, size

Re: [PR] fix CLStream memcpy read 1 extra byte [doris-thirdparty]

2024-09-14 Thread via GitHub
airborne12 commented on code in PR #240: URL: https://github.com/apache/doris-thirdparty/pull/240#discussion_r1759953844 ## src/core/CLucene/util/CLStreams.h: ## @@ -207,7 +207,8 @@ class CLUCENE_EXPORT SStringReader : public Reader { tmp = (T *) realloc(tmp, si

Re: [PR] [fix](inverted index) Handle an empty segment index file [doris-thirdparty]

2024-09-01 Thread via GitHub
zzzxl1993 commented on PR #239: URL: https://github.com/apache/doris-thirdparty/pull/239#issuecomment-2323953294 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] [opt](inverted index) optimize the space usage of the inverted index dictionary file and position information [doris-thirdparty]

2024-08-21 Thread via GitHub
zzzxl1993 commented on PR #238: URL: https://github.com/apache/doris-thirdparty/pull/238#issuecomment-2303615386 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

  1   2   3   4   5   >