Re: [PR] [opt](inverted index) Inverted Index Dictionary Compression [doris-thirdparty]

zzzxl1993 commented on code in PR #245: URL: https://github.com/apache/doris-thirdparty/pull/245#discussion_r1827675233 ## src/core/CLucene/store/v2/GrowableByteArrayDataOutput.h: ## @@ -0,0 +1,108 @@ +#pragma once + +#include +#include +#include +#include +#include + +#inc

Re: [PR] [opt](inverted index) Inverted Index Dictionary Compression [doris-thirdparty]

zzzxl1993 commented on code in PR #245: URL: https://github.com/apache/doris-thirdparty/pull/245#discussion_r1827657172 ## src/core/CLucene/store/v2/ByteArrayDataInput.h: ## @@ -0,0 +1,126 @@ +#pragma once + +#include + +#include +#include +#include +#include + +#include "C

Re: [PR] [opt](inverted index) Inverted Index Dictionary Compression [doris-thirdparty]

zzzxl1993 commented on code in PR #245: URL: https://github.com/apache/doris-thirdparty/pull/245#discussion_r1827644180 ## CMakeLists.txt: ## @@ -17,7 +17,7 @@ MATH(EXPR CLUCENE_INT_VERSION "(${CLUCENE_VERSION_MAJOR} * 100) + (${CLUCENE SET(CLUCENE_VERSION "${CLUCENE_VERS

Re: [PR] [opt](inverted index) Inverted Index Dictionary Compression [doris-thirdparty]

zzzxl1993 commented on code in PR #245: URL: https://github.com/apache/doris-thirdparty/pull/245#discussion_r1827643630 ## CMakeLists.txt: ## @@ -17,7 +17,7 @@ MATH(EXPR CLUCENE_INT_VERSION "(${CLUCENE_VERSION_MAJOR} * 100) + (${CLUCENE SET(CLUCENE_VERSION "${CLUCENE_VERS

Re: [PR] [opt](inverted index) Inverted Index Dictionary Compression [doris-thirdparty]

zzzxl1993 commented on code in PR #245: URL: https://github.com/apache/doris-thirdparty/pull/245#discussion_r1827641782 ## src/core/CLucene/store/v2/ByteArrayDataInput.h: ## @@ -0,0 +1,126 @@ +#pragma once + +#include + +#include +#include +#include +#include + +#include "C

Re: [PR] [opt](inverted index) Inverted Index Dictionary Compression [doris-thirdparty]

zzzxl1993 commented on code in PR #245: URL: https://github.com/apache/doris-thirdparty/pull/245#discussion_r1827639636 ## src/core/CLucene/store/v2/GrowableByteArrayDataOutput.h: ## @@ -0,0 +1,108 @@ +#pragma once + +#include +#include +#include +#include +#include + +#inc

Re: [PR] [opt](inverted index) Inverted Index Dictionary Compression [doris-thirdparty]

xiaokang commented on code in PR #245: URL: https://github.com/apache/doris-thirdparty/pull/245#discussion_r1827601054 ## src/core/CLucene/store/v2/GrowableByteArrayDataOutput.h: ## @@ -0,0 +1,108 @@ +#pragma once + +#include +#include +#include +#include +#include + +#incl

Re: [PR] [opt](inverted index) Inverted Index Dictionary Compression [doris-thirdparty]

zzzxl1993 commented on code in PR #245: URL: https://github.com/apache/doris-thirdparty/pull/245#discussion_r1827566925 ## src/core/CLucene/index/FieldInfos.cpp: ## @@ -257,6 +274,9 @@ void FieldInfos::write(IndexOutput* output) const{ if (fi->getIndexVersion() >

Re: [PR] [opt](inverted index) Inverted Index Dictionary Compression [doris-thirdparty]

zzzxl1993 commented on code in PR #245: URL: https://github.com/apache/doris-thirdparty/pull/245#discussion_r1827565459 ## src/core/CLucene/index/FieldInfos.cpp: ## @@ -114,6 +118,17 @@ IndexVersion FieldInfos::getIndexVersion() { return IndexVersion::kV0; } +uint32_t

Re: [PR] [opt](inverted index) Inverted Index Dictionary Compression [doris-thirdparty]

zzzxl1993 commented on code in PR #245: URL: https://github.com/apache/doris-thirdparty/pull/245#discussion_r1827554420 ## src/core/CLucene/index/FieldConfig.h: ## @@ -0,0 +1,11 @@ +#pragma once + +#include + +enum class FlagBits : uint32_t { +DICT_COMPRESS = 1 << 0, // 000

Re: [PR] [opt](inverted index) Inverted Index Dictionary Compression [doris-thirdparty]

zzzxl1993 commented on code in PR #245: URL: https://github.com/apache/doris-thirdparty/pull/245#discussion_r1827547272 ## src/core/CLucene/index/IndexWriter.cpp: ## @@ -1879,7 +1884,8 @@ void IndexWriter::mergeTerms(bool hasProx, IndexVersion indexVersion) { TermI

[Result] [VOTE] [ROUND2] Release Apache Doris Operator 24.0.0

Hi all, the vote is passed and closed[1]. There are 4 (binding) votes: morningman +1 (binding) diwu +1 (binding) kxiao +1 (binding) lihaopeng +1 (binding) Thank you for your review. [1] https://lists.apache.org/thread/tk15pkl9jc43yo42rdnz141dmgpczbty -- Best wishes！ Yongkang Zhong

Re: [PR] [opt](inverted index) Inverted Index Dictionary Compression [doris-thirdparty]

zzzxl1993 commented on code in PR #245: URL: https://github.com/apache/doris-thirdparty/pull/245#discussion_r1827544734 ## src/core/CLucene/index/TermInfosWriter.cpp: ## @@ -85,6 +87,16 @@ void STermInfosWriter::add(STerm *term, TermInfo *ti) { add(fieldInfos->fieldNumber(t

Re: [PR] [opt](inverted index) Inverted Index Dictionary Compression [doris-thirdparty]

zzzxl1993 commented on code in PR #245: URL: https://github.com/apache/doris-thirdparty/pull/245#discussion_r1827539630 ## cmake/FindZstd.cmake: ## @@ -0,0 +1,68 @@ +# FindZstd.cmake +# Try to find or build the zstd library + +if(NOT TARGET zstd) + message(STATUS "Zstd target n

Re: [PR] [opt](inverted index) Inverted Index Dictionary Compression [doris-thirdparty]

xiaokang commented on code in PR #245: URL: https://github.com/apache/doris-thirdparty/pull/245#discussion_r1827420549 ## src/core/CLucene/index/FieldConfig.h: ## @@ -0,0 +1,11 @@ +#pragma once + +#include + +enum class FlagBits : uint32_t { +DICT_COMPRESS = 1 << 0, //