Re: [PR] [Improve] Estimated column reader memory to control segment cache [doris]

2024-07-10 Thread via GitHub
dataroaring closed pull request #37597: [Improve] Estimated column reader memory to control segment cache URL: https://github.com/apache/doris/pull/37597 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] [Improve] Estimated column reader memory to control segment cache [doris]

2024-07-10 Thread via GitHub
doris-robot commented on PR #37597: URL: https://github.com/apache/doris/pull/37597#issuecomment-2220075157 Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' ``` Load test result on commit 1aeff68efc9a98a8c9feb11736bfa44ede739476 with default session variables Stream

Re: [PR] [Improve] Estimated column reader memory to control segment cache [doris]

2024-07-10 Thread via GitHub
doris-robot commented on PR #37597: URL: https://github.com/apache/doris/pull/37597#issuecomment-2220068379 ClickBench: Total hot run time: 31.05 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools

Re: [PR] [Improve] Estimated column reader memory to control segment cache [doris]

2024-07-10 Thread via GitHub
doris-robot commented on PR #37597: URL: https://github.com/apache/doris/pull/37597#issuecomment-2220057619 TPC-DS: Total hot run time: 205563 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS

Re: [PR] [Improve] Estimated column reader memory to control segment cache [doris]

2024-07-10 Thread via GitHub
doris-robot commented on PR #37597: URL: https://github.com/apache/doris/pull/37597#issuecomment-2220033023 TPC-H: Total hot run time: 50232 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100

Re: [PR] [Improve] Estimated column reader memory to control segment cache [doris]

2024-07-10 Thread via GitHub
github-actions[bot] commented on PR #37597: URL: https://github.com/apache/doris/pull/37597#issuecomment-2219916272 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] [Improve] Estimated column reader memory to control segment cache [doris]

2024-07-10 Thread via GitHub
Lchangliang commented on PR #37597: URL: https://github.com/apache/doris/pull/37597#issuecomment-2219901833 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] [Improve] Estimated column reader memory to control segment cache [doris]

2024-07-10 Thread via GitHub
github-actions[bot] commented on PR #37597: URL: https://github.com/apache/doris/pull/37597#issuecomment-2219897355 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] [Improve] Estimated column reader memory to control segment cache [doris]

2024-07-10 Thread via GitHub
github-actions[bot] commented on PR #37597: URL: https://github.com/apache/doris/pull/37597#issuecomment-2219793011 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] [Improve] Estimated column reader memory to control segment cache [doris]

2024-07-10 Thread via GitHub
Lchangliang commented on PR #37597: URL: https://github.com/apache/doris/pull/37597#issuecomment-2219780603 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] [Improve] Estimated column reader memory to control segment cache [doris]

2024-07-10 Thread via GitHub
doris-robot commented on PR #37597: URL: https://github.com/apache/doris/pull/37597#issuecomment-2219780561 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+y

Re: [PR] [Improve] Estimated column reader memory to control segment cache [doris]

2024-05-08 Thread via GitHub
dataroaring merged PR #34526: URL: https://github.com/apache/doris/pull/34526 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.

Re: [PR] [Improve] Estimated column reader memory to control segment cache [doris]

2024-05-08 Thread via GitHub
xinyiZzz commented on code in PR #34526: URL: https://github.com/apache/doris/pull/34526#discussion_r1593964105 ## be/src/common/config.cpp: ## @@ -1053,6 +1053,10 @@ DEFINE_mInt32(schema_cache_sweep_time_sec, "100"); // max number of segment cache, default -1 for backward co

Re: [PR] [Improve] Estimated column reader memory to control segment cache [doris]

2024-05-08 Thread via GitHub
github-actions[bot] commented on PR #34526: URL: https://github.com/apache/doris/pull/34526#issuecomment-2100425644 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] [Improve] Estimated column reader memory to control segment cache [doris]

2024-05-08 Thread via GitHub
github-actions[bot] commented on PR #34526: URL: https://github.com/apache/doris/pull/34526#issuecomment-2100425578 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

Re: [PR] [Improve] Estimated column reader memory to control segment cache [doris]

2024-05-08 Thread via GitHub
github-actions[bot] commented on PR #34526: URL: https://github.com/apache/doris/pull/34526#issuecomment-2100056418 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] [Improve] Estimated column reader memory to control segment cache [doris]

2024-05-08 Thread via GitHub
Lchangliang commented on PR #34526: URL: https://github.com/apache/doris/pull/34526#issuecomment-2100045752 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] [Improve] Estimated column reader memory to control segment cache [doris]

2024-05-08 Thread via GitHub
Lchangliang commented on code in PR #34526: URL: https://github.com/apache/doris/pull/34526#discussion_r1593541497 ## be/src/olap/rowset/segment_v2/segment.cpp: ## @@ -408,6 +408,7 @@ Status Segment::_create_column_readers(const SegmentFooterPB& footer) { RETURN_IF_ERR

Re: [PR] [Improve] Estimated column reader memory to control segment cache [doris]

2024-05-08 Thread via GitHub
xinyiZzz commented on code in PR #34526: URL: https://github.com/apache/doris/pull/34526#discussion_r1593531683 ## be/src/olap/rowset/segment_v2/segment.cpp: ## @@ -408,6 +408,7 @@ Status Segment::_create_column_readers(const SegmentFooterPB& footer) { RETURN_IF_ERROR(

Re: [PR] [Improve] Estimated column reader memory to control segment cache [doris]

2024-05-08 Thread via GitHub
xinyiZzz commented on PR #34526: URL: https://github.com/apache/doris/pull/34526#issuecomment-2099935198 ![image](https://github.com/apache/doris/assets/13197424/fc167873-4256-4bdb-8407-b9965e75daee) ``` auto* lru_handle = LRUCachePolicy::insert( key.encode(), &value

Re: [PR] [Improve] Estimated column reader memory to control segment cache [doris]

2024-05-08 Thread via GitHub
github-actions[bot] commented on PR #34526: URL: https://github.com/apache/doris/pull/34526#issuecomment-2099934088 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] [Improve] Estimated column reader memory to control segment cache [doris]

2024-05-08 Thread via GitHub
Lchangliang commented on PR #34526: URL: https://github.com/apache/doris/pull/34526#issuecomment-2099924079 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] [Improve] Estimated column reader memory to control segment cache [doris]

2024-05-08 Thread via GitHub
doris-robot commented on PR #34526: URL: https://github.com/apache/doris/pull/34526#issuecomment-2099923883 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+y

[PR] [Improve] Estimated column reader memory to control segment cache [doris]

2024-05-08 Thread via GitHub
Lchangliang opened a new pull request, #34526: URL: https://github.com/apache/doris/pull/34526 ## Proposed changes Issue Number: close #xxx If a segment has many columns, it will use a lot more memory than a segment with few columns, but it is not observable at present. So