Re: [PR] [enhance](S3) Use allocator for s3 buffer's allocation [doris]

2025-01-28 Thread via GitHub
github-actions[bot] closed pull request #38519: [enhance](S3) Use allocator for s3 buffer's allocation URL: https://github.com/apache/doris/pull/38519 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] [enhance](S3) Use allocator for s3 buffer's allocation [doris]

2025-01-27 Thread via GitHub
github-actions[bot] commented on PR #38519: URL: https://github.com/apache/doris/pull/38519#issuecomment-2617236500 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable

Re: [PR] [enhance](S3) Use allocator for s3 buffer's allocation [doris]

2024-07-30 Thread via GitHub
doris-robot commented on PR #38519: URL: https://github.com/apache/doris/pull/38519#issuecomment-2259739781 Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' ``` Load test result on commit 397818ae14a60fab13ce2e8e9a3d982a0199aa2c with default session variables Stream

Re: [PR] [enhance](S3) Use allocator for s3 buffer's allocation [doris]

2024-07-30 Thread via GitHub
doris-robot commented on PR #38519: URL: https://github.com/apache/doris/pull/38519#issuecomment-2259735569 ClickBench: Total hot run time: 31.44 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools

Re: [PR] [enhance](S3) Use allocator for s3 buffer's allocation [doris]

2024-07-30 Thread via GitHub
doris-robot commented on PR #38519: URL: https://github.com/apache/doris/pull/38519#issuecomment-2259729263 TPC-DS: Total hot run time: 203833 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS

Re: [PR] [enhance](S3) Use allocator for s3 buffer's allocation [doris]

2024-07-30 Thread via GitHub
doris-robot commented on PR #38519: URL: https://github.com/apache/doris/pull/38519#issuecomment-2259717637 TeamCity be ut coverage result: Function Coverage: 37.80% (8116/21472) Line Coverage: 29.47% (66562/225884) Region Coverage: 28.95% (34311/118528) Branch Coverage: 24.8

Re: [PR] [enhance](S3) Use allocator for s3 buffer's allocation [doris]

2024-07-30 Thread via GitHub
doris-robot commented on PR #38519: URL: https://github.com/apache/doris/pull/38519#issuecomment-2259716399 TPC-H: Total hot run time: 50080 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100

Re: [PR] [enhance](S3) Use allocator for s3 buffer's allocation [doris]

2024-07-30 Thread via GitHub
ByteYue commented on PR #38519: URL: https://github.com/apache/doris/pull/38519#issuecomment-2259681214 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

Re: [PR] [enhance](S3) Use allocator for s3 buffer's allocation [doris]

2024-07-30 Thread via GitHub
doris-robot commented on PR #38519: URL: https://github.com/apache/doris/pull/38519#issuecomment-2259626816 TeamCity be ut coverage result: Function Coverage: 37.81% (8118/21471) Line Coverage: 29.48% (66581/225880) Region Coverage: 28.96% (34325/118527) Branch Coverage: 24.8

Re: [PR] [enhance](S3) Use allocator for s3 buffer's allocation [doris]

2024-07-30 Thread via GitHub
doris-robot commented on PR #38519: URL: https://github.com/apache/doris/pull/38519#issuecomment-2259623812 Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' ``` Load test result on commit 6db5c2f1ceb5ce7bc954d6bd27850f002fa1cd30 with default session variables Stream

Re: [PR] [enhance](S3) Use allocator for s3 buffer's allocation [doris]

2024-07-30 Thread via GitHub
doris-robot commented on PR #38519: URL: https://github.com/apache/doris/pull/38519#issuecomment-2259621098 ClickBench: Total hot run time: 30.09 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools

Re: [PR] [enhance](S3) Use allocator for s3 buffer's allocation [doris]

2024-07-30 Thread via GitHub
doris-robot commented on PR #38519: URL: https://github.com/apache/doris/pull/38519#issuecomment-2259616783 TPC-DS: Total hot run time: 203707 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS

Re: [PR] [enhance](S3) Use allocator for s3 buffer's allocation [doris]

2024-07-30 Thread via GitHub
doris-robot commented on PR #38519: URL: https://github.com/apache/doris/pull/38519#issuecomment-2259607112 TPC-H: Total hot run time: 50056 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100

Re: [PR] [enhance](S3) Use allocator for s3 buffer's allocation [doris]

2024-07-30 Thread via GitHub
ByteYue commented on PR #38519: URL: https://github.com/apache/doris/pull/38519#issuecomment-2259570107 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

Re: [PR] [enhance](S3) Use allocator for s3 buffer's allocation [doris]

2024-07-30 Thread via GitHub
github-actions[bot] commented on PR #38519: URL: https://github.com/apache/doris/pull/38519#issuecomment-2258434314 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

Re: [PR] [enhance](S3) Use allocator for s3 buffer's allocation [doris]

2024-07-30 Thread via GitHub
github-actions[bot] commented on PR #38519: URL: https://github.com/apache/doris/pull/38519#issuecomment-2258434375 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] [enhance](S3) Use allocator for s3 buffer's allocation [doris]

2024-07-30 Thread via GitHub
doris-robot commented on PR #38519: URL: https://github.com/apache/doris/pull/38519#issuecomment-2258266437 Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' ``` Load test result on commit 7aa70d7780d123c8f52de7108aa4b8536b5ffd72 with default session variables Stream

Re: [PR] [enhance](S3) Use allocator for s3 buffer's allocation [doris]

2024-07-30 Thread via GitHub
doris-robot commented on PR #38519: URL: https://github.com/apache/doris/pull/38519#issuecomment-2258259872 ClickBench: Total hot run time: 31.34 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools

Re: [PR] [enhance](S3) Use allocator for s3 buffer's allocation [doris]

2024-07-30 Thread via GitHub
doris-robot commented on PR #38519: URL: https://github.com/apache/doris/pull/38519#issuecomment-2258249471 TPC-DS: Total hot run time: 202617 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS

Re: [PR] [enhance](S3) Use allocator for s3 buffer's allocation [doris]

2024-07-30 Thread via GitHub
doris-robot commented on PR #38519: URL: https://github.com/apache/doris/pull/38519#issuecomment-2258233256 TeamCity be ut coverage result: Function Coverage: 37.82% (8120/21472) Line Coverage: 29.49% (66604/225887) Region Coverage: 28.97% (34338/118526) Branch Coverage: 24.8

Re: [PR] [enhance](S3) Use allocator for s3 buffer's allocation [doris]

2024-07-30 Thread via GitHub
doris-robot commented on PR #38519: URL: https://github.com/apache/doris/pull/38519#issuecomment-2258226179 TPC-H: Total hot run time: 49921 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100

Re: [PR] [enhance](S3) Use allocator for s3 buffer's allocation [doris]

2024-07-30 Thread via GitHub
ByteYue commented on PR #38519: URL: https://github.com/apache/doris/pull/38519#issuecomment-2258168739 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

Re: [PR] [enhance](S3) Use allocator for s3 buffer's allocation [doris]

2024-07-30 Thread via GitHub
doris-robot commented on PR #38519: URL: https://github.com/apache/doris/pull/38519#issuecomment-2257851826 Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' ``` Load test result on commit d801554ad811d8b7ef56ecb2162ba8ff0813e675 with default session variables Stream

Re: [PR] [enhance](S3) Use allocator for s3 buffer's allocation [doris]

2024-07-30 Thread via GitHub
doris-robot commented on PR #38519: URL: https://github.com/apache/doris/pull/38519#issuecomment-2257845113 ClickBench: Total hot run time: 31.33 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools

Re: [PR] [enhance](S3) Use allocator for s3 buffer's allocation [doris]

2024-07-30 Thread via GitHub
doris-robot commented on PR #38519: URL: https://github.com/apache/doris/pull/38519#issuecomment-2257830769 TPC-DS: Total hot run time: 205193 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS

Re: [PR] [enhance](S3) Use allocator for s3 buffer's allocation [doris]

2024-07-30 Thread via GitHub
doris-robot commented on PR #38519: URL: https://github.com/apache/doris/pull/38519#issuecomment-2257805913 TPC-H: Total hot run time: 50407 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100

Re: [PR] [enhance](S3) Use allocator for s3 buffer's allocation [doris]

2024-07-30 Thread via GitHub
doris-robot commented on PR #38519: URL: https://github.com/apache/doris/pull/38519#issuecomment-2257801339 TeamCity be ut coverage result: Function Coverage: 37.82% (8120/21471) Line Coverage: 29.49% (66596/225862) Region Coverage: 28.97% (34336/118522) Branch Coverage: 24.8

Re: [PR] [enhance](S3) Use allocator for s3 buffer's allocation [doris]

2024-07-30 Thread via GitHub
ByteYue commented on PR #38519: URL: https://github.com/apache/doris/pull/38519#issuecomment-2257746770 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

Re: [PR] [enhance](S3) Use allocator for s3 buffer's allocation [doris]

2024-07-30 Thread via GitHub
ByteYue commented on PR #38519: URL: https://github.com/apache/doris/pull/38519#issuecomment-2257669367 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

Re: [PR] [enhance](S3) Use allocator for s3 buffer's allocation [doris]

2024-07-30 Thread via GitHub
github-actions[bot] commented on code in PR #38519: URL: https://github.com/apache/doris/pull/38519#discussion_r1696432938 ## be/src/io/fs/s3_file_write_bufferpool.cpp: ## @@ -29,13 +30,43 @@ namespace doris { namespace io { + +bvar::Adder s3_file_buffer_allocated("s3_file_b

Re: [PR] [enhance](S3) Use allocator for s3 buffer's allocation [doris]

2024-07-30 Thread via GitHub
doris-robot commented on PR #38519: URL: https://github.com/apache/doris/pull/38519#issuecomment-2257619537 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+y

[PR] [enhance](S3) Use allocator for s3 buffer's allocation [doris]

2024-07-30 Thread via GitHub
ByteYue opened a new pull request, #38519: URL: https://github.com/apache/doris/pull/38519 ## Proposed changes Remove the static buffer pool for S3 upload. Use the dynamic allocator to control the memory usage for S3 upload which is more adaptive and easy to use. Remove all t

Re: [PR] [enhance](S3) Use allocator for s3 buffer's allocation [doris]

2024-07-30 Thread via GitHub
ByteYue commented on PR #38519: URL: https://github.com/apache/doris/pull/38519#issuecomment-2257620922 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu