[
https://issues.apache.org/jira/browse/HBASE-29398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18007681#comment-18007681
]
Hudson commented on HBASE-29398:
--------------------------------
Results for branch master
[build #1320 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1320/]:
(x) *{color:red}-1 overall{color}*
----
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1320/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk17 hadoop3 checks{color}
-- For more information [see jdk17
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1320/JDK17_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk17 hadoop 3.3.5 backward compatibility checks{color}
-- For more information [see jdk17
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1320/JDK17_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk17 hadoop 3.3.6 backward compatibility checks{color}
-- For more information [see jdk17
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1320/JDK17_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk17 hadoop 3.4.0 backward compatibility checks{color}
-- For more information [see jdk17
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1320/JDK17_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test for 3.3.5 {color}
(/) {color:green}+1 client integration test for 3.3.6 {color}
(/) {color:green}+1 client integration test for 3.4.0 {color}
(/) {color:green}+1 client integration test for 3.4.1 {color}
> Server side scan metrics for bytes read from FS vs Block cache vs memstore
> --------------------------------------------------------------------------
>
> Key: HBASE-29398
> URL: https://issues.apache.org/jira/browse/HBASE-29398
> Project: HBase
> Issue Type: Improvement
> Reporter: Sanjeet Malhotra
> Assignee: Sanjeet Malhotra
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.0.0-beta-2
>
>
> Currently, HBase doesn't have a metric on the server side which counts how
> many bytes were read from FS vs block cache vs memstore. Reading cells from
> in-memory like block cache or memstore vs from FS can make latencies vary
> drastically.
> Separate metrics for bytes scanned from block cache vs memstore are
> beneficial for use cases which immediately read (like within 5 sec) after
> writing the data. There the expectation would be that bytes scanned from FS
> or block cache should be negligible unless a flush happened (which can be
> checked from logs) or cache on write is enabled.
> Currently, HBase has a server side scan metric `countOfBlockBytesScanned`
> which aims to capture the block bytes scanned by read request. But it doesn't
> capture:
> * Block bytes scanned as part of KeyValueHeap#pollRealKV(). So, we could end
> up scanning a lot of excess HFiles in case when scanner at the top of heap
> hasn't done a real seek and the row might get returned from memstore. In this
> case the block I/O done in scanning excess HFile won't be accounted in any
> metrics.
> * Index block bytes scanned, bloom filter bytes scanned.
> * Difference between bytes scanned from block cache vs FS.
> The proposal is to add 3 new server side scan metrics, one each for: bytes
> scanned from FS, bytes scanned from block cache and bytes scanned from
> memstore.
>
> I intend to cherry-pick this change to HBase 3 and HBase 2 (till HBase 2.5).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
