luoyuxia commented on code in PR #2454:
URL: https://github.com/apache/fluss/pull/2454#discussion_r2928842441
##########
website/docs/maintenance/observability/monitor-metrics.md:
##########
@@ -395,6 +395,32 @@ Some metrics might not be exposed when using other JVM
implementations (e.g. IBM
<td>The number of tables currently being tiered.</td>
<td>Gauge</td>
</tr>
+ <tr>
+ <td rowspan="5">lakeTiering_table</td>
+ <td>tierLag</td>
+ <td>Time in milliseconds since the last successful tiering operation for
this table. For newly registered tables that have never completed a tiering
round, the lag is measured from the time the table was registered.</td>
+ <td>Gauge</td>
+ </tr>
+ <tr>
+ <td>tierDuration</td>
+ <td>Duration in milliseconds of the last tiering operation for this
table.</td>
+ <td>Gauge</td>
+ </tr>
+ <tr>
+ <td>failuresTotal</td>
+ <td>The total number of tiering failures for this table.</td>
+ <td>Gauge</td>
Review Comment:
Should change to type Counter
##########
fluss-lake/fluss-lake-paimon/src/main/java/org/apache/fluss/lake/paimon/tiering/PaimonLakeCommitter.java:
##########
@@ -153,6 +161,31 @@ public LakeCommitResult commit(
}
}
+ /** Computes cumulative table stats from the latest snapshot by REST API.
*/
+ private TieringStats computeTableStats() {
+ Identifier identifier =
+ new Identifier(tablePath.getDatabaseName(),
tablePath.getTableName());
+ try {
+ Optional<TableSnapshot> snapshot =
paimonCatalog.loadSnapshot(identifier);
+ if (!snapshot.isPresent()) {
+ LOG.warn(
+ "No snapshot found for table {}, "
+ + "fileSize and recordCount will be reported
as -1.",
+ tablePath);
+ return TieringStats.UNKNOWN;
+ }
+ TableSnapshot tableSnapshot = snapshot.get();
+ return new TieringStats(tableSnapshot.fileSizeInBytes(),
tableSnapshot.recordCount());
+ } catch (Exception e) {
Review Comment:
nit:
trace or debug `UnsupportedOperationException` to avoid to many warning logs.
##########
fluss-flink/fluss-flink-common/src/main/java/org/apache/fluss/flink/tiering/event/FinishedTieringEvent.java:
##########
@@ -26,11 +28,23 @@ public class FinishedTieringEvent implements SourceEvent {
private final long tableId;
- public FinishedTieringEvent(long tableId) {
+ /** Statistics collected during this tiering round. */
+ private final TieringStats stats;
+
+ public FinishedTieringEvent(long tableId, TieringStats stats) {
Review Comment:
nit: add nullable for `TieringStats`
##########
fluss-common/src/main/java/org/apache/fluss/lake/committer/LakeCommitResult.java:
##########
@@ -61,23 +61,59 @@ public class LakeCommitResult {
// 2: committedIsReadable is true, committed snapshot is just also readable
@Nullable private final ReadableSnapshot readableSnapshot;
+ // Total file size (bytes) of all live data files in the lake table after
this commit.
+ // {@code null} means the value is not reported by the lake implementation.
+ @Nullable private final Long totalLakeFileSize;
Review Comment:
nit: can `totalLakeFileSize` and `totalLakeRecordCount` be combined into
`TieringStats` directly for code simpility?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
