nastra commented on code in PR #6919:
URL: https://github.com/apache/iceberg/pull/6919#discussion_r1150598541
##########
core/src/test/java/org/apache/iceberg/TestScanPlanningAndReporting.java:
##########
@@ -42,6 +43,33 @@ public TestScanPlanningAndReporting() {
super(2);
}
+ @Test
+ public void scanningWithMutipleReporters() throws IOException {
Review Comment:
```suggestion
public void scanningWithMultipleReporters() throws IOException {
```
##########
core/src/test/java/org/apache/iceberg/TestScanPlanningAndReporting.java:
##########
@@ -42,6 +43,33 @@ public TestScanPlanningAndReporting() {
super(2);
}
+ @Test
+ public void scanningWithMutipleReporters() throws IOException {
+ String tableName = "scan-with-multiple-reporters";
+ Table table =
+ TestTables.create(
+ tableDir, tableName, SCHEMA, SPEC, SortOrder.unsorted(),
formatVersion, reporter);
+ table.newAppend().appendFile(FILE_A).commit();
+ table.refresh();
+
+ AtomicInteger reportedCount = new AtomicInteger();
+ TableScan tableScan =
+ table
+ .newScan()
+ .metricsReporter(
+ (MetricsReporter) -> {
+ reportedCount.getAndIncrement();
+ })
+ .metricsReporter(
+ (MetricsReporter) -> {
+ reportedCount.getAndIncrement();
+ });
Review Comment:
```suggestion
.metricsReporter((MetricsReporter) ->
reportedCount.getAndIncrement())
.metricsReporter((MetricsReporter) ->
reportedCount.getAndIncrement());
```
##########
core/src/test/java/org/apache/iceberg/TestScanPlanningAndReporting.java:
##########
@@ -42,6 +43,33 @@ public TestScanPlanningAndReporting() {
super(2);
}
+ @Test
+ public void scanningWithMutipleReporters() throws IOException {
+ String tableName = "scan-with-multiple-reporters";
+ Table table =
+ TestTables.create(
+ tableDir, tableName, SCHEMA, SPEC, SortOrder.unsorted(),
formatVersion, reporter);
+ table.newAppend().appendFile(FILE_A).commit();
+ table.refresh();
+
+ AtomicInteger reportedCount = new AtomicInteger();
+ TableScan tableScan =
+ table
+ .newScan()
+ .metricsReporter(
+ (MetricsReporter) -> {
+ reportedCount.getAndIncrement();
+ })
+ .metricsReporter(
+ (MetricsReporter) -> {
+ reportedCount.getAndIncrement();
+ });
+ try (CloseableIterable<FileScanTask> fileScanTasks =
tableScan.planFiles()) {
+ fileScanTasks.forEach(task -> {});
+ }
+ assertThat(reportedCount.get()).isEqualTo(2);
Review Comment:
this is also to make sure the default metrics reporter still reports
##########
core/src/test/java/org/apache/iceberg/TestScanPlanningAndReporting.java:
##########
@@ -42,6 +43,33 @@ public TestScanPlanningAndReporting() {
super(2);
}
+ @Test
+ public void scanningWithMutipleReporters() throws IOException {
+ String tableName = "scan-with-multiple-reporters";
+ Table table =
+ TestTables.create(
+ tableDir, tableName, SCHEMA, SPEC, SortOrder.unsorted(),
formatVersion, reporter);
+ table.newAppend().appendFile(FILE_A).commit();
+ table.refresh();
+
+ AtomicInteger reportedCount = new AtomicInteger();
+ TableScan tableScan =
+ table
+ .newScan()
+ .metricsReporter(
+ (MetricsReporter) -> {
+ reportedCount.getAndIncrement();
+ })
+ .metricsReporter(
+ (MetricsReporter) -> {
+ reportedCount.getAndIncrement();
+ });
+ try (CloseableIterable<FileScanTask> fileScanTasks =
tableScan.planFiles()) {
+ fileScanTasks.forEach(task -> {});
+ }
+ assertThat(reportedCount.get()).isEqualTo(2);
Review Comment:
```suggestion
try (CloseableIterable<FileScanTask> fileScanTasks =
tableScan.planFiles()) {
fileScanTasks.forEach(task -> {});
}
assertThat(reportedCount.get()).isEqualTo(2);
// make sure default metrics reporter is still reporting
ScanReport scanReport = reporter.lastReport();
assertThat(scanReport).isNotNull();
assertThat(scanReport.tableName()).isEqualTo(tableName);
assertThat(scanReport.snapshotId()).isEqualTo(1L);
ScanMetricsResult result = scanReport.scanMetrics();
assertThat(result.totalPlanningDuration().totalDuration()).isGreaterThan(Duration.ZERO);
assertThat(result.resultDataFiles().value()).isEqualTo(1);
```
##########
api/src/main/java/org/apache/iceberg/Scan.java:
##########
@@ -171,4 +172,10 @@ default ThisT select(String... columns) {
/** Returns the split open file cost for this scan. */
long splitOpenFileCost();
+
+ /** Create a new scan that will report the scan metrics to the {@code
reporter} */
Review Comment:
+1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
