jtao15 commented on code in PR #15177:
URL: https://github.com/apache/pinot/pull/15177#discussion_r1980503371
##########
pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/test/java/org/apache/pinot/plugin/minion/tasks/mergerollup/MergeRollupTaskGeneratorTest.java:
##########
@@ -283,6 +283,50 @@ public void testGenerateTasksCheckConfigs() {
assertTrue(pinotTaskConfigs.isEmpty());
}
+ /**
+ * Test pre-filter of task generation
+ */
+ @Test
+ public void testFilterSegmentsBasedOnStatus() {
+ ClusterInfoAccessor mockClusterInfoProvide =
mock(ClusterInfoAccessor.class);
+
+
when(mockClusterInfoProvide.getTaskStates(MinionConstants.MergeRollupTask.TASK_TYPE)).thenReturn(new
HashMap<>());
+ // construct 4 following segments, among these, only 0_0 can be scheduled,
others should be filtered out
+ // partition 0, completed 0
+ SegmentZKMetadata realtimeTableSegmentMetadata1 =
+ getSegmentZKMetadata("testTable__0__0__0", 5000, 6000,
TimeUnit.MILLISECONDS,
+ null, "50000", "60000");
+
realtimeTableSegmentMetadata1.setStatus(CommonConstants.Segment.Realtime.Status.DONE);
+ // partition 0, completed 1
+ SegmentZKMetadata realtimeTableSegmentMetadata2 =
+ getSegmentZKMetadata("testTable__0__1__1", 6000, 7000,
TimeUnit.MILLISECONDS,
+ null, "60000", "70000");
+
realtimeTableSegmentMetadata2.setStatus(CommonConstants.Segment.Realtime.Status.DONE);
+ // partition 0, consuming 2
+ SegmentZKMetadata realtimeTableSegmentMetadata3 =
+ getSegmentZKMetadata("testTable__0__2__2", 7000, 8000,
TimeUnit.MILLISECONDS,
+ null, "70000", "80000");
+
realtimeTableSegmentMetadata3.setStatus(CommonConstants.Segment.Realtime.Status.IN_PROGRESS);
+ // partition 1, completed 0
+ SegmentZKMetadata realtimeTableSegmentMetadata4 =
+ getSegmentZKMetadata("testTable__1__0__0", 5500, 6500,
TimeUnit.MILLISECONDS,
+ null, "55000", "65000");
+
realtimeTableSegmentMetadata4.setStatus(CommonConstants.Segment.Realtime.Status.DONE);
+
when(mockClusterInfoProvide.getSegmentsZKMetadata(REALTIME_TABLE_NAME)).thenReturn(
+ Lists.newArrayList(realtimeTableSegmentMetadata1,
realtimeTableSegmentMetadata2,
+ realtimeTableSegmentMetadata3,
realtimeTableSegmentMetadata4));
+ when(mockClusterInfoProvide.getIdealState(REALTIME_TABLE_NAME)).thenReturn(
+ getIdealState(REALTIME_TABLE_NAME,
Lists.newArrayList("testTable__0", "server0", "ONLINE")));
+
+ MergeRollupTaskGenerator generator = new MergeRollupTaskGenerator();
+ generator.init(mockClusterInfoProvide);
+
+
assertEquals(MergeRollupTaskGenerator.filterSegmentsBasedOnStatus(TableType.REALTIME,
Review Comment:
(nit) maybe assert on the name instead of the size?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
