shounakmk219 commented on code in PR #13296:
URL: https://github.com/apache/pinot/pull/13296#discussion_r1717065215
##########
pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/TableViews.java:
##########
@@ -17,15 +17,18 @@
* under the License.
*/
package org.apache.pinot.controller.api.resources;
-
Review Comment:
nit: revert the newline removal
##########
pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotSegmentRestletResource.java:
##########
@@ -231,20 +231,25 @@ public List<Map<TableType, List<String>>> getSegments(
@Path("segments/{tableName}/servers")
@Authorize(targetType = TargetType.TABLE, paramName = "tableName", action =
Actions.Table.GET_SERVER_MAP)
@Produces(MediaType.APPLICATION_JSON)
- @ApiOperation(value = "Get a map from server to segments hosted by the
server",
- notes = "Get a map from server to segments hosted by the server")
+ @ApiOperation(value = "Get a map from server to segments hosted by the
server", notes = "Get a map from server to "
+ + "segments hosted by the server")
public List<Map<String, Object>> getServerToSegmentsMap(
@ApiParam(value = "Name of the table", required = true)
@PathParam("tableName") String tableName,
- @ApiParam(value = "OFFLINE|REALTIME") @QueryParam("type") String
tableTypeStr, @Context HttpHeaders headers) {
+ @ApiParam(value = "OFFLINE|REALTIME") @QueryParam("type") String
tableTypeStr,
+ @QueryParam("detailed") @DefaultValue("true") boolean detailed, @Context
HttpHeaders headers) {
tableName = DatabaseUtils.translateTableName(tableName, headers);
- List<String> tableNamesWithType = ResourceUtils
- .getExistingTableNamesWithType(_pinotHelixResourceManager, tableName,
Constants.validateTableType(tableTypeStr),
- LOGGER);
+ List<String> tableNamesWithType =
ResourceUtils.getExistingTableNamesWithType(_pinotHelixResourceManager,
tableName,
+ Constants.validateTableType(tableTypeStr), LOGGER);
List<Map<String, Object>> resultList = new
ArrayList<>(tableNamesWithType.size());
for (String tableNameWithType : tableNamesWithType) {
Map<String, Object> resultForTable = new LinkedHashMap<>();
resultForTable.put("tableName", tableNameWithType);
- resultForTable.put("serverToSegmentsMap",
_pinotHelixResourceManager.getServerToSegmentsMap(tableNameWithType));
+ Map<String, List<String>> serverToSegmentsMap =
Review Comment:
Let's not build the in memory map if detailed is false
##########
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java:
##########
@@ -448,6 +449,13 @@ public List<InstanceConfig> getAllHelixInstanceConfigs() {
return HelixHelper.getInstanceConfigs(_helixZkManager);
}
+ /**
+ * returns server to segments count
+ */
+ public Map<String, Integer> getServerToSegmentCountMap() {
+ return _serverToSegmentCountMap;
Review Comment:
I don’t think this is a right way to achieve this as it depends on the
invocation of `getServerToSegmentsMap`.
You can have the same code logic as `getServerToSegmentsMap` and aggregate
only the count.
`getServerToSegmentsMap` can call this when `detailed` is false and compute
the count from the response of `getServerToSegmentsMap` itself when `detailed`
is true
##########
pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/TableViews.java:
##########
@@ -110,6 +115,75 @@ public TableView getExternalView(
return getTableState(tableName, EXTERNALVIEW, tableType);
}
+ @GET
+ @Produces(MediaType.APPLICATION_JSON)
+ @Path("/tables/{tableName}/segmentsStatus")
+ @Authorize(targetType = TargetType.TABLE, paramName = "tableName", action =
Actions.Table.GET_SEGMENT_STATUS)
+ @ApiOperation(value = "Get segment names to segment status map", notes =
"Get segment statuses of each segment")
+ public String getSegmentsStatusDetails(
+ @ApiParam(value = "Name of the table", required = true)
@PathParam("tableName") String tableName,
+ @ApiParam(value = "realtime|offline", required = false)
@QueryParam("tableType") String tableTypeStr,
+ @Context HttpHeaders headers)
+ throws JsonProcessingException {
+ tableName = DatabaseUtils.translateTableName(tableName, headers);
+ TableType tableType = validateTableType(tableTypeStr);
+ TableViews.TableView externalView = getTableState(tableName,
TableViews.EXTERNALVIEW, tableType);
+ TableViews.TableView idealStateView = getTableState(tableName,
TableViews.IDEALSTATE, tableType);
+ List<SegmentStatusInfo> segmentStatusInfoListMap = new ArrayList<>();
+ segmentStatusInfoListMap = getSegmentStatuses(externalView,
idealStateView);
+ return JsonUtils.objectToPrettyString(segmentStatusInfoListMap);
+ }
+
+ public List<SegmentStatusInfo> getSegmentStatuses(TableViews.TableView
externalView,
+ TableViews.TableView idealStateView) {
+ Map<String, Map<String, String>> idealStateMap =
getStateMap(idealStateView);
+ Map<String, Map<String, String>> externalViewMap =
getStateMap(externalView);
+ List<SegmentStatusInfo> segmentStatusInfoList = new ArrayList<>();
+
+ for (Map.Entry<String, Map<String, String>> entry :
externalViewMap.entrySet()) {
+ String segment = entry.getKey();
+ Map<String, String> externalViewEntryValue = entry.getValue();
+ if (isErrorSegment(externalViewEntryValue)) {
+ segmentStatusInfoList.add(
+ new SegmentStatusInfo(segment,
CommonConstants.Helix.StateModel.DisplaySegmentStatus.BAD));
+ } else if (isOnlineOrConsumingSegment(externalViewEntryValue) &&
externalViewMap.equals(idealStateMap)) {
Review Comment:
We should check the equality of the entry from EV to the respective segment
entry in IS and not the whole maps.
Also put a check for the entry inequality at the top and mark it `UPDATING`
right away. This way we can skip the entry equality check in remaining blocks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
