mcvsubbu commented on a change in pull request #7102:
URL: https://github.com/apache/incubator-pinot/pull/7102#discussion_r660858409
##########
File path:
pinot-controller/src/main/java/org/apache/pinot/controller/util/ServerSegmentMetadataReader.java
##########
@@ -91,6 +162,24 @@ public ServerSegmentMetadataReader(Executor executor,
HttpConnectionManager conn
return segmentsMetadata;
}
+ private String generateAggregateSegmentMetadataServerURL(String
tableNameWithType, List<String> columns,
+ String endpoint) {
+ try {
+ tableNameWithType = URLEncoder.encode(tableNameWithType,
StandardCharsets.UTF_8.name());
+ String paramsStr = "";
+ if (columns != null) {
+ List<String> params = new ArrayList<>(columns.size());
+ for (String column : columns) {
+ params.add(String.format("columns=%s", column));
Review comment:
Do e need to urlencode `column`?
##########
File path:
pinot-server/src/main/java/org/apache/pinot/server/api/resources/TablesResource.java
##########
@@ -118,6 +126,103 @@ public String listTableSegments(
}
}
+ @GET
+ @Encoded
+ @Produces(MediaType.APPLICATION_JSON)
+ @Path("/tables/{tableName}/metadata")
+ @ApiOperation(value = "List metadata for all segments of a given table",
notes = "List segments metadata of table hosted on this server")
+ @ApiResponses(value = {@ApiResponse(code = 200, message = "Success"),
@ApiResponse(code = 500, message = "Internal server error"), @ApiResponse(code
= 404, message = "Table not found")})
+ public String getTableSize(
+ @ApiParam(value = "Table Name with type", required = true)
@PathParam("tableName") String tableName,
+ @ApiParam(value = "Column name", allowMultiple = true)
@QueryParam("columns") @DefaultValue("") List<String> columns)
+ throws WebApplicationException {
+ InstanceDataManager instanceDataManager =
_serverInstance.getInstanceDataManager();
+
+ if (instanceDataManager == null) {
+ throw new WebApplicationException("Invalid server initialization",
Response.Status.INTERNAL_SERVER_ERROR);
+ }
+
+ TableDataManager tableDataManager =
instanceDataManager.getTableDataManager(tableName);
+ if (tableDataManager == null) {
+ throw new WebApplicationException("Table: " + tableName + " is not
found", Response.Status.NOT_FOUND);
+ }
+
+ for (int i = 0; i < columns.size(); i++) {
+ try {
+ columns.set(i, URLDecoder.decode(columns.get(i),
StandardCharsets.UTF_8.name()));
+ } catch (UnsupportedEncodingException e) {
+ throw new RuntimeException(e.getCause());
+ }
+ }
+ Set<String> columnSet;
+ if (columns.size() == 1 && columns.get(0).equals("*")) {
Review comment:
For robustness, it may be useful to look for `*` in any column, and
ignore the others in `columns`
##########
File path:
pinot-server/src/main/java/org/apache/pinot/server/api/resources/TablesResource.java
##########
@@ -118,6 +126,103 @@ public String listTableSegments(
}
}
+ @GET
+ @Encoded
+ @Produces(MediaType.APPLICATION_JSON)
+ @Path("/tables/{tableName}/metadata")
+ @ApiOperation(value = "List metadata for all segments of a given table",
notes = "List segments metadata of table hosted on this server")
+ @ApiResponses(value = {@ApiResponse(code = 200, message = "Success"),
@ApiResponse(code = 500, message = "Internal server error"), @ApiResponse(code
= 404, message = "Table not found")})
+ public String getTableSize(
+ @ApiParam(value = "Table Name with type", required = true)
@PathParam("tableName") String tableName,
+ @ApiParam(value = "Column name", allowMultiple = true)
@QueryParam("columns") @DefaultValue("") List<String> columns)
+ throws WebApplicationException {
+ InstanceDataManager instanceDataManager =
_serverInstance.getInstanceDataManager();
+
+ if (instanceDataManager == null) {
+ throw new WebApplicationException("Invalid server initialization",
Response.Status.INTERNAL_SERVER_ERROR);
+ }
+
+ TableDataManager tableDataManager =
instanceDataManager.getTableDataManager(tableName);
+ if (tableDataManager == null) {
+ throw new WebApplicationException("Table: " + tableName + " is not
found", Response.Status.NOT_FOUND);
+ }
+
+ for (int i = 0; i < columns.size(); i++) {
+ try {
+ columns.set(i, URLDecoder.decode(columns.get(i),
StandardCharsets.UTF_8.name()));
Review comment:
Modifying the incoming parameter? I would suggest to just create a new
set
##########
File path:
pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotTableRestletResource.java
##########
@@ -625,4 +629,62 @@ public String getTableStatus(
Response.Status.INTERNAL_SERVER_ERROR, e);
}
}
+
+ @GET
+ @Path("tables/{tableName}/metadata")
+ @Produces(MediaType.APPLICATION_JSON)
+ @ApiOperation(value = "Get the aggregate metadata of all segments for a
table", notes = "Get the aggregate metadata of all segments for a table")
+ public String getTableAggregateMetadata(
+ @ApiParam(value = "Name of the table", required = true)
@PathParam("tableName") String tableName,
+ @ApiParam(value = "OFFLINE|REALTIME") @QueryParam("type") String
tableTypeStr,
+ @ApiParam(value = "Columns name", allowMultiple = true)
@QueryParam("columns") @DefaultValue("") List<String> columns) {
+ LOGGER.info("Received a request to fetch aggregate metadata for a table
{}", tableName);
+ TableType tableType = Constants.validateTableType(tableTypeStr);
+ if (tableType == TableType.REALTIME) {
+ throw new ControllerApplicationException(LOGGER, "Table type : " +
tableTypeStr + " not yet supported.",
+ Response.Status.NOT_IMPLEMENTED);
+ }
+ String tableNameWithType = getExistingTableNamesWithType(tableName,
tableType).get(0);
Review comment:
why do we need to get existing table names here? We can get the table
config and if it is null, return 404, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
