Jackie-Jiang commented on code in PR #10052:
URL: https://github.com/apache/pinot/pull/10052#discussion_r1068574956
##########
pinot-server/src/main/java/org/apache/pinot/server/api/resources/TablesResource.java:
##########
@@ -423,6 +414,64 @@ public Response downloadSegment(
}
}
+ /**
+ * Download snapshot for the given immutable segment for upsert table. This
endpoint is used when get snapshot from
+ * peer to avoid recompute when reload segments.
+ */
+ @GET
+ @Produces(MediaType.APPLICATION_OCTET_STREAM)
+ @Path("/segments/{tableNameWithType}/{segmentName}/validDocIds")
+ @ApiOperation(value = "Download validDocIds for an REALTIME immutable
segment", notes = "Download validDocIds for "
+ + "an immutable segment in bitmap format.")
+ public Response downloadValidDocIds(
+ @ApiParam(value = "Name of the table with type REALTIME", required =
true, example = "myTable_REALTIME")
+ @PathParam("tableNameWithType") String tableNameWithType,
+ @ApiParam(value = "Name of the segment", required = true)
@PathParam("segmentName") @Encoded String segmentName,
+ @Context HttpHeaders httpHeaders)
+ throws Exception {
+ LOGGER.info("Received a request to download validDocIds for segment {}
table {}", segmentName, tableNameWithType);
+ // Validate data access
+ ServerResourceUtils.validateDataAccess(_accessControlFactory,
tableNameWithType, httpHeaders);
+
+ TableDataManager tableDataManager =
+ ServerResourceUtils.checkGetTableDataManager(_serverInstance,
tableNameWithType);
+ SegmentDataManager segmentDataManager =
tableDataManager.acquireSegment(segmentName);
+ if (segmentDataManager == null) {
+ throw new WebApplicationException(
+ String.format("Table %s segment %s does not exist",
tableNameWithType, segmentName),
+ Response.Status.NOT_FOUND);
+ }
+
+ try {
+ IndexSegment indexSegment = segmentDataManager.getSegment();
+ if (!(indexSegment instanceof ImmutableSegmentImpl)) {
+ throw new WebApplicationException(
+ String.format("Table %s segment %s is not a immutable segment",
tableNameWithType, segmentName),
+ Response.Status.BAD_REQUEST);
+ }
+ MutableRoaringBitmap validDocIds =
+ indexSegment.getValidDocIds() != null ?
indexSegment.getValidDocIds().getMutableRoaringBitmap() : null;
+
+ if (validDocIds == null) {
+ throw new WebApplicationException(
+ String.format("Missing validDocIds for table %s segment %s does
not exist", tableNameWithType, segmentName),
+ Response.Status.NOT_FOUND);
+ }
+
+ ByteArrayOutputStream byteArrayOutputStream = new
ByteArrayOutputStream();
+ try (DataOutputStream dataOutputStream = new
DataOutputStream(byteArrayOutputStream)) {
+ validDocIds.serialize(dataOutputStream);
+ byteArrayOutputStream.close();
+ }
+ byte[] validDocIdsSnapshot = byteArrayOutputStream.toByteArray();
Review Comment:
You may use `RoaringBitmapUtils.serialize()` to serialize it. You'll need to
change the argument to `ImmutableBitmapDataProvider`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
