junrao commented on code in PR #15825:
URL: https://github.com/apache/kafka/pull/15825#discussion_r1586835182
##########
storage/src/main/java/org/apache/kafka/storage/internals/log/LogOffsetMetadata.java:
##########
@@ -28,6 +28,7 @@ public final class LogOffsetMetadata {
//TODO KAFKA-14484 remove once UnifiedLog has been moved to the storage
module
private static final long UNIFIED_LOG_UNKNOWN_OFFSET = -1L;
+ public static final long REMOTE_LOG_UNKNOWN_OFFSET = -2L;
Review Comment:
We probably don't need this. The existing UNIFIED_LOG_UNKNOWN_OFFSET should
be enough.
##########
storage/src/main/java/org/apache/kafka/storage/internals/log/LogOffsetMetadata.java:
##########
@@ -51,6 +57,8 @@ public LogOffsetMetadata(long messageOffset,
// check if this offset is already on an older segment compared with the
given offset
public boolean onOlderSegment(LogOffsetMetadata that) {
+ if (this.segmentBaseOffset == REMOTE_LOG_UNKNOWN_OFFSET ||
that.segmentBaseOffset == REMOTE_LOG_UNKNOWN_OFFSET)
Review Comment:
We probably don't need this. If messageOffsetOnly() is true, we can just
return false.
##########
storage/src/main/java/org/apache/kafka/storage/internals/log/LogOffsetMetadata.java:
##########
@@ -65,6 +73,8 @@ private boolean onSameSegment(LogOffsetMetadata that) {
// compute the number of bytes between this offset to the given offset
// if they are on the same segment and this offset precedes the given
offset
public int positionDiff(LogOffsetMetadata that) {
+ if (this.segmentBaseOffset == REMOTE_LOG_UNKNOWN_OFFSET ||
that.segmentBaseOffset == REMOTE_LOG_UNKNOWN_OFFSET)
+ return 1;
Review Comment:
It's a bit hacky to do this here. I was thinking of doing this in
DelayedFetch.
```
if (endOffset.messageOffset != fetchOffset.messageOffset) {
if (endOffset.messageOnly() || fetchOffset.messageOnly()) {
accumulatedSize += 1
} else if (endOffset.onOlderSegment(fetchOffset)) {
// Case F, this can happen when the new fetch operation is
on a truncated leader
...
}
```
##########
core/src/main/scala/kafka/log/UnifiedLog.scala:
##########
@@ -1428,7 +1428,11 @@ class UnifiedLog(@volatile var logStartOffset: Long,
*/
private def convertToOffsetMetadataOrThrow(offset: Long): LogOffsetMetadata
= {
checkLogStartOffset(offset)
- localLog.convertToOffsetMetadataOrThrow(offset)
+ if (remoteLogEnabled() && offset < localLogStartOffset()) {
+ new LogOffsetMetadata(offset,
LogOffsetMetadata.REMOTE_LOG_UNKNOWN_OFFSET)
+ } else {
+ localLog.convertToOffsetMetadataOrThrow(offset)
Review Comment:
I was thinking that we could change
localLog.convertToOffsetMetadataOrThrow() such that if read() throws an
exception, it just returns a message-only LogOffsetMetadata.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
