vigyasharma commented on PR #11796: URL: https://github.com/apache/lucene/pull/11796#issuecomment-1255778326
> An alternative implementation would be to add the bytes only in the `IndexOutput.close` method instead of on each method that writes bytes? It might be less error-proned, but, also less real-time since it won't be until the file is closed that we count any bytes in the shared counters. I'm a bit conflicted about this. I like the completeness we get after `close()` is called. But as an API, consumers now have to be careful with `getApproximateWriteAmplificationFactor()`.. There is nothing stopping them from calling it before IndexOutput is closed. The counter will just return 0. Or the value it held before it was reused. The onus of ensuring close() was called is on the caller here i.e. the dir wrapper. However, in the dir wrapper, we don't keep references to different IndexOutputs for flush and merge. We directly read the counter values in getApproximateWriteAmplificationFactor(), and so there's no way to throw an error, if someone calls write amplification early. In other words, we cannot ensure that when write amplification returns 1, it really is 1. It could be because IndexOutput is still open. Maybe, we should let BytesTrackingIndexOutput expose a `bytesWritten()` method, which internally verifies that close was called. A subsequent real-time writes impl. could change this. The dir. wrapper would then keep IndexOutput references around, and use them instead of directly reading counters. Then we don't need to pass shared Atomic counters. We can directly aggregate values across IndexOutput references if we want to. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
