[
https://issues.apache.org/jira/browse/LUCENE-10004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17410114#comment-17410114
]
kkewwei commented on LUCENE-10004:
----------------------------------
Yes, before `bulk` merge, we should `flush` to make it clean.
Also We should reduce the frequency of `flush`, we can use a flag to record
whether the last merge is `bulk` or not, if it' not `bulk` merge, we needn't to
flush it.
We can reduce the `flush` rate by the way, but I don’t know how effective it is.
> Delete unnecessary flush in
> Lucene90CompressingStoredFieldsWriter.copyChunks() to reduce dirty chunks
> -----------------------------------------------------------------------------------------------------
>
> Key: LUCENE-10004
> URL: https://issues.apache.org/jira/browse/LUCENE-10004
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/codecs
> Affects Versions: 8.8.2
> Reporter: kkewwei
> Priority: Major
>
> In CompressingStoredFieldsWriter.merge(): if the segment meet the following
> conditions:
> {code:java}
> else if (matchingFieldsReader.getCompressionMode() == compressionMode &&
> matchingFieldsReader.getChunkSize() == chunkSize &&
> matchingFieldsReader.getPackedIntsVersion()
> ==PackedInts.VERSION_CURRENT &&
> liveDocs == null &&
> !tooDirty(matchingFieldsReader)) {
> ......
> // flush any pending chunks
> if (numBufferedDocs > 0) {
> flush();
> numDirtyChunks++; // incomplete: we had to force this flush
> }
> ......
> }
> {code}
> We will copy the the all chunk to the new fdt, before copying the chunk, we
> will flush the buffer docs if numBufferedDocs >0, but the flush is
> unnecessary.
> The bufferedDocs in memory have nothing to do with copyChunk. We just need to
> ensure that it will be flush at the end of merge(In finish()).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
