jaisonbi commented on a change in pull request #2213:
URL: https://github.com/apache/lucene-solr/pull/2213#discussion_r565767001
##########
File path:
lucene/core/src/java/org/apache/lucene/codecs/lucene80/Lucene80DocValuesConsumer.java
##########
@@ -790,6 +833,28 @@ private void addTermsDict(SortedSetDocValues values)
throws IOException {
writeTermsIndex(values);
}
+ private int compressAndGetTermsDictBlockLength(
+ ByteArrayDataOutput bufferedOutput, LZ4.FastCompressionHashTable ht)
throws IOException {
+ int uncompressedLength = bufferedOutput.getPosition();
+ data.writeVInt(uncompressedLength);
+ long before = data.getFilePointer();
+ LZ4.compress(termsDictBuffer, 0, uncompressedLength, data, ht);
+ int compressedLength = (int) (data.getFilePointer() - before);
+ // Block length will be used for creating buffer for decompression, one
corner case is that
+ // compressed length might be bigger than un-compressed length, so just
return the bigger one.
+ return Math.max(uncompressedLength, compressedLength);
+ }
+
+ private ByteArrayDataOutput maybeGrowBuffer(ByteArrayDataOutput
bufferedOutput, int termLength) {
+ int pos = bufferedOutput.getPosition(), originalLength =
termsDictBuffer.length;
+ if (pos + termLength >= originalLength - 1) {
+ int targetLength = (originalLength + termLength) << 1;
Review comment:
I just worry about rely on ArrayUtil.grow is slower..I re-tested this
method and try to understand the growing behavior..
suppose the original length is 16 * 1024, and the termLength is 112...so the
new length will be: 18560
so remove line 851 makes sense to me. will change it immediately. Thanks
@bruno-roustant
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
