ChrisHegarty commented on code in PR #13985:
URL: https://github.com/apache/lucene/pull/13985#discussion_r1842566686
##
lucene/core/src/java/org/apache/lucene/codecs/KnnVectorsReader.java:
##
@@ -123,4 +123,11 @@ public abstract void search(
public KnnVectorsReader getMergeIn
ChrisHegarty commented on code in PR #13985:
URL: https://github.com/apache/lucene/pull/13985#discussion_r1842566686
##
lucene/core/src/java/org/apache/lucene/codecs/KnnVectorsReader.java:
##
@@ -123,4 +123,11 @@ public abstract void search(
public KnnVectorsReader getMergeIn
shatejas commented on code in PR #13985:
URL: https://github.com/apache/lucene/pull/13985#discussion_r1842587936
##
lucene/core/src/java/org/apache/lucene/codecs/KnnVectorsReader.java:
##
@@ -123,4 +123,11 @@ public abstract void search(
public KnnVectorsReader getMergeInstan
shatejas commented on code in PR #13985:
URL: https://github.com/apache/lucene/pull/13985#discussion_r1842587936
##
lucene/core/src/java/org/apache/lucene/codecs/KnnVectorsReader.java:
##
@@ -123,4 +123,11 @@ public abstract void search(
public KnnVectorsReader getMergeInstan
ChrisHegarty commented on code in PR #13985:
URL: https://github.com/apache/lucene/pull/13985#discussion_r1842566686
##
lucene/core/src/java/org/apache/lucene/codecs/KnnVectorsReader.java:
##
@@ -123,4 +123,11 @@ public abstract void search(
public KnnVectorsReader getMergeIn
ChrisHegarty commented on code in PR #13985:
URL: https://github.com/apache/lucene/pull/13985#discussion_r1842566686
##
lucene/core/src/java/org/apache/lucene/codecs/KnnVectorsReader.java:
##
@@ -123,4 +123,11 @@ public abstract void search(
public KnnVectorsReader getMergeIn
shatejas commented on PR #13985:
URL: https://github.com/apache/lucene/pull/13985#issuecomment-2476919288
> Generally, I think that the direction in this PR is good. I wanna help get
it moved forward. I'll do some local testing and perf runs to verify the
impact. I can also commit some test
ChrisHegarty commented on PR #13985:
URL: https://github.com/apache/lucene/pull/13985#issuecomment-2476880937
Generally, I think that the direction in this PR is good. I wanna help get
it moved forward. I'll do some local testing and perf runs to verify the
impact. I can also commit some te
ChrisHegarty commented on PR #13985:
URL: https://github.com/apache/lucene/pull/13985#issuecomment-2476285629
Thanks @shatejas
For clarity, the bottleneck that is being fixed here is with the reading of
all vector data from the to-be-merged segments, when copying that data to the
ne
ChrisHegarty commented on code in PR #13985:
URL: https://github.com/apache/lucene/pull/13985#discussion_r1842167895
##
lucene/core/src/java/org/apache/lucene/codecs/KnnVectorsReader.java:
##
@@ -123,4 +123,11 @@ public abstract void search(
public KnnVectorsReader getMergeIn
shatejas commented on PR #13985:
URL: https://github.com/apache/lucene/pull/13985#issuecomment-2474147590
> I'm curious how much this actually helps, and I know that you said that
benchmark results would be posted.
@ChrisHegarty Preliminary results showed approximately 40mins (~13%)
uschindler commented on PR #13985:
URL: https://github.com/apache/lucene/pull/13985#issuecomment-2471068437
Hi,
I am currently on travel, so I can't review this. Will look into it posisbly
later this week. Greetings from Costa Rica!
--
This is an automated message from the Apache Git S
ChrisHegarty commented on PR #13985:
URL: https://github.com/apache/lucene/pull/13985#issuecomment-2470311965
@shatejas I'm curious how much this actually helps, and I know that you said
that benchmark results would be posted.
I do like that we can update the ReadAdvice on an index in
shatejas commented on code in PR #13985:
URL: https://github.com/apache/lucene/pull/13985#discussion_r1836913408
##
lucene/core/src/java/org/apache/lucene/codecs/lucene99/Lucene99HnswVectorsReader.java:
##
@@ -113,6 +114,25 @@ public Lucene99HnswVectorsReader(SegmentReadState st
shatejas commented on code in PR #13985:
URL: https://github.com/apache/lucene/pull/13985#discussion_r1836001654
##
lucene/test-framework/src/java/org/apache/lucene/tests/store/BaseDirectoryTestCase.java:
##
@@ -1554,6 +1555,42 @@ public void testPrefetchOnSlice() throws IOExcep
shatejas commented on code in PR #13985:
URL: https://github.com/apache/lucene/pull/13985#discussion_r1836000686
##
lucene/core/src/java/org/apache/lucene/codecs/lucene99/Lucene99FlatVectorsReader.java:
##
@@ -327,4 +336,61 @@ static FieldEntry create(IndexInput input, FieldInfo
shatejas commented on code in PR #13985:
URL: https://github.com/apache/lucene/pull/13985#discussion_r1835999858
##
lucene/core/src/java/org/apache/lucene/codecs/lucene99/Lucene99HnswVectorsReader.java:
##
@@ -113,6 +114,25 @@ public Lucene99HnswVectorsReader(SegmentReadState st
shatejas commented on code in PR #13985:
URL: https://github.com/apache/lucene/pull/13985#discussion_r1835999756
##
lucene/core/src/java/org/apache/lucene/codecs/KnnVectorsReader.java:
##
@@ -123,4 +123,11 @@ public abstract void search(
public KnnVectorsReader getMergeInstan
shatejas commented on code in PR #13985:
URL: https://github.com/apache/lucene/pull/13985#discussion_r1835999756
##
lucene/core/src/java/org/apache/lucene/codecs/KnnVectorsReader.java:
##
@@ -123,4 +123,11 @@ public abstract void search(
public KnnVectorsReader getMergeInstan
jpountz commented on code in PR #13985:
URL: https://github.com/apache/lucene/pull/13985#discussion_r1835530675
##
lucene/core/src/java/org/apache/lucene/codecs/lucene99/Lucene99FlatVectorsReader.java:
##
@@ -170,6 +170,15 @@ public void checkIntegrity() throws IOException {
shatejas commented on PR #13985:
URL: https://github.com/apache/lucene/pull/13985#issuecomment-2466151296
cc: @uschindler @jpountz @navneet1v
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
shatejas opened a new pull request, #13985:
URL: https://github.com/apache/lucene/pull/13985
reading from IndexInput
The change is needed to be able to reduce the force merge time.
Lucene99FlatVectorsReader is opened with IOContext.RANDOM, this optimizes
searches with madvise as RAND
22 matches
Mail list logo
