Re: [I] Move vector search from IndexInput to RandomAccessInput [lucene]

2024-12-04 Thread via GitHub
jpountz commented on issue #13938: URL: https://github.com/apache/lucene/issues/13938#issuecomment-2517680513 I was thinking of it differently, that `IndexInput` is for sequential reading (possibly with skipping, like we do in postings) while `RandomAccessInput` is for random access like we

Re: [I] Move vector search from IndexInput to RandomAccessInput [lucene]

2024-12-04 Thread via GitHub
rmuir commented on issue #13938: URL: https://github.com/apache/lucene/issues/13938#issuecomment-2517601394 @jpountz is this really appropriate? RandomAccessInput is to reduce the overhead when doing tiny (not bulk) reads, it was added to help move from fieldcache to docvalues, where you ne

Re: [I] Move vector search from IndexInput to RandomAccessInput [lucene]

2024-11-13 Thread via GitHub
dungba88 commented on issue #13938: URL: https://github.com/apache/lucene/issues/13938#issuecomment-2475621958 @jpountz it's only a draft (I need to add tests), but can you give some feedbacks on https://github.com/apache/lucene/pull/13981. I'm not sure if I have fully captured the intentio

Re: [I] Move vector search from IndexInput to RandomAccessInput [lucene]

2024-11-05 Thread via GitHub
dungba88 commented on issue #13938: URL: https://github.com/apache/lucene/issues/13938#issuecomment-2458667635 I looked at some implementation of RandomAccessInput, such as BufferedIndexInput. This particular class holds a single buffer for all reads, thus it cannot be shared. If we use tem

Re: [I] Move vector search from IndexInput to RandomAccessInput [lucene]

2024-10-31 Thread via GitHub
dungba88 commented on issue #13938: URL: https://github.com/apache/lucene/issues/13938#issuecomment-2451355944 > I think this will be helpful since currently we cannot share these readers across threads -- they retain the state information about the current position. Not sure how much benef

Re: [I] Move vector search from IndexInput to RandomAccessInput [lucene]

2024-10-31 Thread via GitHub
msokolov commented on issue #13938: URL: https://github.com/apache/lucene/issues/13938#issuecomment-2449719834 I think this will be helpful since currently we cannot share these readers across threads -- they retain the state information about the current position. Not sure how much benefi

Re: [I] Move vector search from IndexInput to RandomAccessInput [lucene]

2024-10-31 Thread via GitHub
dungba88 commented on issue #13938: URL: https://github.com/apache/lucene/issues/13938#issuecomment-2449356939 Hi, I'm learning Lucene KNN and this seems to be a workable PR for beginner. Just curious about the motivation behind this change. Is it only for cleaner code, or are we also suppo

[I] Move vector search from IndexInput to RandomAccessInput [lucene]

2024-10-21 Thread via GitHub
jpountz opened a new issue, #13938: URL: https://github.com/apache/lucene/issues/13938 ### Description Vector search currently loads vectors from disk by issuing a `seek()` followed by a `readFloats()`. We should instead: - Add an absolute `readFloats()` method to `RandomAccessInp