jpountz commented on PR #13337: URL: https://github.com/apache/lucene/pull/13337#issuecomment-2103397669
> also, i'm a little concerned about low-level parallelization of e.g. individual stored documents. seems like a lot of overhead! if you need 10,000 documents ranges, at least make a single mincore() call first before issuing 10,000 madvise() calls? This sounds like a good idea. If a user wants to return 10k stored documents, I wonder if we should also split this into smaller batches to avoid running into a case when some pages from the cache gets claimed by something else before we have a chance to retrieve all these stored documents. > we should think of adding another random API to RandomAccessIndexInput, something like Thanks Uwe, this sounds like a good suggestion. I'll start looking into using this API for terms dictionary lookups of boolean queries of term queries, which I don't think would need it since we'd do a single seek per clone anyway. But when we later move to vectors, stored fields and term vectors this could be useful. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
