benwtrent commented on code in PR #13124:
URL: https://github.com/apache/lucene/pull/13124#discussion_r1499916055
##########
lucene/core/src/java/org/apache/lucene/codecs/lucene99/Lucene99HnswVectorsFormat.java:
##########
@@ -152,7 +153,25 @@ public Lucene99HnswVectorsFormat() {
* @param beamWidth the size of the queue maintained during graph
construction.
*/
public Lucene99HnswVectorsFormat(int maxConn, int beamWidth) {
- this(maxConn, beamWidth, DEFAULT_NUM_MERGE_WORKER, null);
Review Comment:
@zhaih I see what you are saying about the weighing of different codecs.
Where certain codecs could merged in parallel using a different number of
threads. My concern is how this would ever be effectively communicate to the
merge scheduler.
While it isn't a 1-1 relationship, adding more threads here in HNSW will
increase I/O. I assume that for other codecs that could use parallelism,
additional I/O will also be a concern.
The scheduler needs to know how to throttle I/O if this gets too large so
that merges and indexing can be smoothed out correct?
This indicates to me the scheduler should be in charge of handing out
threads to do work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
