benwtrent commented on PR #14932:
URL: https://github.com/apache/lucene/pull/14932#issuecomment-3058632703
Hi @aylonsk ! Thank you for digging into this issue. I am sure you are still
working on it, but I had some feedback:
- It would be interesting to get statistics around resulting index size
changes and performance changes (index & search). Lucene util is the preferred
tool for this: https://github.com/apache/lucene/pull/14932
- As with most Lucene formats, changes like this need to be backwards
compatible. Readers are loaded via their names. Consequently, users might have
indices with the Lucene99Hnsw format name that do not have group-varint
applied, and consequently cannot be read by your change here. There are a
couple of options to handle this:
- Add versioning to the format
- Create a new format (Lucene103Hnsw...) and move Lucene99Hnsw... to the
bwc formats package for readers (there are many example PRs in the past doing
this).
Handling the format change can be complicated. So, my first step would be to
justify the change with performance metrics. Then do all the complicated format
stuff.
Good luck!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
