dsmiley opened a new pull request, #12306: URL: https://github.com/apache/lucene/pull/12306
The principle objective of this PR is to make it easier for users to use Lucene with higher dimensions. While it's [technically possible to circumvent the existing limit](https://lists.apache.org/thread/pc8280kn99s0lf2gjd50chk0nftzmzmt), it's a non-obvious awkward hack that either the user would need to figure out or would be baked into higher level search platforms. It's so obscure that most Lucene committers didn't even know it was possible! Thus the practical effect for users now is that it's not possible so they don't use Lucene. The system property proposed here is "lucene.hnsw.maxDimensions". I also deprecated the field in anticipation that it will move to a codec specific place. Regardless of if/when that happens, I don't think we want to advertise this limit where it is now, which is at a surface level Lucene API with a present value based on the default codec that may not make sense for other vector codecs. Moving that is out of scope of this PR. AFAICT, this limit is merely an ergonomics kind of limit to help the user from shooting themselves in the foot. The underlying codec we have can read/write an arbitrary number of dimensions. CheckIndex validates the number of dimensions is greater than zero but doesn't enforce a maximum. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
