dsmiley opened a new pull request, #12306:
URL: https://github.com/apache/lucene/pull/12306

   The principle objective of this PR is to make it easier for users to use 
Lucene with higher dimensions.  While it's [technically possible to circumvent 
the existing 
limit](https://lists.apache.org/thread/pc8280kn99s0lf2gjd50chk0nftzmzmt), it's 
a non-obvious awkward hack that either the user would need to figure out or 
would be baked into higher level search platforms.  It's so obscure that most 
Lucene committers didn't even know it was possible!  Thus the practical effect 
for users now is that it's not possible so they don't use Lucene.
   
   The system property proposed here is "lucene.hnsw.maxDimensions".
   
   I also deprecated the field in anticipation that it will move to a codec 
specific place.  Regardless of if/when that happens, I don't think we want to 
advertise this limit where it is now, which is at a surface level Lucene API 
with a present value based on the default codec that may not make sense for 
other vector codecs.  Moving that is out of scope of this PR.
   
   AFAICT, this limit is merely an ergonomics kind of limit to help the user 
from shooting themselves in the foot.  The underlying codec we have can 
read/write an arbitrary number of dimensions.  CheckIndex validates the number 
of dimensions is greater than zero but doesn't enforce a maximum.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

Reply via email to