benwtrent commented on PR #15090: URL: https://github.com/apache/lucene/pull/15090#issuecomment-3292210203
> I guess people working in Java are here at least in part for the type-safety and compile-time checking, but perhpas we could create a format - builder? Java developers love builders, right :)? Right LOL > FAISS expects its users to provide a complex string describing the format it supports including the algorithm name, the parameters specific to that algorithm, pre-steps like dimensionality reduction and quantization, and even ganging together multiple steps in case there is a re-scoring phase.... Yeah, I definitely don't want to do that. I think something pretty simple, like simply writing "inner" names to the metadata and using the named loader works pretty well. Then things that provide more "options" can provide those options via some nicer API (whatever that is, possibly just different ctors). This seems pretty nice to me (vs. what we have now, fully fqdn stuff): ``` new Lucene99Hnsw(int M, ..., new Lucene104Quantized(int bits, ..., new Lucene99FlatRaw(...))); ``` Then the writers keep track of their inner values, which recursively keep track in their own metadata. Of course, we can make the format declarations nicer (e.g. a builder). But that is just fancy `ctor`s. My concern is that: Are we cool with writing that stuff to metadata and loading? If so, I can make a chnage this week so we can see it in action and then go through the process of adding a new HNSW format that satisfies this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
