msokolov opened a new pull request, #913: URL: https://github.com/apache/lucene/pull/913
This patch revisits the idea of storing vectors in 8-bit precision. It incorporates some @rcmuir and @jpountz suggestions from https://github.com/apache/lucene/pull/899 which was the first attempt. The highlights here are: 1. Adds a new VectorSimilarity to represent this use case 2. Expects vectors to be supplied (as float[]) with values in the range [-128,127] 3. All dot-product operations are done over byte using integer math when this similarity is used The good parts: Index size is reduced; I ran a test with 100K (200-dimensional) GloVe vectors and the index size goes from 231M to 173M for an M=200 graph and 154M->97M for an M=100 graph. The vector portion of the storage (.vec files) is reduced nearly 75% as expected. With careful tuning of the scaling factor, recall is pretty much unchanged compared to the full-precision baseline. I think these GloVe vectors are not well-conditioned numerically, so to do the scaling I did not use min/max values as the limits, rather I discarded outlying values and used 0.1/0.99 pctile values. Indexing and search performance as measured with KnnGraphTester improved somewhere in the range of 15-25%. I'd like to run luceneutil and/or ann-benchmarks to get another look, but this was a very pleasant surprise. I think the prospect is bright for vectorization too, but this improvement is achievable using our scalar impl. The not-so-good-parts: I think we probably wouldn't want to commit as-is since it would make changes to the codec that aren't forwards-compatible. To go forward I guess we'd need to create a new version of the HnswVectorsFormat. I had to add some branches and generics to handle the two different data representations. I don't think it's too terribly ugly, but I know we've had concerns about this kind of thing in the past - maybe there's a better way that doesn't require massive code duplication? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
