msokolov opened a new pull request, #913:
URL: https://github.com/apache/lucene/pull/913

   This patch revisits the idea of storing vectors in 8-bit precision. It 
incorporates some @rcmuir and @jpountz suggestions from 
https://github.com/apache/lucene/pull/899 which was the first attempt. The 
highlights here are:
   
   1. Adds a new VectorSimilarity to represent this use case
   2. Expects vectors to be supplied (as float[]) with values in the range 
[-128,127]
   3. All dot-product operations are done over byte using integer math when 
this similarity is used
   
   The good parts:
   
   Index size is reduced; I ran a test with 100K (200-dimensional) GloVe 
vectors and the index size goes from 231M to 173M for an M=200 graph and 
154M->97M for an M=100 graph. The vector portion of the storage (.vec files) is 
reduced nearly 75% as expected.
   
   With careful tuning of the scaling factor, recall is pretty much unchanged 
compared to the full-precision baseline. I think these GloVe vectors are not 
well-conditioned numerically, so to do the scaling I did not use min/max values 
as the limits, rather I discarded outlying values and used 0.1/0.99 pctile 
values.
   
   Indexing and search performance as measured with KnnGraphTester improved 
somewhere in the range of 15-25%. I'd like to run luceneutil and/or 
ann-benchmarks to get another look, but this was a very pleasant surprise. I 
think the prospect is bright for vectorization too, but this improvement is 
achievable using our scalar impl.
   
   The not-so-good-parts:
   
   I think we probably wouldn't want to commit as-is since it would make 
changes to the codec that aren't forwards-compatible. To go forward I guess 
we'd need to create a new version of the HnswVectorsFormat.
   
   I had to add some branches and generics to handle the two different data 
representations. I don't think it's too terribly ugly, but I know we've had 
concerns about this kind of thing in the past - maybe there's a better way that 
doesn't require massive code duplication?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to