goankur commented on PR #16771: URL: https://github.com/apache/lucene/pull/16771#issuecomment-6183917976
> There is currently only one problem: If the index uses CFS files, then the whole optimization does not help, as CFS file boundaries add an additional alignment... So generally, if we have CFS files enabled, the whole alignment could be left out and the previous approach is better. Just saying this, I am not a big fan of this PR, but I would let it go. @uschindler Agreed. The 4096 is relative to the start of .vec, and inside a CFS each inner file starts at a 64-byte boundary, so there the vector data ends up only 64-byte aligned in the .cfs file. That's the same as float32 today, and better than the 4 bytes FLOAT16 and BYTE get today, so CFS segments don't gain from this change but don't lose either. By default only segments up to 64 MB are packed, so in a large index they hold a small share of the vectors. The writer also can't tell at write time whether a segment will end up in a CFS, so it can't skip the padding there. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
