benwtrent commented on issue #13519: URL: https://github.com/apache/lucene/issues/13519#issuecomment-2206891266
@MilindShyani I am not 100% your equation is correct. (NOTE: `scale` is just the inverse of alpha). It seems to me that: ``` byte = (float - minQuantile) * alpha - c ``` This implies that float is actually: ``` scale(byte + c) + minQuantile = Y ``` So that this is more readable, lets' assume our two `byte` values are `x` & `y`, `m=minQuantile`, and `alpha=a` and the corrective constant is `c` Expanded out algebra: ``` a^2 c^2 + a^2 c x + a^2 c y + a^2 x y + 2 a c m + a m x + a m y + m^2 ``` `a^2 c^2` can be pre-calculated. Now, since both correction offsets will have this, I will simply divid it by `2` `a^2 c x` & `a^2 c y` can be pre-calculated and added `2 a c m` is precalculated, but since both corrections will have it, lets divide by 2 again. Let's make the assumption that the other values are already calculated. I am getting positive results with this. I will return back once I run some tests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
