On Sat, Feb 11, 2017 at 09:58:10AM +0100, Richard Cochran wrote:
> If I am not mistaken, then you can skip the cases val==2 and val==3,
> because they are equivalent to val==4 and 6.
I took a stab at this, and you can see the result, below. My version
has lower average error than yours in the interval 1 < ppb < 60000,
and it uses only 8 64-bit divisions. Outside of that interval, your
version has lower error.
So, at the very least, you should introduce a threshold and use this
algorithm for adjustments under 60 ppm. Better yet, find a way to use
fewer divisions for adjustments greater and 60 ppm...
Thanks,
Richard
---
#include <stdint.h>
#define TEN9 (1000000000UL)
unsigned int calc_min_integer(uint64_t ppb, uint64_t *M)
{
uint64_t err, m, min, n, N, p2, reg;
min = TEN9;
for (n = 4; n <= 7; n++) {
m = n * TEN9;
m = (m + ppb/2) / ppb;
/*truncate to HW resolution*/
reg = (m + 8) / 16;
m = reg * 16;
p2 = (n * TEN9 + m/2) / m;
err = ppb > p2 ? ppb - p2 : p2 - ppb;
if (min >= err) {
min = err;
N = n;
*M = m;
}
}
return N;
}
