On Sat, Feb 11, 2017 at 09:58:10AM +0100, Richard Cochran wrote:
> If I am not mistaken, then you can skip the cases val==2 and val==3,
> because they are equivalent to val==4 and 6.

I took a stab at this, and you can see the result, below.  My version
has lower average error than yours in the interval 1 < ppb < 60000,
and it uses only 8 64-bit divisions.  Outside of that interval, your
version has lower error.

So, at the very least, you should introduce a threshold and use this
algorithm for adjustments under 60 ppm.  Better yet, find a way to use
fewer divisions for adjustments greater and 60 ppm...

Thanks,
Richard

---
#include <stdint.h>

#define TEN9            (1000000000UL)

unsigned int calc_min_integer(uint64_t ppb, uint64_t *M)
{
        uint64_t err, m, min, n, N, p2, reg;

        min = TEN9;
        for (n = 4; n <= 7; n++) {
                m = n * TEN9;
                m = (m + ppb/2) / ppb;

                /*truncate to HW resolution*/
                reg = (m + 8) / 16;
                m = reg * 16;

                p2 = (n * TEN9 + m/2) / m;

                err = ppb > p2 ? ppb - p2 : p2 - ppb;

                if (min >= err) {
                        min = err;
                        N = n;
                        *M = m;
                }
        }
        return N;
}

Reply via email to