Re: conversion from double to fixed and back fails with certain values

2013-09-26 Thread Jason Ekstrand
Micah, It simply means that you can't assume too much precision. For most things such as pointer movement this isn't a problem because there is no good reason why you would want to know the pointer's position more precicely than in units 1/256 of a pixel. Recently, however, there was some discuss

Re: conversion from double to fixed and back fails with certain values

2013-09-26 Thread Micah Nordland
OK, I are there any parts of the Wayland spec where this might cause problems/make things interesting? -- Sent from my Android device with K-9 Mail. Please excuse my brevity.___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists

Re: conversion from double to fixed and back fails with certain values

2013-09-26 Thread Jason Ekstrand
Micah, This is because wl_fixed is a fixed-point format. In particular, it is 24.8 fixed point meaning that the top 24 bits represent the integer part. This means that wl_fixed effectively stores n if the number is written as the (possibly improper) fraction n/256. In your example, 3568.005 = 91

conversion from double to fixed and back fails with certain values

2013-09-26 Thread Micah Nordland
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 The value 3568.005 is not correctly converted back from fixed. The following patch to tests/fixed-test.c demonstrates. Is this expected behavior? diff --git a/tests/fixed-test.c b/tests/fixed-test.c index 739a3b1..89ec188 100644 - --- a/tests/fixed-te