Pardon me for the delayed response. I was on a vacation. Thank you very much for the reply. As I told I am new to Wayland (indeed new to Display server). I am going through the documentation of the Wayland which is available in the website.
My understanding of Wayland is very little, so I won't be able to develop the test suite myself. I will now start going through the code and logs to understand how Wayland actually works. If any of you can guide me initially, I would be happy to develop. Thank-you, Sri On Thu, Apr 17, 2014 at 12:23 PM, Pekka Paalanen <[email protected]>wrote: > On Wed, 16 Apr 2014 20:41:50 +0000 > "Bryce W. Harrington" <[email protected]> wrote: > > > On Wed, Apr 16, 2014 at 08:51:21AM +0300, Pekka Paalanen wrote: > > > On Tue, 15 Apr 2014 19:08:12 +0000 > > > "Bryce W. Harrington" <[email protected]> wrote: > > > > > > > Maybe adding more tests? Any particular features recently added that > > > > could benefit from tests added? > > > > > > Yes! That is a good one. Either adding completely new tests, or maybe > > > working with Marek Chalupa in converting all possible FAIL_TEST cases > to > > > normal TEST cases, as permitted by Marek's > > > > http://lists.freedesktop.org/archives/wayland-devel/2014-April/014190.html > > > > > > I can't list any features off-hand that would need new tests which > would > > > be easy to implement. > > > > Would there be value to just basic unit tests on some of the established > > API's? E.g. like take xdg-shell's API, check passing in negative or > > zero dimentions, crazy long strings for string params, stuff like that. > > A lot of the protocol's code is generated so it's unclear to me whether > > this level of testing would be useful or not, but if it is, it tends to > > be pretty newbie-friendly. > > Even when the wrappers are generated, argument validity checks are > written manually, so yes, there would be value in writing those. > > Another easy bit is testing object destruction orders for somehow > related objects. Like if you create a wl_viewport for wl_surface, and > then the wl_surface is destroyed, the wl_viewport should be inert and > using it should only be ignored in the compositor, not cause crashes or > protocol errors. "Inert" is a specific term in the specification > language for exactly that. > > The sub-surface tests do extensive destroy order testing, but I think > it might lack testing the "ignore" part, and I'm sure there would be > other places to add more such tests. > > Also, every defined protocol error in the protocol specifications should > be excercised by a test, that should be easy, too. > > However, I'm not sure if xdg_shell particularly is a good target at the > moment, as I think there might still be changes coming? > > > > But I guess we should at some point hook up a > > > real renderer to the headless backend, and add infrastructure for the > > > test framework to do screencaptures, so we could test also rendering > > > and things like window stacking and surface/buffer transformations. > > > > That sounds keen, and could open up a lot of testing potential. Shame > > it's not already in place! Could you elaborate on how you think this > > could be created? I'd like to explore this further myself. > > Well, there is the test protocol extension, you could add > client-initiated screenshooting feature to it, perhaps on a designated > sub-rectangle. > > Easiest for headless would probably be to start by plugging in the > pixman renderer, and defining a hardcoded single-resolution output. > > Since tests also run the desktop-shell client which draws "random" > things like takes colors from configs and paints the clock, maybe it > could on init check if the test extension is advertised by the > compositor. If the test extension is there, desktop-shell client would > use only hardcoded configs, and paint a hardcoded time in the clock, > etc. so that the rendered output should always be the same. > > Window initial position randomization in shell.c might be a problem. > > You'd also need to save the screenshots into image files, so a > developer can check the result. > > > > Pixman's fuzzer test might be a good inspiration on how to actually do > > > the tests once the infrastructure is up: > > > http://lists.freedesktop.org/archives/pixman/2014-April/003229.html > > > Provided we can guarantee pixel-perfect rendering, which might not > > > always be true, so a checksum-based checking might not always work. > > > > I take it this is similar to the Cairo testsuite's perceptual diff > > stuff? (Checksumming two bitmaps pixel by pixel) > > I suppose, though I haven't looked at Cairo. > > > What we're seeing in Cairo is that sometimes pixel differences indicate > > issues, but a lot of time it's ignorable, and deciding which is which is > > fairly hard. But used in situations where you are 100% sure exactly > > what the images should look like, this can be quite effective. > > Yes, Pixman does the same separation. Some tests require bit-perfect > results, others are known to have a little variance. > > I suppose with a Pixman renderer, we can have the guarantees Pixman > has since we know which operations we use (or do we?), but with a > gl-renderer, or say, rpi-renderer, I don't think we can require > bit-perfect reproduction. > > We can already choose the backend to use when running 'make check', we > could have the headless backend expose pixman renderer by default and > software rendered gl-renderer as an option. > > > In practice, just having all the different rendering tests can be a > > handy workload for exposing crashes. I suspect coupling it with some > > timing measurements it might also help expose performance regressions. > > I don't think we can make automated performance testing too reliable. I > wouldn't want a build-bot for some distribution to fail 'make check' > just because it runs in a virtual machine on some heavily loaded > hardware. > > So performance testing would be a separate case. > > > > But I think that is getting quite heavy for just introductory work. > > > > For something like this, 80% of the work is just deciding on a good > > design. Most of the implementation work is going to be basic coding. > > There's going to be a few hard tasks (like attaching a snapshot renderer > > functionality to the headless backend), but even those are likely to get > > owners if they're well enough described. > > Right. We need someone to put in the infrastructure. :-) > > > Thanks, > pq >
_______________________________________________ wayland-devel mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/wayland-devel
