On 20/02/14 12:07, Pekka Paalanen wrote:
Hi Mario,
Ok, now i am magically subscribed. Thanks to the moderator!
I have replies to your comments below, but while reading what you said,
I started wondering whether Wayland would be good for you after all.
It seems that your timing sensitive experiment programs and you and
your developers use a great deal of effort into
- detecting the hardware and drivers,
- determining how the display server works, so that you can
- try to make it do exactly what you want, and
- detect if it still does not do exactly like you want it and bail,
while also
- trying to make sure you get the right timing feedback from the kernel
unmangled.
Yes. It's "trust buf verify". If i know that the api / protocol is well
defined and suitable for my purpose and have verified that at least the
reference compositor implements the protocol correctly then i can at
least hope that all other compositors are also implemented correctly, so
stuff should work as expected. And i can verify that at least some
subset of compositors really works, and try to submit bug reports or
patches if they don't.
Sounds like the display server is a huge source of problems to you, but
I am not quite sure how running on top a display server benefits you.
Your experiment programs want to be in precise control, get accurate
timings, and they are always fullscreen. Your users / test subjects
never switch away from the program while it's running, you don't need
windowing or multi-tasking, AFAIU, nor any of the application
interoperability features that are the primary features of a display
server.
They are fullscreen and timing sensitive in probably 95% of all typical
application cases during actual "production use" while experiments are
run. But some applications need the toolkit to present in regular
windows and GUI thingys, a few even need compositing to combine my
windows with windows of other apps. Some setups run multi-display, where
some displays are used for fullscreen stimulus presentation to the
tested person, but another display may be used for control/feedback or
during debugging by the experimenter, in which case the regular desktop
GUI and UI of the scripting environment is needed on that display. One
popular case during debugging is having a half-transparent fullscreen
window for stimulus presentation, but behind that window the whole
regular GUI with the code editor and debugger in the background, so one
can set breakpoints etc. - The window is made transparent for mouse and
keyboard input, so users can interact with the editor.
So in most cases i need a display server running, because i sometimes
need compositing and i often need a fully functional GUI during at the
at least 50% of the work time where users are debugging and testing
their code and also don't want to be separated from their e-mail clients
and web browsers etc. during that time.
Why not take the display server completely out of the equation?
I understand that some years ago, it would probably not have been
feasible and X11 was the de facto interface to do any graphics.
However, it seems you are already married to DRM/KMS so that you get
accurate timing feedback, so why not port your experiment programs
(the framework) directly on top of DRM/KMS instead of Wayland?
Yes and no. DRM/KMS will be the most often used one and is the best bet
if i need timing control and it's the one i'm most familiar with. I also
want to keep the option of running on other backends if timing is not of
much importance, or if it can be improved on them, should the need arise.
With Mesa EGL and GBM, you can still use hardware accelerated openGL if
you want to, but you will also be in explicit control of when you
push that rendered buffer into KMS for display. Software rendering by
direct pixel poking is also possible and at the end you just push that
buffer to KMS as usual too. You do not need any graphics card specific
code, it is all abstracted in the public APIs offered by Mesa and
libdrm, e.g. GBM. The new libinput should make hooking into input
devices much less painful, etc. All this thanks to Wayland, because on
Wayland, there is no single "the server" like the X.org X server is.
There will be lots of servers and each one needs the same
infrastructure you would need to run without a display server.
No display server obfuscating your view to the hardware, no
compositing manager fiddling with your presentation, and most likely no
random programs hogging the GPU at random times. Would the trade-off
not be worth it?
I thought about EGL/GBM etc. as a last resort for especially demanding
cases, timing-wise. But given that the good old X-Server was good enough
for almost everything so far, i'd expect Wayland to perform as good as
or better timing-wise. If that turns out to be true, it would be good
enough for hopefully almost all use cases, with all the benefits of
compositing and GUI support when needed, and not having to reimplement
my own display server. E.g., i'm also using GStreamer as media backend
(still 0.10 though) and while there is Wayland integration, i doubt
there will be Psychtoolbox integration. Or things like Optimus-style
hybrid graphics laptops with one gpu rendering, the other gpu
displaying. I assume Wayland will tackle this stuff at some point,
whereas i'm not too keen to potentially learn and tackle all the ins and
outs of rendernodes or dmabuf juggling.
Of course your GUI tools and apps could continue using a display server
and would probably like to be ported to be Wayland compliant, I'm just
suggesting this for the sensitive experiment programs. Would this be
possible for your infrastructure?
Not impossible when absolutely needed, just inconvenient for the user +
yet another display backend to maintain and test for me. Psychtoolbox is
a set of extensions for both GNU/Octave and Mathworks Matlab - both use
the same scripting language, so users can choose which to use and have
portable code between them. Matlab uses a GUI based on Java/Awt/Swing on
top of X11, whereas Octave just gained a QT-based GUI beginning this
year. You can run both apps in a terminal or console, but most of my
users are mostly psychologists/neuro-biologists/physicians, and most of
them only have basic programming skills and are somewhat frightened by
command line environments. Most would touch a non-GUI environment only
in moments of highest despair, let alone learn how to switch between a
GUI environment and a console.
On Thu, 20 Feb 2014 04:56:02 +0100
Mario Kleiner <[email protected]> wrote:
On 17/02/14 14:12, Pekka Paalanen wrote:
On Mon, 17 Feb 2014 01:25:07 +0100
Mario Kleiner <[email protected]> wrote:
Hello Pekka,
i'm not yet subscribed to wayland-devel, and a bit short on time atm.,
so i'll take a shortcut via direct e-mail for some quick feedback for
your Wayland presentation extension v2.
Hi Mario,
I'm very happy to hear from you! I have seen your work fly by on
dri-devel@ (IIRC) mailing list, and when I was writing the RFCv2 email,
I was thinking whether I should personally CC you. Sorry I didn't. I
will definitely include you on v3.
I hope you don't mind me adding wayland-devel@ to CC, your feedback is
much appreciated and backs up my design nicely. ;-)
Hi again,
still not subscribed, but maybe the server accepts the e-mail to
wayland-devel anyway as i'm subscribed to other xorg lists? I got an
"invalid captcha" error when trying to subscribe, despite no captcha was
ever presented to me? You may need to cc wayland-devel again for me.
I guess the web interface is still down. Emailing
[email protected] should do, if I recall the
address correctly. And someone might process the moderation queue, too.
No worries anyway, I won't cut anything from you out, so it's all
copied below.
1. Wrt. an additional "preroll_feedback" request
<http://lists.freedesktop.org/archives/wayland-devel/2014-January/013014.html>,
essentially the equivalent of glXGetSyncValuesOML(), that would be very
valuable to us.
...
Indeed, the "preroll_feedback" request was modeled to match
glXGetSyncValuesOML.
Do you need to be able to call GetSyncValues at any time and have it
return ASAP? Do you call it continuously, and even between frames?
Yes, at any time, even between frames, with a return asap. This is
driven by the specific needs of user code, e.g., to poll for a vblank,
or to establish a baseline of current (msc, ust) for timing stuff
relative. Psychtoolbox is an extension to a scripting language, so
usercode often decides how this is used.
Internally to the toolkit these calls are used on X11/GLX to translate
target timestamps into target vblank counts for glXSwapBufferMscOML(),
because OML_sync_control is based on vblank counts, not absolute system
time, as you know, but ptb exposes an api where usercode specifies
target timestamps, like in your protocol proposal. The query is also
needed to work around some problems with the blocking nature of the X11
protocol when one tries to swap multiple independent windows with
different rates and uses glXWaitForSbcOML to wait for swap completion.
E.g., what doesn't work on X11 is using different x-display connections
- one per windows - and create GLX contexts which share resources across
those connections, so if you need multi-window operation you have to
create all GLX contexts on the same x-display connection and run all glx
calls over that connection. If you run multiple independent animations
in different windows you have to avoid blocking that x-connection, so i
use glXGetSyncValuesOML to poll current msc and ust to find out when it
is safe to do a blocking call.
I hope that the way Waylands protocol works will make some of these
hacks unneccessary, but user script code can call glXGetSyncValues at
any time.
Wayland won't allow you to use different connections even harder,
because there simply are no sharable references to protocol objects. But
OTOH, Wayland only provides you a low-level direct async interface to
the protocol as a library, so you will be doing all the blocking in your
app or GUI-toolkit.
Yes. If i understood correctly, with Wayland, rendering is separated
from the presentation, all rendering and buffer management client-side,
only buffer presentation server-side. So i hope i'll be able to untangle
everything rendering related (like hope many OpenGL contexts to have and
what resources they should or should not share) from everything
presentation related. If the interface is async and has thread-safety in
mind, that should hopefully help a lot on the presentation side.
The X11 protocol and at least XLib is not async and not thread-safe by
default, and at least under DRI2 the x-server controlled and owned the
back- and front-buffers, so you have lots of roundtrips and blocking
behavior on the single connection to make sure no backbuffer is touched
too early (while a swap is still pending), and many hacks to get around
that, and some race conditions in DRI2 around drawable invalidation.
So i'd be really surprised if Wayland wouldn't be an improvement for me
for multi-threaded or multi-window operations.
Sounds like we will need the "subscribe to streaming vblank events
interface" then. The recommended usage pattern would be to subscribe
only when needed, and unsubscribe ASAP.
There is one detail, though. Presentation timestamp is defined as "turns
to light", not vblank. If the compositor knows about monitor latency, it
will add this time to the presentation timestamp. To keep things
consistent, we'd need to define it as a stream of turned-to-light
events.
Yes, makes sense. The drm/kms timestamps are defined as "first pixel of
frame leaves the graphics cards output connector" aka start of active
scanout. In the time of CRT monitors, that was ~ "turns to light". In my
field CRT monitors are still very popular and actively hunted for
because of that very well defined timing behaviour. Or very expensive
displays which have custom made panel or dlp controllers with well
defined timing specifically for research use.
If the compositor knew the precise monitor latency it could add that as
a constant offset to those timestamps. Do you know of reliable ways to
get this info from any common commercial display equipment? Apple's OSX
has API in their CoreVideo framework for getting that number, and i
implement it in the OSX backend of my toolkit, but i haven't ever seen
that function returning anything else than "undefined" from any display?
Or would it be enough for you to present a dummy frame, and just wait
for the presentation feedback as usual? Since you are asking, I guess
this is not enough.
Wouldn't be enough in all use cases. Especially if i want to synchronize
other modalities like sound or digital i/o, it is nice to not add
overhead or complexity on the graphics side for things not related to
rendering.
We could take it even further if you need to monitor the values
continuously. We could add a protocol interface, where you could
subscribe to an event, per-surface or maybe per-output, whenever the
vblank counter increases. If the compositor is not in a continuous
repaint loop, it could use a timer to approximately sample the values
(ask DRM), or whatever. The benefit would be that we would avoid a
roundtrip for each query, as the compositor is streaming the events to
your app. Do you need the values so often that this would be worth it?
For special applications this would be ok, as in those cases power
consumption is not an issue.
I could think of cases where this could be useful to have per-output if
it is somewhat accurate. E.g., driving shutter glasses or similar
mechanisms in sync with refresh, or emitting digital triggers or network
packets at certain target vblank counts. Neuro-Scientists often have to
synchronize stimulation or recording hardware to visual stimulus
presentation or simply log certain events, e.g., start of refresh
cycles, so it is possible synchronize the timing of different devices. I
hoped to use glXWaitForMscOML() for this in the past, but it didn't work
out because of the constraint that everything would have to go over one
x-display connection and that connection wasn't allowed to block for
more than fractions of a millisecond.
I really do hope something like driving shutter glasses would be done
by the kernel drivers, but I guess we're not there yet. Otherwise it'd
be the compositor, but we don't have Wayland protocol for it yet,
either. I have seen some discussion fly by about TV 3D modes.
I'd hope that as well, long-term. Until then i have to do hacks in my
client app...
On drm/kms, queuing a vblank event would do the trick without the need
for Wayland to poll for vblanks.
Of course i don't know how much use would there be for such
functionality outside of my slightly exotic application domain, except
maybe for "things to do with stereo shutter-glasses", like NVidia's
NVision stuff.
2. As far as the decision boundary for your presentation target
timestamps. Fwiw, the only case i know of NV_present_video extension,
and the way i expose it to user code in my toolkit software, is that a
target timestamp means "present at that time or later, but never
earlier" ie., at the closest vblank with tVblank >= tTargetTime.
E.g., NV_present_video extension:
https://www.opengl.org/registry/specs/NV/present_video.txt
Iow. presentation will be delayed somewhere between 0 and 1 refresh
duration wrt. the requested target time stamp tTargetTime, or on average
half a refresh duration late if one assumes totally uniform distribution
of target times. If i understand your protocol correctly, i would need
to add half a refresh duration to achieve the same semantic, ie. never
present earlier than requested. Not a big deal.
In the end it doesn't matter to me, as long as the behavior is well
specified in the protocol and consistent across all Wayland compositor
implementations.
Yes, you are correct, and we could define it either way. The
differences would arise when the refresh duration is not a constant.
Wayland presentation extension (RFC) requires the compositor to be able
to predict the time of presentation when it is picking updates from the
queue. The implicit assumption is that the compositor either makes its
predicted presentation time or misses it, but never manages to present
earlier than the prediction.
Makes sense.
For dynamic refresh displays, I guess that means the compositor's
prediction must be optimistic: the earliest possible time, even if the
hardware usually takes longer.
Does it sound reasonable to you?
Yes, makes sense to me. For my use case, presenting a frame too late is
ok'ish, as that is the expected behaviour - on time or delayed, but
never earlier than wanted. So an optimistic guess for dynamic refresh
would be the right thing to do when picking a frame for presentation
from the queue.
But i don't have any experience with dynamic refresh displays, so i
don't know how they would behave in practice or where you'd find them? I
read about NVidia's G-Sync stuff lately and about AMD's Freesync
variant, but are there any other solutions available? I know that some
(Laptop?) panels have the option for a lower freq. self-refresh? How do
they behave when there's suddenly activity from the compositor?
You know roughly as much as I do. You even recalled the name
Freesync. :-)
I heard Chrome(OS?) is doing some sort of dynamic scanout rate
switching based on update rates or something, so that would count as
dynamic refresh too, IMO.
I wrote it this way, because I think the compositor would be in a
better position to guess the point of time for this screen update,
rather than clients working on old feedback.
Ah, but the "not before" vs. "rounded to" is orthogonal to
predicting or not the time of the next screen update.
I assume in your use cases the refresh duration will be required to be
constant for predictability, so it does not really matter whether we
pick "not before" or "rounding" semantics for the frame target time.
Yes. Then i can always tweak my target timestamps to get the behavior i
want. For your current proposal i'd add half a refresh duration to what
usercode provides me, because the advice for my users is to provide
target timestamps which are half a refresh before the target vblank for
maximum robustness. This way your "rounding" semantics would turn into
the "not before" semantics i need to expose to my users.
Exactly, and because you aim for constant refresh rate outputs, there
should be no problems. Your users depend on constant refresh rate
already, so that they actually can compute the timestamp according to
your advice.
I can also always feed just single frames into your presentation queue
and wait for present_feedback for those single frames, so i can be sure
the "proper" frame was presented.
Making that work so that you can actually hit every scanout cycle with
a new image is something the compositor should implement, yes, but the
protocol does not guarantee it. I suspect it would work in practise
though.
It works well enough on the X-Server, so i'd expect it to work on
Wayland as well. This link...
<https://github.com/Psychtoolbox-3/Psychtoolbox-3/blob/master/Psychtoolbox/PsychDocumentation/ECVP2010Poster_VisualTimingPrecision.pdf?raw=true>
...points to some results of tests i did a couple of years ago. Quite
outdated by now, but Linux + X11 came out as more reliable wrt. timing
precision than Windows and OSX, especially when realtime scheduling or
even a realtime kernel was used, with both proprietary graphics drivers
and the open-source drivers. I was very pleased with that :) - The other
thing i learned there is how much dynamic power management on the gpu
can bite you if the rendering/presentation behavior of your app doesn't
match the expectations of the algorithms used to control up/downclocking
on the gpu. That would be another topic related to a presentation
extension btw., having some sort of hints to the compositor about what
scheduling priority to choose, or if gpu power management should be
somehow affected by the timing needs of clients...
But, this might be a reason to have the "frame callback" be queueable.
We have just had lots of discussion about what do we do with the frame
callback, what it means, and how it should work. It's not clear yet.
Ideally i could select when queuing a frame, if the compositor will skip
the frame if it is late, that is, when frames with a later target
presentation time are already queued, which are closer to the predicted
presentation time. Your current design is optimized for video playback,
where you want to drop late frames to maintain audio-video sync. I have
both cases. For video playback and periodic/smooth animations i want to
drop frames which are late, but for some other purposes i need to be
sure that although no frame is ever presented too early, all frames are
presented in sequence without dropping any, even if that increases the
mean error of target time vs. real presentation time.
Things like this stuff:
http://link.springer.com/article/10.3758/s13414-013-0605-z
Such studies need to present sequences of potentially dozens of images
in rapid succession, without dropping a single frame. Duplicating a
frame is not great, but not as bad as dropping one, because certain
frames in such sequences may have a special meaning.
This can also be achieved by my client just queuing one frame for
presentation and waiting for present feedback before queuing the next
one. I would lose the potential robustness advantage of queuing in the
server instead of one round trip away in my client. But that's how it's
done atm. with X11/GLX/DRI2 and on other operating systems because they
don't have presentation queues.
A flag for "do not skip", that sounds simpler than I thought. I've
heard that QtQuick would want no-skip for whatever reason, too, and so
far thought posting one by one rather than queueing in advance would do
it, since I didn't understand the use case. You gave another one.
But for dynamic refresh, can you think of reasons to prefer one or the
other?
My reason for picking the "rounding" semantics is that an app can queue
many frames in advance, but it will only know the one current or
previous refresh duration. Therefore subtracting half of that from the
actually intended presentation times may become incorrect if the
duration changes in the future. When the compositor does the equivalent
extrapolation for each framebuffer flip it schedules, it has better
accuracy as it can use the most current refresh duration estimate for
each pick from the queue.
The "rounding" semantics has the problem that if the client doesn't want
it, as in my case, it needs to know the refresh duration to counteract
your rounding by shifting the timestamps. On a dynamic refresh display i
wouldn't know how to do that. But this is basically your thinking
backwards, for a different definition of robust.
Right. When you have to set time based on an integer frame counter, it
is hard to reason about when it actually is. The only possibility is to
use the "not before" definition, because you just cannot subtract half
with integers, and the counter ticks when the buffer flips, basically.
But with nanosecond timestamps we actually have a choice.
Since the definition is turns-to-light and not vblank, I still think
rounding is more appropriate, since it is the closest we can do for a
request to "show this at time t". But yes, I agree the wanted behaviour
may depend on the use case.
Agreed.
As far as i'm concerned i'd tell my users that if they want it to work
well enough on a dynamic refresh display, they'd have to provide
different target timestamps to accommodate the different display
technology - in these cases ones that are optimized for your "rounding"
semantics.
So i think your proposal is fine. My concern is mostly to not get any
surprises on regular common fixed refresh rate displays, so that user
code written many years ago doesn't suddenly behave in weird and
backwards incompatible ways, even on exactly the same hardware, just
because they run it on Wayland instead of X11 or on some other operating
system. Having to rewrite dozens of their scripts wouldn't be the
behavior my users would expect after upgrading to some future
distribution with Wayland as display server ;-)
Keeping an eye on such concerns is exactly where I need help. :-)
3. I'm not sure if this applies to Wayland or if it is already covered
by some other part of the Wayland protocol, as i'm not yet really
familiar with Wayland, so i'm just asking: As part of the present
feedback it would be very valuable for my kind of applications to know
how the present was done. Which backend was used? Was the presentation
performed by a kms pageflip or something else, e.g., some partial buffer
copy? The INTEL_swap_event extension for X11/GLX exposes this info to
classic X11/GLX clients. The reason is that drm/kms-pageflips have very
reliable and precise timestamps attached, so i know those timestamps are
trustworthy for very timing sensitive applications like mine. If page
flipping isn't used, but instead something like a (partial) framebuffer
copy/blit, at least on drm the returned timestamps are then not
reliable/trustworthy enough, as they could be off by up to 1 video
refresh duration. My software, upon detecting anything but a
page-flipped presentation, logs errors or even aborts a session, as
wrong timestamps or presentation timestamping could be a disaster for
those applications if it went unnoticed.
So this kind of feedback about how the present was executed would be
useful to us.
How presentation was executed is not exposed in the Wayland protocol in
any direct nor reliable way.
But rather than knowning how presentation was executed, it sounds like
you would really want to know how much you can trust the feedback,
right?
Yes. Which means i also need to know how much i can trust the feedback
about how much i can trust the feedback ;-).
Therefore i prefer low level feedback about what is actually happening,
so i can look at the code myself and also sometimes perform testing to
find out, which presentation methods are trustworthy and which not. Then
have some white-lists or black-lists of what is ok and what is not.
In Weston's case, there are many ways the presentation can be executed.
Weston-on-DRM, regardless of using the GL (hardware) or Pixman
(software) renderer, will possibly composite and always flip, AFAIU. So
this is both vsync'd and accurate by your standards.
That's good. This is the main use case, so knowing i use the drm backend
would basically mean everything's fine and trustworthy.
Most of my users applications run as unoccluded, undecorated fullscreen
windows, just filling one or multiple outputs, so they use page-flipping
on X11 as well. For windowed presentation i couldn't make any
guarantees, but if Wayland always composes a full framebuffer and flips
in such a case then timestamps should be trustworthy.
Yes.
Weston-on-fbdev (only Pixman renderer) is not vsync'd, and the
presentation timing is based on reading a software clock after an
arbitrary delay. IOW, the feedback and presentation is rubbish.
Weston-on-X11 is basically as useless as Weston-on-fbdev, in its current
implementation.
Probably both not relevant for me.
Weston-on-rpi (rpi-renderer) uses vsync'd flips to the best of our
knowledge, but can and will fall back to "firmware compositing" when
the scene is too complex for its "hardware direct scanout engine". We
do not know when it falls back. The presentation time is recorded by
reading a software clock (clock_gettime) in a thread that gets called
asynchronously by the userspace driver library on rpi. It should be
better than fbdev, but for your purposes still a black box of rubbish I
guess.
There were a few requests for Rasperry Pi, but its hardware is mostly
too weak for our purposes. So also not a big deal for my purpose.
Well, it depends. If you have only ever seen the X desktop on RPi, then
you have not seen what it can actually do.
DispmanX (the proprietary display API) can easily allow you to flip
full-HD buffers at framerate, if you first have enough memory to
preload them. There is a surprisingly lot of power compared to the
"size", but it's all behind proprietary interfaces.
FWIW, Weston on rpi uses DispmanX, and compared to X, it flies.
I've seen a couple of impressive things with those in the lab. The
limitations for my toolkit are more wrt. RAM size, not wrt. gpu. Model
B's 512 MB RAM are the minimum i'd probably need, but i should probably
play with one and see how far i get.
It seems like we could have several flags describing the aspects of
the presentation feedback or presentation itself:
1. vsync'd or not
2. hardware or software clock, i.e. DRM/KMS ioctl reported time vs.
compositor calls clock_gettime() as soon as it is notified the screen
update is done (so maybe kernel vs. userspace clock rather?)
3. did screen update completion event exist, or was it faked by an
arbitrary timer
4. flip vs. copy?
Yes, that would be sufficient for my purpose. I can always get the
OpenGL renderer/vendor string to do some basic checks for which kms
driver is in use, and your flags would give me all needed dynamic
information to judge reliability.
Except that the renderer info won't help you, if the machine has more
than one GPU. It is quite possible to render on one GPU and scan out
on another.
Yes, but i already use libpciaccess to enumerate gpu's on the bus, and
other more scary things, so i guess there will be a few more scary and
shady low level things to add ;-)
Now, I don't know what implements the "buffer copy" update method you
referred to with X11, or what it actually means, because I can think of
two different cases:
a) copy from application buffer to the current framebuffer on screen,
but timed so that it won't tear, or
b) copy from application buffer to an off-screen framebuffer, which is
later flipped to screen.
Case b) is the normal compositing case that Weston-on-DRM does in GL and
Pixman renderers.
Case a) was the typical case on classic X11 for swaps of non-fullscreen
windows. Case b) is fine. On X11 + old compositors there was no suitable
protocol in place. The Xserver/ddx would copy to some offscreen buffer
and the compositor would do something with that offscreen buffer, but
timestamps would refer to when the offscreen copy was scheduled to
happen, not when the final composition showed on the screen. Thereby the
timestamps were overly optimistic and useless for my purpose, only good
enough for some throttling.
Weston can also flip the application buffer directly on screen if the
buffer is suitable and the scene graph allows; a decision it does on a
frame-by-frame basis. However, this flip does not imply that *all*
compositing would have been skipped. Maybe there is a hardware overlay
that was just right for one wl_surface, but other things are still
composited, i.e. copied.
Given that my app mostly renders to opaque fullscreen windows, i'd
expect Weston to probably just flip its buffer to the screen.
Indeed, if you use EGL. At the moment we have no other portable ways to
post suitable buffers.
I'm not sure what we should be distinguishing here. No matter how
weston updates the screen, the flags 1-3 would still be applicable and
fully describe the accuracy of presentation and feedback, I think.
However, the copy case a) is something I don't see accounted for. So
would the flag 4 be actually "copy case a) or not"?
Yes, if Weston uses page-flipping in the end on drm/kms and the returned
timestamp is the page-flip completion timestamp from the kernel, then it
doesn't matter if copies were involved somewhere.
The problem is that I do not know why the X11 framebuffer copy case you
describe would have inaccurate timestamps. Do you know where it comes
from, and would it make sense to describe it as a flag?
That case luckily doesn't seem to exist on Weston + drm/kms, if i
understand you correctly - and the code in drm_output_repaint(), if i
look at the right bits?
You're right on Weston. It still does not prevent someone else writing
a compositor that does the just-in-time copy into a live framebuffer
trick, but if someone does that, I would expect them to give accurate
presentation feedback nevertheless. There is no excuse to give
anything else.
FYI, while Weston is the reference compositor, there is no "the
server". Every major DE is writing their own compositor-servers. Most
will run on DRM/KMS directly, and I think one has chosen to run only on
top of another Wayland compositor.
Yes, that will be a whole lot of extra "fun" of testing, once i have a
Wayland backend ;-) -- In the end i'll have to focus on maybe two or
three compositors like weston, kwin, gnome-shell wrt. to timing.
Old compositors like, e.g., Compiz, would do things like for partial
screen updates only recompose the affected part of the backbuffer and
then use functions like glXCopySubBufferMESA() or glCopyPixels() to copy
the affected backbuffer area to the frontbuffer, maybe after waiting for
vblank before executing the command.
Client calls glXSwapBuffer/glXSwapBufferMscOML -> server/ddx queues
vblank event in kernel -> kernel delivers vblank event to server/ddx at
target msc count -> ...random delay... -> ddx copies client window
backbuffer pixmap to offscreen surface and posts damage and returns the
vblank timestamp of the triggering vblank event as swap completion
timestamp to the client -> ...random delay... -> compositor does its
randomly time consuming thing which eventually leads to an update of the
display.
I regret I asked. ;-)
So in any case the returned timestamp from the vblank event delivered by
the kernel was always an (overly) optimistic one, also because all the
required rendering and copy operations would be queued through the gpu
command stream where they might get delayed by other pending rendering
commands, or maybe the gpu waiting for the scanout being outside the
target area for a copy to avoid tearing.
Without a compositor it worked the same for windowed windows, just that
the ddx queued some blit (i think via exa or uxa or maybe now sna on
intel iirc) into the gpu's command stream, with the same potential
delays due to queued commands in the command stream.
Only if the compositor was unredirecting fullscreen windows, or no
compositor was there at all + fullscreen window, would the client get
kms page-flipping and the reliable page flip completion timestamps from
the kernel.
This system of flags was the first thing that came to my mind. Using
any numerical values like "the error in presentation timestamp is
always between [min, max]" seems very hard to implement in a
compositor, and does not even convey information like vsync or not.
Indeed. I would have trust issues with that.
I suppose a single flag "timestamp is accurate" does not cut it? :-)
What do you think?
Yes, the flags sound good to me. As long as i know i'm on a drm/kms
backend they should be all i need.
That... is a problem. :-)
From your feedback so far, I think you have only requested additional
features:
- ability to subscribe to a stream of vblank-like events
- do-not-skip flag for queued updates
Yes, and the present_feedback flags.
One more useful flag for me could be to know if the presented frame was
composited - together with some other content - or if my own buffer was
just flipped onscreen / no composition took place. More specifically -
and maybe there's already some other event in the protocol for that -
i'd like to know if my presented surface was obscured by something else,
e.g., some kind of popup window like "system updates available", "you
have new mail", a user ALT+Tabbing away the window etc. On X11 i find
out indirectly about such unwanted visual disruptions because the
compositor would fall back to compositing instead of simple
page-flipping. On Wayland something similar would be cool if it doesn't
already exist.
Not sure if that still belongs into a present_feedback extension though.
I will think about those.
Cool, thanks!
-mario
_______________________________________________
wayland-devel mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/wayland-devel
