Hi Pekka On Wed, Mar 6, 2013 at 1:49 PM, Pekka Paalanen <[email protected]> wrote: > Hi, > > I have been pondering about sub-surface nesting, i.e. allowing > sub-sub-surfaces and so on, and I have hard time figuring out how > commits should work. > > The v2 sub-surface protocol proposal is here: > http://cgit.collabora.com/git/user/pq/weston.git/tree/protocol/subsurface.xml?h=subsurface-v2 > > Adding support for nesting does not require any API changes to the > protocol. We just need to define, that it is ok to have a sub-surface > as a parent for another sub-surface, and how commits work. Commit > behaviour is again the hard part. > > Suppose we have a window consisting of the following: > > main surface, wl_surface A > ↳ sub-surface, wl_surface B > > Every surface has current state (how it is currently drawn on screen), > and pending state (to accumulate pieces of state, that will then be > made current atomically on wl_surface.commit). > > Additionally, sub-surfaces have a state cache. When a sub-surface > itself is explicitly committed, the pending state either becomes > current, or goes to the cache, depending on the commit mode of the > sub-surface. The pending state can only be used when a commit request > was received for the surface itself, never via parent commit. > > If the commit mode for the sub-surface B is parent-cached, then > B.commit() will push the pending state into the cache. The cached state > is made current on A.commit(), so that all surfaces get updated > atomically. > > If the commit mode for the sub-surface B is independent, then > B.commit() will make the pending state current. A.commit() does not > affect B's current state, except for the wl_subsurface.set_position. > > The position set by a wl_subsurface.set_position request is always > stored, and applied only on parent surface commit (A.commit()). This > way the contents of wl_surface B can be updated independently, but > positioning it is still tied to the wl_surface A updates. > > (I'm not considering the corner cases arising from transitions from one > commit mode to another at arbitrary times here, but they are solved > for non-nested sub-surfaces.) > > > Now, add a sub-sub-surface. Suppose we have a window consisting of the > following: > > main surface, wl_surface A > ↳ sub-surface, wl_surface B, parent-cached > ↳ sub-sub-surface, wl_surface C, parent-cached > > Assume the commit mode of all sub-surfaces is parent-cached, as > written above. C.commit() will push C's pending state into C's cache. > B.commit() will push B's pending state into B's cache. But does > B.commit() do anything to C? > > As B is a sub-surface itself, needing a parent commit to change its > current state, B.commit() certainly cannot make C's cached or pending > state current. > > Suppose we have this sequence in protocol: > > C.commit(C1) > B.commit(B1) > C.commit(C2) > A.commit(A1) > > What state should be current for each surface? > For A, it is A1, of course. > For B, it is B1, trivially. > For C, should it be C1 which has been "blessed" by B, or C2 which is > the latest but not "blessed" by B? > > If we say C1, then we will need a multi-level cache for each > sub-surface, with a level of cache for each level of nesting. > Sub-surface C would need a 2-level cache. C.commit() pushes pending > state to C.cache-1. B.commit() pushes C.cache-1 to C.cache-2. > A.commit() makes C.cache-2 as current state of C. > > Furthermore, each level of cache may hold a reference to a wl_buffer. > This means, that the component driving wl_surface C will actually need > a pool of 4 buffers: > - a buffer that is currently on screen > - a buffer that is in cache-2 > - a buffer that is in cache-1 > - a buffer that it is rendering to > > I do not think this is feasible. So, the answer must be that C2 is the > current state for C. In other words, whether B.commit is called or not, > it has no effect on what happens with C. (But does it make sense for > applications?)
My first thought was "sure, C1 it is", but considering the huge cache consumption if we have deeper sub-sub*-surface nesting, this really doesn't make sense. So I was wondering whether C2 is actually that bad. So lets assume a user uses parent-cached sub-surfaces. This means, they want the child to be updated only when the parent got updated. This implicates, that the user has some kind of synchronization between child and parent. Otherwise using parent-cached just doesn't make sense as they could use the independent mode instead. With this in mind, we know that if we have multi-layer nested parent-cached sub-surfaces, then there already is a synchronization between _all_ of them. So saying "parent-cached" means "top-most-parent-cached" is actually fine for these use-cases, isn't it?. Of course, "top-most-parent" means the top most parent with only "parent-cached" in between. Ok, but are there situations where parent-cached makes sense but there is no synchronization in the client already? I actually cannot think of anything. The thing is, if we have parent-cached without synchronization in the client, the parent surface does not know the current state of the client. So it has no idea which frame the client currently has pending, it just knows that it is only displayed _together_ with the parent. So when the parent does the commit, it knows the client is updated atomically, but it has no idea what frame the client actually drew. So what is the reason that you actually want the atomic update in this case? The independent mode would be just fine here. The only drawback is that the parent no longer controls the frame-rate of the client. But does that really matter? So if we can conclude that the client does synchronization between parent-cached subsurfaces, we can actually rename parent-cached to top-most-parent-cached and be fine. If anyone comes up with a use-case for _real_ parent-cached (that is, bottom-most-parent-cached), we can always implement that with the heavy caching, can't we? And top-most-parent-cached and independent mode are both trivial to implement, if I understand you correctly? > > Next a case with mixed commit modes: > > main surface, wl_surface A > ↳ sub-surface, wl_surface B, independent > ↳ sub-sub-surface, wl_surface C, parent-cached > > C.commit() pushes C's pending state to C's cache. > B.commit() make B's pending state current, and make C's cached state current, > too. > A.commit() make A's pending state current, and... does nothing to B, > since B is independent. Also, does not even look past B, so nothing for C. > > Ok, I guess that works. And the case where B is parent-cached and C is > independent should be obvious, too. > > > > This leads to the following commit algorithm: > > - Main surface commit, and sub-surface commit in independent mode, will > make own pending state current, and will call recursive commit on all > its immediate sub-surfaces. > > - Sub-surface commit in parent-cached mode will push the pending state > into the sub-surface's cache. No recursion. > > - A recursive commit on a sub-surface with independent commit mode does > nothing. No recursion. > > - A recursive commit on a sub-surface with parent-cached commit mode > will make the surface's cached state current, and call recursive > commit on all its immediate sub-surfaces. > > And then add all the corner cases when commit modes are changing, and > e.g. a sub-surface is committed in independent commit mode with some > state in its cache. > > Oh, and I forgot to specify what happens with > wl_subsurface.set_position. D'oh. > > > So, what are the benefits from nesting sub-surfaces? > - Each sub-surface's position is relative to its parent, so you don't > have to add up the offsets in the client. > - An independent sub-surface can have its own sub-surfaces, and update > the whole wl_surface sub-tree in sync. > - ...? > > > Is this all really worth it? > Does anyone have a good use case for nested sub-surfaces, which would > be inconvenient to implement with just sibling sub-surfaces without > nesting? > > Unless there is a good use case I can design for, I'm tempted to just > not bother with this, this time. If we later see, that nesting > would actually have value, we can add the support by just bumping the > wl_subcompositor interface version, and without any changes to the > protocol signature. Use-case for sub-sub-surfaces: - flash-player embedded in webkit embedded in some application I often see applications that embed a browser (for instance STEAM). If that browser embeds other stuff, you want it to be independent of the parent. This use case requires top-most-parent-cached or independent. But I cannot think of a reason why it requires bottom-most-parent-cached, can you? If we don't allow sub-surface nesting, then clients will have to provide some sub-surface allocation forwarding. So for instance in my use case described above, the application would provide a sub-surface allocator to webkit, which would allocate the sub-surface for flashplayer. But webkit would not be able to allocate the sub-surface itself from the Wayland compositor as it itself is a sub-surface. So in this case, we already have the synchronization of the clients via the sub-surface allocator in the application. So there _is_ a communication channel between the application and it's sub-sub-sub-...-surfaces. This allows us to use top-most-parent-cached instead of (bottom-most-)parent-cached. => Implement sub-surface nesting in the compositor to avoid having each and every application to implement the sub-surface forwarding if they themselves run as sub-surfaces. Anyway, I really like the proposal. Apart from few typos the protocol looks really good and I agree that we can implement nesting later. Thanks David _______________________________________________ wayland-devel mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/wayland-devel
