Re: [dev-servo] Compositor Tree Redesign
Hi there, A few words about TextureClient/Host (since it's been mentioned in this thread): we have a bit of design doc about it at: * http://dxr.mozilla.org/mozilla-central/source/gfx/doc/MozSurface.md * http://dxr.mozilla.org/mozilla-central/source/gfx/doc/SharedMozSurface.md Basically TextureClient is the content how the content process views a shared texture and TextureHost is how the compositor process views it. long term, we'd like to share more code between TextureClient/Host. We call MozSurface the hypothetical abstraction over TextureClient and TextureHost that we are in the process of adding). One point that I should insist on is whether the shared texture data should be accessible on both sides at the same time: At first we thought that we could just send ownership of the texture along with it when sending a texture to the compositor. This feels like the right thing to do but has bitten us very hard because at any time in gecko the main thread can synchronously ask read access to any layer's content, for instance to take a screen shot, or do some of the optimization we do (copy-on-write tiles, synchronizing the front and back buffers' valid regions to repaint as few texels as possible, etc.). It turns out there are very few places (canvas/video) in the gecko's compositor where we can just get away with a classic swap-chain like most other compositors do. It may turn out that some of the reasons pure message passing failed for gecko will not apply to servo, but I think that, if only for the sake of having copy-on-write textures, shared ownership is a better solution in the end (at the expense of some complicated logic around deallocating the textures). We also need to support having a TextureClient/Host pair used by several layers (for instance when the source of a video element is the output of another one). So our current architecture lets us attach a texture to several layers which adds some constraints. It also took some effort to cleanly support the child process potentially crashing at any time, without taking down the main process with it. What was very hard to get right is ownership of the shared data. There are tons of constraints from how gecko works and from the underlying rendering backends we support. Also there has been a very unhealthy habit of rushing things for one platform without caring about how other platforms differ, and have to do some major refactorings afterward to get it to work everywhere. A thing that is working very well for us is the separation between the structure of the layer tree (Layer[Client/Host] objects), what manipulates textures (and handle things like double-buffering, tiling, swap-chains, etc) (Compositable[Client/Host] objects), and the textures themselves which abstract out the underlying backends (Texture[Client/Host] objects). It's now rather easy for us to have layers be manipulated within transactions on the main thread while a swap chain (Compositable object) sends textures from, say, a webgl worker or a video decoder to the compositor thread without touching the main thread. Two years ago, all of the logic was contained into Layer classes so we had an awful lot of code duplication and it was hard to decouple the position of elements in the page from the data they contained (which is desirable for efficient video compositing and canvas workers). I also think that it is good to have a solid and universal texture abstraction that content can draw into and share with the compositor without copies early on, since it is rather hard to retro-fit the constraints of shareable textures into the rendering code after the fact, and that as long as there are textures being copied, they will show up in profiles. Sorry for the long email, don't hesitate to ask me questions about anything layers-related in gecko. Cheers, Nical On Tue, Jul 1, 2014 at 7:14 AM, Cameron Zwarich wrote: > We discussed this a bit on #servo, but it isn’t obvious from your diagram > if every Layer will have child layers, or if that will be reserved for > specific container layers. > > CoreAnimation takes the former approach, but I think the latter might be > more applicable for Servo, since it would give a better separation of > concerns. We would then have the following layer types: > > 1. ContainerLayer, which can only parent other layers. > 2. TileLayer, which would manage tiling, BufferRequests, and all double > buffering of individual tile backing stores. > 3. TextureLayer (maybe ImageLayer would be a better name?), which is > backed by a simple image. Would we even want it to be double buffered? > > How are iframe layer trees handled? I would prefer a model where each Rust > task owns a single ‘rendering context’, or whatever we want to call it, and > another layer type (HostLayer, ProxyLayer, ???) that hosts one rendering > context in another. Maybe this is more general than what we actually need, > but synchronization problems will be easier to solve if each task has it
Re: [dev-servo] Compositor Tree Redesign
Thanks for the detailed reply. I never expected that Servo would be able to be able to avoid the possibility of concurrent access to surfaces, for a lot of those same reasons that you mention. I have a few questions about the docs. 1. What copy-on-write optimizations is this referring to? "* We do some copy-on-write optimizations on surfaces that are shared with the compositor in order to keep invalid regions as small as possible. Out tiling implementation is an example of that.” 2. How do you do front/back buffer management differently for tiled and untiled layers? Cameron On Jul 1, 2014, at 4:58 AM, Nicolas Silva wrote: > Hi there, > > A few words about TextureClient/Host (since it's been mentioned in this > thread): > > we have a bit of design doc about it at: > * http://dxr.mozilla.org/mozilla-central/source/gfx/doc/MozSurface.md > * http://dxr.mozilla.org/mozilla-central/source/gfx/doc/SharedMozSurface.md > > Basically TextureClient is the content how the content process views a shared > texture and TextureHost is how the compositor process views it. long term, > we'd like to share more code between TextureClient/Host. We call MozSurface > the hypothetical abstraction over TextureClient and TextureHost that we are > in the process of adding). > > One point that I should insist on is whether the shared texture data should > be accessible on both sides at the same time: At first we thought that we > could just send ownership of the texture along with it when sending a texture > to the compositor. This feels like the right thing to do but has bitten us > very hard because at any time in gecko the main thread can synchronously ask > read access to any layer's content, for instance to take a screen shot, or do > some of the optimization we do (copy-on-write tiles, synchronizing the front > and back buffers' valid regions to repaint as few texels as possible, etc.). > It turns out there are very few places (canvas/video) in the gecko's > compositor where we can just get away with a classic swap-chain like most > other compositors do. It may turn out that some of the reasons pure message > passing failed for gecko will not apply to servo, but I think that, if only > for the sake of having copy-on-write textures, shared ownership is a better > solution in the end (at the expense of some complicated logic around > deallocating the textures). > We also need to support having a TextureClient/Host pair used by several > layers (for instance when the source of a video element is the output of > another one). So our current architecture lets us attach a texture to several > layers which adds some constraints. > It also took some effort to cleanly support the child process potentially > crashing at any time, without taking down the main process with it. > What was very hard to get right is ownership of the shared data. There are > tons of constraints from how gecko works and from the underlying rendering > backends we support. Also there has been a very unhealthy habit of rushing > things for one platform without caring about how other platforms differ, and > have to do some major refactorings afterward to get it to work everywhere. > > A thing that is working very well for us is the separation between the > structure of the layer tree (Layer[Client/Host] objects), what manipulates > textures (and handle things like double-buffering, tiling, swap-chains, etc) > (Compositable[Client/Host] objects), and the textures themselves which > abstract out the underlying backends (Texture[Client/Host] objects). It's now > rather easy for us to have layers be manipulated within transactions on the > main thread while a swap chain (Compositable object) sends textures from, > say, a webgl worker or a video decoder to the compositor thread without > touching the main thread. Two years ago, all of the logic was contained into > Layer classes so we had an awful lot of code duplication and it was hard to > decouple the position of elements in the page from the data they contained > (which is desirable for efficient video compositing and canvas workers). I > also think that it is good to have a solid and universal texture abstraction > that content can draw into and share with the compositor without copies early > on, since it is rather hard to retro-fit the constraints of shareable > textures into the rendering code after the fact, and that as long as there > are textures being copied, they will show up in profiles. > > Sorry for the long email, don't hesitate to ask me questions about anything > layers-related in gecko. > > Cheers, > > Nical > > > > On Tue, Jul 1, 2014 at 7:14 AM, Cameron Zwarich wrote: > We discussed this a bit on #servo, but it isn’t obvious from your diagram if > every Layer will have child layers, or if that will be reserved for specific > container layers. > > CoreAnimation takes the former approach, but I think the latter might be more >
Re: [dev-servo] Compositor Tree Redesign
On 06/30/2014 10:14 PM, Cameron Zwarich wrote: > We discussed this a bit on #servo, but it isn’t obvious from your > diagram if every Layer will have child layers, or if that will be > reserved for specific container layers. The way I've designed it now, TileLayers are not siblings of ContainerLayers, but are instead data members on ContainerLayers. All ContainerLayers can potentially have children, but none of those children will be TileLayers. > CoreAnimation takes the former approach, but I think the latter might > be more applicable for Servo, since it would give a better separation > of concerns. We would then have the following layer types: > > 1. ContainerLayer, which can only parent other layers. 2. TileLayer, > which would manage tiling, BufferRequests, and all double buffering > of individual tile backing stores. 3. TextureLayer (maybe ImageLayer > would be a better name?), which is backed by a simple image. Would we > even want it to be double buffered? I still need to look a lot more closely at how tiling works in the next phase, but this design works for me. I think most of the pieces are already in place actually. --Martin ___ dev-servo mailing list dev-servo@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-servo
