wl_surface::attach(NULL) release previous buffer?
Hi A, hopefully, simple question - should I expect a wl_buffer::release event from the buffer previously committed to a surface after I've attached (and invalidated & committed) a NULL buffer to the surface? it doesn't seem to happen for me (I have WAYLAND_DEBUG=1 logs showing it not happening). If I shouldn't expect a release - when is it safe to reuse/free the buffer storage? Thanks John Cox
Re: Questions about object ID lifetimes
On Wed, 13 Sep 2023 20:16:09 -0400 jleivent wrote: > Forgive the long post. Tl;dr: what are the rules of object ID lifetime > and reuse in the Wayland protocol? Hi, congratulations, I think you may have found everything that is not quite right in the fundamental Wayland protocol design. :-) As an aside, we collect unfixable issues under https://gitlab.freedesktop.org/wayland/wayland/-/issues/?label_name%5B%5D=Protocol-next These are issues that are either impossible or very difficult or annoying to fix while keeping backward compatibility with both servers and clients. -- Object ID re-use is what I would call "aggressive": in the libwayland C implementation, the object ID last freed is the first one to be allocated next. There are two separate allocation ranges each with its own free list: server and client allocated IDs. The C implementation also poses an additional restriction: a new ID can be at most the largest ever allocated ID + 1. All this is to keep the ID map as compact as possible without a hash table. These details are in the implementation of the private 'struct wl_map' in libwayland. > I am attempting to understand the rules of object ID lifetime within > the Wayland protocol in order to construct Wayland middleware (similar > to some of the tools featured on > https://wayland.freedesktop.org/extras.html). I could not find a > comprehensive discussion of the details online. If one exists, I would > greatly appreciate a link! > > Middleware tools that wish to decode Wayland messages sent between the > compositor and its clients need to maintain an accurate mapping between > object ID and object interface (type). This is needed because the wire > protocol's message header includes only the target object ID and an > opcode that is relative to the object's type (the message header also > includes the message length - about which I also have questions - to be > pursued later...). The message (request or event) and its argument > encoding can only be determined if the object ID -> type and type + > opcode -> message mappings are accurately maintained. The type + > opcode -> message mapping is static and can be extracted offline from > the protocol XML files. > > Since object IDs can be reused, it is important for the middleware to > understand when an ID can be reused and when it cannot be to avoid > errors in the ID -> type mapping. > > Because the Wayland protocol is asynchronous, any message that implies > destruction of an object should be acknowledged by the receiver before > the destructed object's ID is reused. > > Fortunately, certain events and requests have been tagged as > destructors in the protocol descriptions! > > Also fortunately, it appears (based on reading the wl_resource_destroy > code in wayland-server.c) that for many object IDs, specifically for > IDs of objects created by a client request (the ID appears as a new ID > arg of a request, and is thus in the client side of the ID range) and > for which the client makes a destructor request, the compositor will > always send a wl_display::delete_id event (assuming the > display_resource still exists for the client, which apparently would > only not be the case after the client connection is severed) to > acknowledge the destructor request. Any attempt to reuse that ID prior > to the wl_display::delete_id event can lead to confusion, and should be > avoided. Reuse of the ID after the wl_display::delete_id event should > not result in any confusion. > > [BTW: for the purpose of this discussion, an object is "created" when > it is introduced into a protocol message for the first time via a new_id > argument. It does not refer to the actual allocation of the object in > memory or to its initialization.] Your whole above analysis is completely correct! > However, the other cases are not as easy to identify. > > The other cases are: > 1. an object created by a client request that has destructor events > 2. an object created by the compositor > > It might be true that case 1 does not exist. Is there a general rule > against that such cases would never be considered in future expansions > of the Wayland protocol? Destructor events do exist. Tagging them as such in the XML was not done from the beginning though, it was added later in a backward-compatible manner which makes the tag more informational than something libwayland could automatically process. The foremost example is wl_callback.done event. This is only safe because it is guaranteed that the client cannot be sending a request on wl_callback at the same time the server is sending 'done' and destroying the object: wl_callback has no requests defined at all. It also requires that nothing passes an existing wl_callback object as an argument in any request. We have been merely lucky that no-one has done that. It's really hard to imagine a use case where you would want to pass an existing wl_callback to anything. Extensions may have similar objects
Re: Questions about object ID lifetimes
On Thu, 14 Sep 2023 16:32:06 +0300 Pekka Paalanen wrote: > ... > > congratulations, I think you may have found everything that is not > quite right in the fundamental Wayland protocol design. :-) Oh, you flatter me. I'm sure there's more! > > As an aside, we collect unfixable issues under > https://gitlab.freedesktop.org/wayland/wayland/-/issues/?label_name%5B%5D=Protocol-next > These are issues that are either impossible or very difficult or > annoying to fix while keeping backward compatibility with both servers > and clients. Only 7 of them? > > -- > > Object ID re-use is what I would call "aggressive": in the libwayland > C implementation, the object ID last freed is the first one to be > allocated next. There are two separate allocation ranges each with its > own free list: server and client allocated IDs. After I sent the initial post, I realized that the two separate ID ranges help in the following way: For any object ID in the allocation range of side A, a destructor message from side B does not need acknowledgement. This is because B can't introduce a new object bound to that ID, only A can. Hence, any new_id arg for that ID is an acknowledgement of the destruction. However, B has to be careful to ignore messages containing that ID until it sees one with the ID as a new_id arg. After the destructor message from B but before a subsequent new_id for that ID from A, B should not use the ID as arguments to other messages (and attempts to do so can be dropped). And this can be automated provided the destructor tag can be relied on. > > The C implementation also poses an additional restriction: a new ID > can be at most the largest ever allocated ID + 1. > > All this is to keep the ID map as compact as possible without a hash > table. These details are in the implementation of the private 'struct > wl_map' in libwayland. Obviouly, that helps middleware as well, for the same reasons. It also makes more automatic error detection possible. > ... > > Your whole above analysis is completely correct! I was rather hoping things would turn out less complex than they seemed... > > > However, the other cases are not as easy to identify. > > > > The other cases are: > > 1. an object created by a client request that has destructor events > > 2. an object created by the compositor > > > > It might be true that case 1 does not exist. Is there a general > > rule against that such cases would never be considered in future > > expansions of the Wayland protocol? > > Destructor events do exist. Tagging them as such in the XML was not > done from the beginning though, it was added later in a > backward-compatible manner which makes the tag more informational than > something libwayland could automatically process. The foremost example > is wl_callback.done event. This is only safe because it is guaranteed > that the client cannot be sending a request on wl_callback at the same > time the server is sending 'done' and destroying the object: > wl_callback has no requests defined at all. Fortunately, my point above about the advantage of the separate ID ranges helps here. If wl_callback is created by the client, then a wl_callback.done event tagged as a destructor does not need acknowledgement AND is always safe provided that messages involving the wl_callback ID (other than it's eventual reuse as a new_id arg) are ignored above libwayland. But again, this means the destructor tag is important and not merely informational. I did notice that the destructor tagging was added mostly (or solely) to help with code generation by wayland-scanner implementations in programming languages where destructors require some specific syntactic notation. But maybe destructor tagging is even better than that? Maybe it would allow libwayland to automate more in a more robust way AND also allow for middleware that doesn't have to simulate all of the semantic level interactions induced by protocol messages in order to merely keep track of how to decode messages. > > It also requires that nothing passes an existing wl_callback object as > an argument in any request. We have been merely lucky that no-one has > done that. It's really hard to imagine a use case where you would want > to pass an existing wl_callback to anything. Again, the above separate ID ranges point addresses this, I think. > > Extensions may have similar objects that only deliver some one-off > events and then "self-destruct" by the final event. All this is simply > documented and not marked in the XML. That's what I was hoping to avoid. If there are object types where object lifetime can only be understood by simulating all of the relevant semantic content of the messages involved, then that's not good for middleware. Isn't it also problematic towards the goals of libwayland, because it makes it impossible for libwayland to ensure that messages are properly decoded without trusting that the client and/or compositor have implemen
