On Mon, 9 Sep 2013 08:01:51 +0000 <[email protected]> wrote: > Hi all, > > let me give some technical background on the extension, and why we > have decided to design the first approach of the extensions like it > is published from Tanibata-san. In general, we would like to get the > feedback on the extensions, to get it fitting in the FOSS approach, > fulfilling the ivi requirements. See my comments in line.
Jason Ekstrand had an excellent point asking which parts you are actually looking for to standardize. > >> Hi, > >> > >> I would like to get feedback about that if somebody has a similar > >> motivation to support ivi as well as desktop and tablet. So it is > >> not a stone, just a proposal. If somebody has good idea, I would > >> like to use it or collaborate it. > > > > Ok, I just had the understanding that the layer manager simply has > > to be a separate process and not built into the compositor. > > Not really. Currently in the compliance from genivi, we have the > separation of Graphic Backend Server and LayerManagement. But this > should now change we want to skip that approach and want define a > protocol + ivi extensions, including a reference compositor. We have > decided to use wayland + ivi extension as compositor protocol and > weston as the reference implementation for that. Very cool! > > If that is > > not the case, then that is very good news indeed. Everything that > > manages surfaces, layers, windows, or whatever belongs into the > > compositor process, where they are much easier to implement and you > > don't need to introduce interfaces and IPC which are later hard to > > develop further and cause latencies. > > Sure that is the technical best fitting solution for that, we > do not want to use separate IPC for the composition approach. > > So I'm roughly on the same track as Andreas Pokorny. > > We too, but unfortunately it is not possible all the time to get > the complete user interface included in the compositor itself. We > have to guarantee fast startup time to get early video running in 2 > seconds. On different systems we can not achieve that with complete > HMI-UI is up. Oh, no, I am not suggesting to put user interfaces in the compositor, not at all. Only the window management, which means window positioning, z-order, etc. For instance, in Weston virtual desktops are implemented in the compositor, and not by an external process juggling window lists via IPC and setting visibilities and positions. Proposing just like with weston and desktop-shell, there is the weston plugin called desktop-shell.so which does window management, and then there is the helper program weston-desktop-shell, that does all the UI drawing in a separate process. Like Jason Ekstrand explained, desktop-shell.so and weston-desktop-shell communicate with each other not only using the core Wayland protocol, but also using a private (or proprietary) protocol extension. While the basic IPC mechanism is all the same (Wayland), there are different parts: extensions. The desktop-shell extension is actually a protected one, only weston-desktop-shell as started by desktop-shell.so can access it. > > Have you tried to map your IVI concepts of surface/layer/display to > > Wayland wl_surface, wl_subsurface, and wl_output? I don't really > > see what kind of interfaces your applications (Wayland clients?) > > expect to use. > > > > When I look at the protocol in ivi-shell.xml, I get the feeling > > that: > > - Interfaces ivi_layer, ivi_controller_surface, > > ivi_controller_layer, ivi_controller_screen, and ivi_controller > > should be internal implementation details inside the weston > > process, not protocol. Having these as interfaces looks like the X > > architecture, where the X server process and window manager process > > continuously struggle to keep each other up-to-date, and carefully > > try to keep state in sync (and fail), which also makes races and > > glitches practically unavoidable. - Interface ivi_client is just a > > reinvention of wl_compositor and wl_subcompositor. > > - Interface ivi_surface is a reinvention of wl_surface. > > Let me explain, why this separation occurs. > From the ivi perspective we have separated the different applications > in ivi-client, ivi-control. > > ivi-client: Is just a client, like browser, navigation which is > embedded in the browser. That belongs to the ivi-surface. These > process have really limited access the can just create the surface > and that's all. The only important point which the want to know is if > the visibility has changed and the size to adapt the rendered > content. A navigation should not render the content if it is not > visible or assigned to a layer. That is actually very well fitting to what we already have in Wayland. Just replace ivi-client with a wl_client (that's not a protocol object, it represents a display connection from a client process), and ivi-surface with wl_surface+wl_shell_surface. The limited access is already there: in Weston, clients are free to create any number of wl_surfaces. However, no surface is shown anywhere on screen, unless it is assigned a suitable role. Examples of roles are: "pointer cursor", "top-level window", "drag icon", and "popup (menu)". Without a role, the compositor cannot know what to do with a surface. "top-level window" and "popup menu" are shell concepts. If you replace wl_shell with your own ivi_shell, you could have roles like: "TV output", "rear camera", "speedometer". I'm totally pulling these out of my hat, I don't know what actually makes sense for you. Then, your shell module in weston could handle such surfaces as you want. For example, if you get a surface with the role "TV output", the shell plugin could show that surface cloned on all entertainment monitors. Or whatever. Or if something signals that now would be a /really/ good time to show the rear view camera, then your shell plugin can make it so, if there is a surface with such role assigned. Now, something like a "rear view camera" sounds like a pretty important role for a surface, so you may want to make it privileged. You don't want any random application to claim to be a rear view camera. In that case, you can simply add whatever authentication you need to do in the shell plugin. For example, only this one process started directly by the shell plugin is allowed to claim the rear view camera role. Anyone else trying that will get killed. On the protocol design, such protection is pretty irrelevant. All you need is for the shell plugin to know who is authorized for what, and it can then enforce the restriction, whether the privileged role is requested by a method in a otherwise public protocol extension or in its own extension. > ivi-controller : This is the main application which is controlling > the compositor like the UI. That could be the hmi, but it can also > some debug applications and application which is triggering a screen > shot of surfaces, layer, displays. If you driving a car during the > development you want to get some screenshots which are normally > triggered by an external application like the log and trace framework. Ok. I am proposing to have window management in the compositor plugin. But, you can offer additional debugging interfaces in the protocol. For example, Weston does not allow clients to control *any* input devices. Clients simply cannot feed fake input events to other clients. But for testing and debugging, we need a way to fake input from a client. The solution: a test plugin. The test pluging, which is loaded by the compositor only for testing (in your case debugging) purposes registers a new protocol extension: the test interface. That test interface then allows to do all kinds of ugly things from clients. I'm not sure what you mean by "UI", but we separate controlling windows from drawing content, like I explained of the desktop shell. As a recap, the shell plugin is what enforces and executes policies. Clients only indicate intent or purpose, like "This surface is a menu, and I want it to pop up right there on my window." A menu is an interesting example, since it is more than just a surface role: it comes with an input device grab. While the menu is up, input is directed to it. But, it is the shell plugin that can decide, that ok, you're done now, grab is broken, the menu surface hidden, and the client is signalled that something dismissed the menu. Also, clients cannot put up a menu at random times, it has to be done as a response to an input event. All this prevents many kinds of client misbehaviour. > > Yes, I see there are some details to may want to control like > > surface opacity, that the current Wayland protocols do not support, > > but I don't think that replacing everything is a good way to start. > > It is also very hard to see how objects from all these interfaces > > are created, and how (if?) they associate to any other protocol > > objects. > > On goal is to get setup a complete scenery during the startup of the > compositor. Therefore the compositor is able to just initialize the > main visible attribute during startup. That implies too that it > should be possible, without the final application is running (like > navigation). Therefore we need some logical objects like ivi-surface, > ivi-layer. Which can be already controlled, whithout a application > which holds the content is already running. But if all layer and window management is internal to the compositor process, then you don't need any of this control protocol. Just create some internal data structures, and when a client comes up with a surface and requests a particular role, you can plug that surface into the slot you prepared in advance. So, yes, this does imply that the compositor or the shell plugin knows the scenegraph, but... isn't that exactly the compositor's purpose, to gather the scenegraph and get it on screen? Clients "only" provide content. For example, the desktop-shell protocol extension has special surface roles "background" and "panel". When weston-desktop-shell client assigns these roles to its surfaces, the compositor (shell plugin) will immediately plug them into the correct places on screen. If the shell plugin wanted to reserve screen space for them in advance, it could. After all, the shell plugin is responsible for arranging everything on (and off) screen. > > Btw. if you need support for surface scaling and cropping, there > > have been discussion on the Wayland mailing list to bring a crop & > > scale protocol extension to Wayland. It is actually necessary for > > efficient video playback etc., so pushing that forward would be > > nice. > > Yeah I see that, and we will consider it. I think we can redesign the > approach like source and destination region to the crop and scale > instead. > > > After looking through the two links you gave, the ivi-shell.xml, > > and what you have wrote in the emails, I still have no clue what is > > the big picture here. > > > > - What processes are going to use which interfaces? It looks to me > > like some interfaces are not meant for all Wayland clients, but > > how is it supposed to work? > > - What components are in a whole IVI system, from the point of view > > of Wayland protocol? What are the responsibilities of each > > component and how are these distributed into processes? > > ivi-control : Is the only process which is allowed to control the > compositor itself. But they need some informations about other > applications which are running. In the ivi-domain, each graphical > application has a unique ID which is defined by the OEM or platform > provider. ivi-client : are just other applications like browser or > navigation. Ok, so you probably need a protocol interface to tell the compositor the unique ID, and perhaps an authentication/authorization interface, so that a shell plugin has all the needed information to do the controlling. > > - What does a typical IVI application do in terms of Wayland > > protocol? Are you using wl_compositor at all? Or any other > > Wayland core interfaces? > > We were focused on just using the wayland core interface. But we are > open to extend this approach. For the record, "Wayland core" is everything in http://cgit.freedesktop.org/wayland/wayland/tree/protocol/wayland.xml except wl_shell and wl_shell_surface, which are specific to desktop applications. But you are free to pick and choose individual global interfaces. > > The protocol you propose seems to have many references to > > "id_native" and "native content", what is all this "native" stuff > > about? Or all the integer id's you seem to be sending back and > > forth, why can't you use real protocol objects to refer to those? > > This is a little bit confusing, the initial set of these objects like > it is defined here : > > http://git.projects.genivi.org/?p=layer_management.git;a=blob;f=wayland-ivi-extension/protocol/ivi-shell.xml;h=6092b13ef31e55f3db338be2e471d0e956be5b9e;hb=ivi-extension > > Is focused on using the ivi-surface instead of the native id. The > rational behind that is to define unique ids over the complete > process space. Therefore the HMI is able to control the different > ivi-surface properties. Ok, you are inventing global names for these protocol objects, so you could do "window management" in a separate process, right? If you made the HMI controller a compositor plugin, these global names would not be needed, perhaps? Perhaps I still do not fully grasp what the HMI controller is. And I have more of your emails to read still. You may have a very good reason to keep the HMI controller separate, but the thing I am questioning is, is the interface towards the compositor the right one. Should the controller really be responsible for window management, or should it just send simple messages like "rear view camera on/off" or even "gear: reverse". Even if the HMI controller was one big block in your architecture diagrams, you could split it into a compositor plugin and an external process. Then the private Wayland protocol extension they use to communicate with each other will be just an implementation detail of the whole block, which you might not need to standardize. There actually exists examples of such a design: most of the Wayland-enabled EGL implementations use a private protocol extension to communicate between the server and the client. The trick is, both the server and the clients load the same libEGL, so the protocol becomes a completely hidden implementation detail. > In the end we are very lucky about any input to get a final best > fitting solution for both the ivi aspects and the final > implementation for that in the FOSS domain. I am very glad you came forward. :-) Thanks, pq _______________________________________________ wayland-devel mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/wayland-devel
