Hi all, we will come up with a new solution, at clarify what needs to be standardized and why.
Regards Michael. >> Hi Michael-san, >> >> As Pekka and Jason suggests in feedback, shall we clearly explain two >> topics before stepping in details. - what ivi-shell tries to >> standardizes - why ivi-controller tries to conrol surfaces and layers >> in separated process from a compositor. >> >> @Pekka and Jason, >> Thank you for many feedbacks. does it make sense about two topics to > be clarified at first? > > Yes, I think that sounds like a good idea. > --Jason Ekstrand > >> >> Best regards, >> Tanibata >> >> >> 2013-09-12 20:13 に Pekka Paalanen さんは書きました: >> >>> On Mon, 9 Sep 2013 08:01:51 +0000 >>> <[email protected]> wrote: >>> >>>> Hi all, >>>> >>>> let me give some technical background on the extension, and why we >>>> have decided to design the first approach of the extensions like it >>>> is published from Tanibata-san. In general, we would like to get the >>>> feedback on the extensions, to get it fitting in the FOSS approach, >>>> fulfilling the ivi requirements. See my comments in line. >>> >>> >>> Jason Ekstrand had an excellent point asking which parts you are >>> actually looking for to standardize. >>> >>>>>> Hi, >>>>>> >>>>>> I would like to get feedback about that if somebody has a similar >>>>>> motivation to support ivi as well as desktop and tablet. So it is >>>>>> not a stone, just a proposal. If somebody has good idea, I would >>>>>> like to use it or collaborate it. >>>>> >>>>> Ok, I just had the understanding that the layer manager simply has >>>>> to be a separate process and not built into the compositor. >>>> >>>> Not really. Currently in the compliance from genivi, we have the >>>> separation of Graphic Backend Server and LayerManagement. But this >>>> should now change we want to skip that approach and want define a >>>> protocol + ivi extensions, including a reference compositor. We have >>>> decided to use wayland + ivi extension as compositor protocol and >>>> weston as the reference implementation for that. >>> >>> >>> Very cool! >>> >>>>> If that is not the case, then that is very good news indeed. >>>>> Everything that manages surfaces, layers, windows, or whatever >>>>> belongs into the compositor process, where they are much easier to >>>>> implement and you don't need to introduce interfaces and IPC which >>>>> are later hard to develop further and cause latencies. >>>> >>>> Sure that is the technical best fitting solution for that, we >>>> do not want to use separate IPC for the composition approach. >>>>> So I'm roughly on the same track as Andreas Pokorny. >>>> >>>> We too, but unfortunately it is not possible all the time to get >>>> the complete user interface included in the compositor itself. We >>>> have to guarantee fast startup time to get early video running in 2 >>>> seconds. On different systems we can not achieve that with complete >>>> HMI-UI is up. >>> >>> >>> Oh, no, I am not suggesting to put user interfaces in the compositor, >>> not at all. Only the window management, which means window >>> positioning, z-order, etc. For instance, in Weston virtual desktops >>> are implemented in the compositor, and not by an external process >>> juggling window lists via IPC and setting visibilities and positions. >>> >>> Proposing just like with weston and desktop-shell, there is the weston >>> plugin called desktop-shell.so which does window management, and then >>> there is the helper program weston-desktop-shell, that does all the UI >>> drawing in a separate process. >>> >>> Like Jason Ekstrand explained, desktop-shell.so and >>> weston-desktop-shell communicate with each other not only using the >>> core Wayland protocol, but also using a private (or proprietary) >>> protocol extension. >>> >>> While the basic IPC mechanism is all the same (Wayland), there are >>> different parts: extensions. The desktop-shell extension is actually a >>> protected one, only weston-desktop-shell as started by desktop- >>> shell.so can access it. >>> >>>>> Have you tried to map your IVI concepts of surface/layer/display to >>>>> Wayland wl_surface, wl_subsurface, and wl_output? I don't really see >>>>> what kind of interfaces your applications (Wayland clients?) expect >>>>> to use. >>>>> >>>>> When I look at the protocol in ivi-shell.xml, I get the feeling >>>>> that: >>>>> - Interfaces ivi_layer, ivi_controller_surface, >>>>> ivi_controller_layer, ivi_controller_screen, and ivi_controller >>>>> should be internal implementation details inside the weston process, >>>>> not protocol. Having these as interfaces looks like the X >>>>> architecture, where the X server process and window manager process >>>>> continuously struggle to keep each other up-to-date, and carefully >>>>> try to keep state in sync (and fail), which also makes races and >>>>> glitches practically unavoidable. - Interface ivi_client is just a >>>>> reinvention of wl_compositor and wl_subcompositor. - Interface >>>>> ivi_surface is a reinvention of wl_surface. >>>> >>>> Let me explain, why this separation occurs. From the ivi perspective >>>> we have separated the different applications in ivi-client, >>>> ivi-control. >>>> >>>> ivi-client: Is just a client, like browser, navigation which is >>>> embedded in the browser. That belongs to the ivi-surface. These >>>> process have really limited access the can just create the surface >>>> and that's all. The only important point which the want to know is if >>>> the visibility has changed and the size to adapt the rendered >>>> content. A navigation should not render the content if it is not >>>> visible or assigned to a layer. >>> >>> >>> That is actually very well fitting to what we already have in Wayland. >>> Just replace ivi-client with a wl_client (that's not a protocol >>> object, it represents a display connection from a client process), and >>> ivi-surface with wl_surface+wl_shell_surface. >>> >>> The limited access is already there: in Weston, clients are free to >>> create any number of wl_surfaces. However, no surface is shown >>> anywhere on screen, unless it is assigned a suitable role. Examples of >>> roles are: "pointer cursor", "top-level window", "drag icon", and >>> "popup (menu)". Without a role, the compositor cannot know what to do >>> with a surface. >>> >>> "top-level window" and "popup menu" are shell concepts. If you replace >>> wl_shell with your own ivi_shell, you could have roles like: "TV >>> output", "rear camera", "speedometer". I'm totally pulling these out >>> of my hat, I don't know what actually makes sense for you. >>> >>> Then, your shell module in weston could handle such surfaces as you >>> want. For example, if you get a surface with the role "TV output", the >>> shell plugin could show that surface cloned on all entertainment >>> monitors. Or whatever. >>> >>> Or if something signals that now would be a /really/ good time to show >>> the rear view camera, then your shell plugin can make it so, if there >>> is a surface with such role assigned. >>> >>> Now, something like a "rear view camera" sounds like a pretty >>> important role for a surface, so you may want to make it privileged. >>> You don't want any random application to claim to be a rear view >>> camera. In that case, you can simply add whatever authentication you >>> need to do in the shell plugin. For example, only this one process >>> started directly by the shell plugin is allowed to claim the rear view >>> camera role. Anyone else trying that will get killed. On the protocol >>> design, such protection is pretty irrelevant. All you need is for the >>> shell plugin to know who is authorized for what, and it can then >>> enforce the restriction, whether the privileged role is requested by a >>> method in a otherwise public protocol extension or in its own >>> extension. >>> >>>> ivi-controller : This is the main application which is controlling >>>> the compositor like the UI. That could be the hmi, but it can also >>>> some debug applications and application which is triggering a screen >>>> shot of surfaces, layer, displays. If you driving a car during the >>>> development you want to get some screenshots which are normally >>>> triggered by an external application like the log and trace > framework. >>> >>> >>> Ok. I am proposing to have window management in the compositor plugin. >>> But, you can offer additional debugging interfaces in the protocol. >>> >>> For example, Weston does not allow clients to control *any* input >>> devices. Clients simply cannot feed fake input events to other >>> clients. But for testing and debugging, we need a way to fake input >>> from a client. The solution: a test plugin. >>> >>> The test pluging, which is loaded by the compositor only for testing >>> (in your case debugging) purposes registers a new protocol extension: >>> the test interface. That test interface then allows to do all kinds of >>> ugly things from clients. >>> >>> I'm not sure what you mean by "UI", but we separate controlling >>> windows from drawing content, like I explained of the desktop shell. >>> >>> As a recap, the shell plugin is what enforces and executes policies. >>> Clients only indicate intent or purpose, like "This surface is a menu, >>> and I want it to pop up right there on my window." A menu is an >>> interesting example, since it is more than just a surface role: it >>> comes with an input device grab. While the menu is up, input is >>> directed to it. But, it is the shell plugin that can decide, that ok, >>> you're done now, grab is broken, the menu surface hidden, and the >>> client is signalled that something dismissed the menu. Also, clients >>> cannot put up a menu at random times, it has to be done as a response >>> to an input event. All this prevents many kinds of client misbehaviour. >>> >>>>> Yes, I see there are some details to may want to control like >>>>> surface opacity, that the current Wayland protocols do not support, >>>>> but I don't think that replacing everything is a good way to start. >>>>> It is also very hard to see how objects from all these interfaces >>>>> are created, and how (if?) they associate to any other protocol >>>>> objects. >>>> >>>> On goal is to get setup a complete scenery during the startup of the >>>> compositor. Therefore the compositor is able to just initialize the >>>> main visible attribute during startup. That implies too that it >>>> should be possible, without the final application is running (like >>>> navigation). Therefore we need some logical objects like ivi- >>>> surface, ivi-layer. Which can be already controlled, whithout a >>>> application which holds the content is already running. >>> >>> >>> But if all layer and window management is internal to the compositor >>> process, then you don't need any of this control protocol. >>> >>> Just create some internal data structures, and when a client comes up >>> with a surface and requests a particular role, you can plug that >>> surface into the slot you prepared in advance. >>> >>> So, yes, this does imply that the compositor or the shell plugin knows >>> the scenegraph, but... isn't that exactly the compositor's purpose, to >>> gather the scenegraph and get it on screen? Clients "only" provide >>> content. >>> >>> For example, the desktop-shell protocol extension has special surface >>> roles "background" and "panel". When weston-desktop-shell client >>> assigns these roles to its surfaces, the compositor (shell plugin) >>> will immediately plug them into the correct places on screen. If the >>> shell plugin wanted to reserve screen space for them in advance, it >>> could. After all, the shell plugin is responsible for arranging >>> everything on (and off) screen. >>> >>>>> Btw. if you need support for surface scaling and cropping, there >>>>> have been discussion on the Wayland mailing list to bring a crop & >>>>> scale protocol extension to Wayland. It is actually necessary for >>>>> efficient video playback etc., so pushing that forward would be >>>>> nice. >>>> >>>> Yeah I see that, and we will consider it. I think we can redesign the >>>> approach like source and destination region to the crop and scale >>>> instead. >>>> >>>>> After looking through the two links you gave, the ivi-shell.xml, and >>>>> what you have wrote in the emails, I still have no clue what is the >>>>> big picture here. >>>>> >>>>> - What processes are going to use which interfaces? It looks to me >>>>> like some interfaces are not meant for all Wayland clients, but >>>>> how is it supposed to work? >>>>> - What components are in a whole IVI system, from the point of > view >>>>> of Wayland protocol? What are the responsibilities of each >>>>> component and how are these distributed into processes? >>>> >>>> ivi-control : Is the only process which is allowed to control the >>>> compositor itself. But they need some informations about other >>>> applications which are running. In the ivi-domain, each graphical >>>> application has a unique ID which is defined by the OEM or platform >>>> provider. ivi-client : are just other applications like browser or >>>> navigation. >>> >>> >>> Ok, so you probably need a protocol interface to tell the compositor >>> the unique ID, and perhaps an authentication/authorization interface, >>> so that a shell plugin has all the needed information to do the >>> controlling. >>> >>>>> - What does a typical IVI application do in terms of Wayland >>>>> protocol? Are you using wl_compositor at all? Or any other >>>>> Wayland core interfaces? >>>> >>>> We were focused on just using the wayland core interface. But we are >>>> open to extend this approach. >>> >>> >>> For the record, "Wayland core" is everything in >>> http://cgit.freedesktop.org/wayland/wayland/tree/protocol/wayland.xml >>> except wl_shell and wl_shell_surface, which are specific to desktop >>> applications. But you are free to pick and choose individual global >>> interfaces. >>> >>>>> The protocol you propose seems to have many references to >>>>> "id_native" and "native content", what is all this "native" stuff >>>>> about? Or all the integer id's you seem to be sending back and >>>>> forth, why can't you use real protocol objects to refer to those? >>>> >>>> This is a little bit confusing, the initial set of these objects like >>>> it is defined here : >>>> >>>> > http://git.projects.genivi.org/?p=layer_management.git;a=blob;f=wayland- > ivi-extension/protocol/ivi- > shell.xml;h=6092b13ef31e55f3db338be2e471d0e956be5b9e;hb=ivi-extension >>>> >>>> Is focused on using the ivi-surface instead of the native id. The >>>> rational behind that is to define unique ids over the complete >>>> process space. Therefore the HMI is able to control the different >>>> ivi-surface properties. >>> >>> >>> Ok, you are inventing global names for these protocol objects, so you >>> could do "window management" in a separate process, right? If you made >>> the HMI controller a compositor plugin, these global names would not >>> be needed, perhaps? >>> >>> Perhaps I still do not fully grasp what the HMI controller is. And I >>> have more of your emails to read still. >>> >>> You may have a very good reason to keep the HMI controller separate, >>> but the thing I am questioning is, is the interface towards the >>> compositor the right one. Should the controller really be responsible >>> for window management, or should it just send simple messages like >>> "rear view camera on/off" or even "gear: reverse". >>> >>> Even if the HMI controller was one big block in your architecture >>> diagrams, you could split it into a compositor plugin and an external >>> process. Then the private Wayland protocol extension they use to >>> communicate with each other will be just an implementation detail of >>> the whole block, which you might not need to standardize. >>> >>> There actually exists examples of such a design: most of the >>> Wayland-enabled EGL implementations use a private protocol extension >>> to communicate between the server and the client. The trick is, both >>> the server and the clients load the same libEGL, so the protocol >>> becomes a completely hidden implementation detail. >>> >>>> In the end we are very lucky about any input to get a final best >>>> fitting solution for both the ivi aspects and the final >>>> implementation for that in the FOSS domain. >>> >>> >>> I am very glad you came forward. :-) >>> >>> >>> Thanks, >>> pq >> >> _______________________________________________ >> wayland-devel mailing list >> [email protected] >> http://lists.freedesktop.org/mailman/listinfo/wayland-devel > ------------------------------------------------------- BMW Group Michael Schuldt Plattformtechnol. Infotainment-Kompon. Max-Diamand-Straße 13 80788 München Mailto: [email protected] URL: http://www.bmwgroup.com ------------------------------------------------------- Bayerische Motoren Werke Aktiengesellschaft Vorstand: Norbert Reithofer, Vorsitzender, Milagros Caiña Carreiro-Andree, Herbert Diess, Klaus Draeger, Friedrich Eichiner, Harald Krüger, Ian Robertson, Peter Schwarzenbauer. Vorsitzender des Aufsichtsrats: Joachim Milberg Sitz und Registergericht: München HRB 42243 -------------------------------------------------------- _______________________________________________ wayland-devel mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/wayland-devel
