Hello All, In order to support features in the WebVR 1.0 API (https://mozvr.com/webvr-spec/) and to improve performance for WebVR, I would like to implement an optimized path for submitting Canvas and OffscreenCanvas frames to VR headsets. The WebVR 1.0 API introduces "VR Layers", explicit frame submission, and presenting different content to the head mounted display independently of the output the regular 2d monitor. I would like some feedback on a proposed “VR Compositor” concept that would enable this.
What would be common between the “VR Compositor” and the regular “2d Compositor”? - TextureHost and TextureChild would be used to transfer texture data across processes. - When content processes crash, the VR Compositor would continue to run. - There is a parallel between regular layers created by layout and “VR Layers”. - There would be one VR Compositor serving multiple content processes. - The VR Compositor would not allow unprivileged content to read back frames submitted by other content and chrome ux. - Both compositors would exist in the “Compositor” process, but in different threads. What is different about the “VR Compositor”? - The VR Compositor would extend the PVRManager protocol to include VR Layer updates. - The VR Compositor will not obscure the main 2d output window or require entering full screen to activate a VR Headset. - In most cases, there will be no visible window created by the VR Compositor as the VR frames are presented using VR specific API’s that bypass the OS-level window manager. - The VR Compositor will not run synchronously with a refresh driver as it can simultaneously present content with mixed frame rates. - Texture updates submitted for VR Layers would be rendered as soon as possible, often asynchronously with other VR Layer updates. - VR Layer textures will be pushed from both Canvas elements and OffscreenCanvas objects, enabling WebVR in WebWorkers. - The VR compositor will guarantee perfect frame uniformity, with each frame associated with a VR headset pose frame explicitly passed into VRDisplay.SubmitFrame. No frames will be dropped, even if multiple frames are sent within a single hardware vsync. - For most devices (i.e. Oculus and HTC Vive), the VR Compositor will perform front-buffer rendering. - VR Layers asynchronously move with the user’s HMD pose between VR Layer texture updates if given geometry and a position within space. - The VR Compositor implements latency hiding effects such as Asynchronous Time Warp and Pose Prediction. - The VR Compositor will be as minimal as possible. In most cases, the VR Compositor will offload the actual compositing to the VR device runtimes. (Both Oculus and HTC Vive include a VR compositor) - When the VR device runtime does not supply a VR Compositor, we will emulate this functionality. (i.e. for Cardboard VR) - All VR hardware API calls will be made exclusively from the VR Compositor’s thread. - The VR Compositor will implement focus handling, window management, and other functionality required for Firefox to be launched within environments such as Oculus Home and SteamVR. - To support backwards compatibility and fall-back views of 2d web content within the VR headset, the VR compositor could provide an nsWidget / nsWindow interface to the 2d compositor. The 2d compositor output would be projected onto the geometry of a VR Layer and updated asynchronously with HMD poses. - The VR Compositor will not allocate unnecessary resources until either WebVR Content is accessed or the browser is launched from within a VR-only environment such as Oculus home, SteamVR, or GearVR. Early WIP patches of implementation of the WebVR 1.0 API are in Bug 1250244. I expect the first implementation to be minimal, but lay the foundation that will eventually become the VR Compositor. Thanks for giving this a review — I look forward to your feedback! Cheers, - Kearwood “Kip” Gilbert _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
