Hi great news. Hope you guys will combine efforts to make the solution multiplatform. For example, compatible with GearVR btw, did you checked this project: https://github.com/google/lullaby ?
On Tue, Jul 18, 2017 at 11:22 AM, Sean Harmer <sean.har...@kdab.com> wrote: > Hi Daniel, > > awesome, thank you for this! We too have been starting work on something > similar: > > https://codereview.qt-project.org/#/c/195892/ > > The latency you mention can be avoided if we update the head tracking > directly > on the qt3d renderer backend. That way it can be used immediately on the > current frame. The frontend doesn't necessarily need to know the eye > matrices > but we can make tracking these on the frontend opt-in if necessary (if > something wants to bind to them - but even that use case may become > obsolete > soon with some backend bindings work we have ongoing). > > I've pointed a couple of colleagues at your work and I'll try it out myself > too once I get a few urgent things sorted locally. > > You may not have seen it but, we now have some infrastructure for renderer > plugins to allow deeper integration than having to go via the existing > public > Qt 3D frontend types such as QTransform. I think this will be the way to > remove the latency you mention. > > We should certainly coordinate efforts and see how we can make this ready > for > production use. Feel free to ping us on development@ or on irc (#qt-3d) or > even direct email if you want to discuss this. > > Thank you once again for contributing! > > Sean > > On Tuesday 18 July 2017 10:29:25 Daniel Bulla wrote: > > Hi, > > > > I started adding support for OpenVR/Vive and native OculusSDK to Qt3D. > > > > You can find it on Github: https://github.com/dabulla/qt3d-vr-example > > > > It can easily be setup, as described in the Readme. > > > > > > *What works?* > > > > - Stereoscopic 3D Rendering of Qt3D Qml scenes in the Headset. > > - Headtracking > > > > *What is still in progress?* > > > > - Headtracking has high latency > > - Mirroring to Monitor > > - Motion controllers + Input > > - ... (lots of things imaginable) > > > > *Implementation* > > > > There is QHeadmountedDisplay which does the job of a Qt3DQuickWindow but > > uses Direct-Mode Rendering. It can be used just like a Window that is > > placed at the headset and will sync to it's refresh rate. OpenVR or > > OculusSDK (aka 'SDKs') also handle distortion correctly, so there is no > > need to do this manually. > > > > The rendered Qt3D Qml scene must use a VrCamera to enable head-tracking > and > > correct stereoscopic rendering. VrCamera configures projection-parameters > > for the headset by querying the SDK. > > > > I only tested this on windows at the moment. > > > > *Future* > > > > I'd really like to proceed with the implementation and look forward to > > getting some feedback at this point. I read the qt styling guide and > > started a clean implementation on github and wanted to contribute it to > > qt3d at some point. I'm open for suggestions and could also need advice > > with the following problem: > > > > *Tackling latency* > > > > The pose read from the headset needs approx. 3 frames before it finally > > ends up as a shader-uniform for rendering. This takes 3 frames too long > (it > > is extremely noticable and you get nausea very quickly)! > > > > - In the main-thread "getEyeMatrices" is called, this gets the > > prediction of the head pose for the very next frame. > > - The matrices are handed to Qt3D using the QTransform of the > > Camera-Entity (VrCamera). > > > > I suspect that it takes several frames before the transform ends up as a > > uniform in the shader and wonder how I can tweak Qt3D to keep this > latency > > down. A tip of a Qt3D expert or architect would be really handy here. > > > > My ideas thus far: > > > > - Skip the frontend and introduce a QVirtualRealityAspect (which has > to > > be done earlier or later anyway). I wonder if this could apply > transforms > > faster. I guess it would not run in the same thread as QRenderAspect? - > > Introduce a way to set a callback function for a shader uniform. The > shader > > could query a very recent head-pose just before rendering ("late > > latching"). > > - Ensure that UpdateWorldTransformJob is executed after setting the > > transform, but before rendering (would this kill parallelism?). > > - Don't use synchronous rendering. But this would require me to > heavily > > patch QRenderAspect I guess. At least two not-yet-customizable steps > must > > be altered: > > - Render to a texture owned by the SDK which we only have a > > GLuint texture-id > > of (this is only true for Oculus). > > - Call a custom swapBuffers method. > > > > Hope you enjoy it and don't get sea sick :-)! > > > > > > -- > > > > Daniel Bulla > > -- > Dr Sean Harmer | sean.har...@kdab.com | Managing Director UK > Klarälvdalens Datakonsult AB, a KDAB Group company > Tel. UK +44 (0)1625 809908, Sweden (HQ) +46-563-540090 > KDAB - Qt Experts - Platform-independent software solutions > _______________________________________________ > Interest mailing list > Interest@qt-project.org > http://lists.qt-project.org/mailman/listinfo/interest > -- Best regards, Vlad
_______________________________________________ Interest mailing list Interest@qt-project.org http://lists.qt-project.org/mailman/listinfo/interest
