Hi, I started adding support for OpenVR/Vive and native OculusSDK to Qt3D.
You can find it on Github: https://github.com/dabulla/qt3d-vr-example It can easily be setup, as described in the Readme. I've been fiddling around with this for a while and finally found a working approach to get an image to the headsets. *What works?* - Stereoscopic 3D Rendering of Qt3D Qml scenes in the Headset. - Headtracking *What is still in progress?* - Headtracking has high latency - Mirroring to Monitor - Motion controllers + Input - ... (lots of things imaginable) *Implementation* There is QHeadmountedDisplay which does the job of a Qt3DQuickWindow but uses Direct-Mode Rendering. It can be used just like a Window that is placed at the headset and will sync to it's refresh rate. OpenVR or OculusSDK (aka 'SDKs') also handle distortion correctly, so there is no need to do this manually. The rendered Qt3D Qml scene must use a VrCamera to enable head-tracking and correct stereoscopic rendering. VrCamera configures projection-parameters for the headset by querying the SDK. I only tested this on windows at the moment. *Future* I'd really like to proceed with the implementation and look forward to getting some feedback at this point. I read the qt styling guide and started a clean implementation on github and wanted to contribute it to qt3d at some point. I'm open for suggestions and could also need advice with the following problem: *Tackling latency* The pose read from the headset needs approx. 3 frames before it finally ends up as a shader-uniform for rendering. This takes 3 frames too long (it is extremely noticable and you get nausea very quickly)! - In the main-thread "getEyeMatrices" is called, this gets the prediction of the head pose for rendering. - The matrices are commited to Qt3D using the QTransform of the Camera-Entity (VrCamera). I suspect that it takes a lot of time before the transform ends up as a uniform in the shader and wonder how I can tweak Qt3D to keep this latency down. A tip of a Qt3D expert or architect would be really handy here. My ideas thus far: - Skip the frontend and introduce a QVirtualRealityAspect (which has to be done earlier or later anyway). I wonder if this could apply transforms faster. I guess it would not run in the same thread as QRenderAspect? - Introduce a way to set a callback function for a shader uniform. The shader could query a very recent head-pose just before rendering. - Ensure that UpdateWorldTransformJob is executed after setting the transform, but before rendering (would this kill parallelism?). - Don't use synchronous rendering. But this would require me to heavily patch QRenderAspect I guess. At least two not-yet-customizable steps must be altered: - Render to a texture owned by the SDK which we only have a GLuint texture-id of (this is only true for Oculus). - Call a custom swapBuffers method. Hope you enjoy it and don't get sea sick :-)! -- Daniel Bulla
_______________________________________________ Interest mailing list Interest@qt-project.org http://lists.qt-project.org/mailman/listinfo/interest
