Hi,
We are developing a video processing system that runs on an NXP imx8
processor using a Yocto embedded Linux system that has Qt6, GStreamer,
Wayland and Weston.
We are having a problem displaying the video stream from GStreamer on a
QWidget. In the past we had this working with Qt5 and older GStreamer,
Wayland and Weston.
A simple test program also shows the issue on Fedora37 with QT6 and
KDE/Plasma/Wayland.
The technique we are using is to get the Wayland surface from the
QWidget is using (It has been configured to use a Qt::WA_NativeWindow)
and pass this to the GStreamer's waylandsink which should then update
this surface with video frames (via hardware). This works when the
QWidget is a top level Window widget (QWidget(0)), but if this QWidget
is below others in the hierarchy no video is seen and the gstreamer
pipeline line is stalled.
It appears that waylandsink does:
Creates a surface callback:
callback = wl_surface_frame (surface);
wl_callback_add_listener (callback, &frame_callback_listener, self);
Then adds a buffer to a surface:
gst_wl_buffer_attach (buffer, priv->video_surface_wrapper);
wl_surface_set_buffer_scale (priv->video_surface_wrapper, priv->scale);
wl_surface_damage_buffer (priv->video_surface_wrapper, 0, 0,
G_MAXINT32, G_MAXINT32);
wl_surface_commit (priv->video_surface_wrapper);
But never gets a callback and just sits in a loop awaiting that callback.
I assume that the surface waylandsink is using, which is created using
the original QWidget surface (sub-surface ? with window ?) is not
"active" for some reason.
I am trying to debug this, but this graphics stack is quite complicated
with waylandsink, qtwayland, wayland-lib and Weston not to mention the
NXP hardware levels. My thoughts are that it is something qtwayland is
doing with the surface stack or thread locking issues (gstreamer uses
separate threads). I also don't understand Wayland or Weston in detail.
So some questions:
1. Anyone seen something like this ?
2. Anyone any idea one where to look ?
3. Given the wl_surface in the Qt app or in waylandsink is there a way I
can print out its state and the surface hierarchy easily ?
4. Any idea on any debug methods to use ?
Cheers
Terry
