回复：Could we improve video rendering for totem/gtk （wayland）?

["HalleyZhao" <halleyzhao sina com>]

I tried to look into bacon-video-widget.c and totem-object.c

1. seems bacon-video-widget.c is a clutter based backend for totem, it is possible to create other backend (follow the interface defined in bacon-video-widget.h); right?

the backend should subclass from GtkWidget and implement the interface defined in bacon-video-widget.h.

for example, we can implement another backend to render video directly to native window, like xvimagesink or waylandsink. the custom GtkWidget could be a GtkContainer which includes a GtkDrawing area for video rendering and some other buttons for control.

here, I have some question against current Gtk/Gdk implementation. a GtkWidget/GdkWindow maps to a native window like XWindow/wl_surface, right? 

(Gtk isn't like Clutter to share the same XWindow/wl_surface for all the Actor of the same Stage, right?)

2. we can also implement another backend basing on gtkglsink to render video as texture.

since gtkglsink draw video to the GtkGlArea, which is also a GtkWidget; what's the advantage comparing to rendering video directly to XWindow/wl_surface as xvimagesink/waylandsink does?

Is it possible to draw controls on top of the video(GtkGlArea)? like the controls are drawn as ClutterActor on top of video.

--------------------------------

----- 原始邮件 -----
发件人："HalleyZhao" <halleyzhao sina com>
收件人："gnome-multimedia" <gnome-multimedia gnome org>
主题：Could we improve video rendering for totem/gtk （wayland）?
日期：2019年09月16日 17点01分

Hi experts：

    hardware accelerated video decoder becomes basic component in current device, and the video frame type is usually a dma_buf fd ( though the frame type varied in the past years).

    the dma_buf fd can be sent to compositor (weston for example), it can also bind to a texture for openGL rendering.

   however, when I looked into totem code, I found video rendering hasn't been well supported. totem still uses clutter-gst which is out of maintaince for several years. hardware accelerated video rendering is usually broken. (gst-vaapi used to be supported for rgbx frame, but disabled by default).

    so, I want to ask some questions: (let's consimer gtk+3 and wayland environments)

option 1: Is there gtkwidget which can connect waylandsink?

it means video frames can be directly sent to weston w/o compositiong inside app ( to avoid performance hit).

option 2: Is it possible to draw dma_buf fd as texture in gtk context?

I noticed that there is gtkglsink/glupload, however, I'm not sure whether they support wayland/weston; and I'm still not clear how they interact with gtk widget.

thanks.

--------------------------------