Re: Gnome Media Library

[Erik Walthinsen <omega cse ogi edu>]

On Fri, 24 Mar 2000, Lauris Kaplinski wrote:

> OpenML probably does not fit well with the style of OpenGL & OpenAL,
> as the latter are:
> - extremely low-level
> - 100% procedural
> We need high-level, object-oriented library - probably sitting on top of
> OpenAL.

OK, I'm going to stick my neck out and suggest GStreamer as low-level
library for GML.  It's been mentioned before in this thread, and in fact I
think it was partially responsible for the thread in the first place
(Nicholas and I've been talking about this stuff for a couple days).

GStreamer (gstreamer.sourceforge.net) has been around publicly since
October 31st 1999 (the 'GScreamer' release, think Halloween), but it was
in development since before the Linux Expo 99 (the first iteration was
finished and tossed on the way there, in the Atlanta airport IIRC).  It's
based heavily on the concepts of both DirectShow and the multimedia
pipeline developed at the Oregon Graduate Institute, where I'm a staff
programmer doing research in multimedia streaming.

It's currently written in the GtkObject style, which as of now makes it
dependent on Gtk and an X display, but that will change as soon as GLib
1.4 comes out with GObject.  That will make it portable to any
architecture that handles GLib, with some exceptions (which I guess I
should talk to the GLib list about, they could be added there).

GStreamer is based on filter graphs, where you've got elements that are
responsible for one function or another, with connections between them
designed for streaming of buffers with varying forms of media.  An AVI
player would look like:

DiskSrc  ->  AVI parser  ->  Codec  ->  VideoSink

Media control handled by the parser, which affects the source as
necessary.  This means device control doesn't exist in the typical sense,
it's really stream control that tells the device what to do.  Of course,
this isn't strictly the case, you can bypass any of this.

Currently GStreamer has almost 50 plugins that do anything from reading
VCD's to decoding mp3 (I've got three of those).  At one time in the
recent past (it seems broken at the moment) it was capable of nicely
playing MPEG streams with both audio and video, sycnronized, and faster
(aka cheaper) than SMPEG IIRC. 

More core features include a simple object-oriented (a la GtkObject) C
interface, plugin system (so binary-only codecs are trivial), rudimentary
media-type system (needs more work), ability to save (not load yet) entire
filter graphs to XML, and soon to be in CVS you'll find an advanced
multi-level threaded execution model, using both pthreads and cothreads
(this is the bit GLib needs to have to be fully portable).  I've even got
a graphical pipeline editor, though it needs another rewrite in order to
be useful.  With that puppy I can *draw* an mp3 player and play music in
15 seconds flat.

For more information, I've got a set of slides I used to present GStreamer
to the group at work, as well as some basic (very incomplete, I'm not
puling a GVoice here...) docs.  More recent info is on the mailing list on
sourceforge, I've been on an explanation binge for the last 24hrs.  In
particular, the gory details of the threaded execution model were posted
*early* this morning.  More is coming, too.


Nicholas's idea as I understand it is to build GML on top of GStreamer.
This means providing the things that GStreamer doesn't, which means any
aggregate component.  I'll quote from a post on gstreamer-devel by Wim
Taymans (who's doing a lot of work on GStreamer right now):

> - have a stream inside a word/spreadsheet document.
>      * we probably need autodetect
>      * the current videosink can probably handle it
> - a little gnome applet to watch TV (in the panel)
>      * need more fluf around the basic plugins (channel selection)
> - arbitrary locations for the plugin elements.
>      * need network protocols (might kill performance?)

Thus GML would provide a Bonobo component that would have set_file(),
play(), stop(), seek(), and a few other random interfaces.  It's effect
would be to provide a window that plays a video of any format.  It could
optionally provide the actual control widgets that correspond to the above
interfaces, thus it's a complete video-playing component.

I just took a quick look at some of the BeOS documentation, and this looks
quite similar in basic design to what they have.  Their example (a sound
recorder) has a SoundProducer and SoundConsumer, and a CaptureWindow.  In
the GStreamer/GML model, the Sound bits would be GStreamer elements that
would be wire together, and the CaptureWindow would be part of GML, a
component that 'owns' the Sound pipeline, as well as providing the
graphical representation, controls, etc.

I assume M$ has the same setup of a sort, where DirectShow (or whatever
they call it these days) is the low-level provider of media-handling
capabilities, and the core component of the Windows Media Player ("WiMP"?) 
simply wraps them up into a friendly little thing with a sanitized set of
interfaces and a GUI.  Of course, WiMP comes with all sorts of other
cruft, which is why we all want an Open replacement.

Work will be proceeding at an increased pace on GStreamer in the near
term.  I'm getting some core stuff dealt with right now, and Wim and I are
developing some of the optimization interfaces.  I'll also try to get the
website redone to make it more useful for developers interested in the
guts of the project.

TTYL,
    Omega

         Erik Walthinsen <omega@cse.ogi.edu> - Staff Programmer @ OGI
        Quasar project - http://www.cse.ogi.edu/DISC/projects/quasar/
   Video4Linux Two drivers and stuff - http://www.cse.ogi.edu/~omega/v4l2/
        __
       /  \             SEUL: Simple End-User Linux - http://www.seul.org/
      |    | M E G A           Helping Linux become THE choice
      _\  /_                          for the home or office user