RFC: GNOME 2.0 Multimedia strategy

[Christian Schaller <Uraeus linuxrising org>]

During GUADEC 2 there was some discussion about what we should do with
multimedia support in regards to GNOME 2.0. The general concensus is
that ESD has to go, and there was also a leaning towards replacing it
with artsd in order to both get a better soundserver and to have
increased compatibility with KDE. In this document I will try to outline
why I think that will be a to limited approach. I would like to point
out that I have personally been trying to help out with GStreamer for
some time now, so I could be percieved as biased :)


Why is this important to decide now:

Due to the limited time until the GNOME 2.0 freeze we need to quickly
finalize the strategy for Multimedia in GNOME 2.0 in order for audio and
video applications developers to be able to port their applications in
time.


What should be done:

The basis for my proposal is that we adopt GStreamer
(http://www.gstreamer.net) as the official sound and video API for GNOME
2.0. This will give wide ranging advantages for GNOME.

a)  We will not only have sound but also video support. Gstreamer today
supports AVI, Quicktime, Mpeg, FLI and VOB files and more are in the
works.

b) Greater flexibility in regards to sound output. All GNOME
applications based upon GStreamer will then be able to output either
directly to the native sound architecture or through sound servers such
as ESD, artsd or gnostream. The idea here is to have a controlcenter
applet which will let the user choose a default output type. Today many
of the GNOME multimedia applets and applications either just support ESD
or they have a large library of their own output plug-ins. This we can
now solve in a much better way.  Just porting these apps to artsd would
just give us the same situation with a better soundserver.

c) Using GStreamer will be a better long term solution for GNOME since
it is created using glib for the backend and all the current sample
applications are developed using the GNOME libraries.

d) While artsd is really good as a soundserver, its design is not that
ideal for videoserving. Using GStreamer we can output to artsd for KDE
compatability (gstreamer already supports artsd) but also switch to a
more video&audio based setup like gnostream
(http://gnostream.sourceforge.net)
when it becomes available witouth having to rewrite applications. This
also solves the problem with audio/video sync when using a standalone
soundserver as those of you who talked with Jim Gettys during GUADEC2
probably are aware of.

e) We will have a full featured multimedia architecture making GNOME the
natural development environment for MM applications. Remember GStreamer
isn't a mediaplayer, it is a multimedia architecture which already
supports encoding of videos and sound too.

f) We will have a multimedia architecture with broad developer support
and commercial support. RidgeRun has 1 fulltime and 2 people part-time
on GStreamer. GStreamer has in addition to that we have around 12-13
developers helping out in their spare time or as partime on company
time. Considering the number of commits to GNOME multimedia currently in
CVS I think this will give us a incredible incredible increase in
developer time on our multimedia support.

g) Long time GNOME developers are already working on GStreamer
integration and improvement. Arik Devens for instance is the maintainer
of the gstmediaplay Mediaplayer application bundled with Gstreamer. Erdi
Gergo is working with us to make his bonobo-media compoents work with
GStreamer. Bastien Nocera (hadess) has created gnome-vfs input and
output plugins and is creating a iTunes clone for GNOME based upon
GStreamer.

h) Having something as GStreamer as part of the GNOME development
platform means that we could for instance use it in Nautilus to easily
create a music view which supports all audio formats GStreamer supports
instead of having to create new code for each format. We could also do
video preview if that is of interest.

i) A GStreamer Mozilla plugin is in the works which would be a great
benefit since all our applications using Mozilla for html like Mozilla,
Galeon and Nautilus would be able to easily support multimedia webpages
out of the box.

For a full rundown on what is supported and is worked upon in GStreamer
I suggest taking a look at the current roadmap for the upcomming 0.2.0
release of GStreamer.
http://www.linuxrising.com/files/gstreroadmap2.html


Challenges for the GStreamer project:

Currently GStreamer has a small core component written in assembler
(cothreads). This part is currently only ported to Intel, Sparc, ARM,
Alpha and PowerPC. The remaining asm cores can be writen blind (like
succesfully done with SPARC) but testers to confirm success would be
needed. In the long run these asm parts will probably be replaced by GNU
Pth with IBM's N:M mods which is a pthread safe C-only implementation.

Another problem is that while GStreamer very much want to be part of
GNOME, they also want to keep their independence due to being also
targeted at non-gnome systems, especially embeded ones.  So moving
GStreamer from SF CVS will probably not be of interest to the GStreamer
developers. Maintaining a release library at ftp.gnome.org should
however be no problem.

A major feaure currently missing in GStreamer is MIDI. There is no
special reason for this except that none of the developers have gotten
around to implementing it yet.

Another thing needed is a simple API layer between GStreamer and GNOME
to enable simple stuff like `gnome_play_sound("bleep.wav")` for all
those apps not in need of the full GStreamer API. This will be based
upon the current gstmediaplay API (libgstplay).


What is currently needed:

In order to get this kickstarted and ready to run for the GNOME 2.0
release there need to be a general consensus from the GNOME
hackers that this is the way forward (that would be true either way
actually), so we can send out the message to applet and applications
developers to start looking into porting their applications to the new
API. This decision doesn't need to mean we don't bundle and use artsd
with GNOME 2.0 and use that as the default GStreamer audio output, but
it gives us flexibility for the future and users the option of using
other technologies which might fit their needs better.

Christian