Re: GStreamer regression analysis [was: GNOME and GStreamer]

[Tim Müller <t i m zen co uk>]

On Sun, 2006-01-15 at 20:28 -0500, Ronald S. Bultje wrote:

Hi Ronald,

first of all thanks for your analysis and for filing bugs about most of
these issues.

> Summary of results: totem does some good things and also does not do
> some things, it has quite many regressions (as Christian already
> mentioned). The mozilla plugin and nautilus properties pages are
> completely broken.

I haven't actually done any work at all yet on either of these two yet.
The properties page should be easy to fix up, and I don't know much
about the mozilla plugin, but I can't imagine there being big issues
that make it hard to fix for the 0.10 backend. I think the mozilla
plugin worked at least at some point with 0.10.


> The thumbnailer sometimes works and sometimes not.

Yes, there are indeed some issues, and I had noticed the drop in
performance as well. Most of these issues are directly related to
playback issues and will get fixed as a side-effect of fixing those
issues in the various GStreamer plugins.


> Part 1, totem.
> 
> First, general:
> - appears unstable; I've reported several crashes, hangs, 100% cpu
> cases, etc. in bugzilla.

Thanks for the detailed bug reports. We'll be working on those. Some are
known issues. It might be worth mentioning here that Edward has set up
an extensive media test suite that shows how we'll we're doing with a
whole bunch of media files, so we can monitor progress and keep an eye
out for regressions.


> ..., I get multiple or no error dialogs, which gives the
> impression of bad integration of GStreamer backend in Totem
> frontend.
> - the video screen does not update when a menu is expanded and then
> clicked away. The video screen becomes cyan (?!), instead of the nice
> Totem logo from Jakub.

Ugly, but rather minor issues IMHO.


> - first start takes hellishly long (>10s on my laptop), apparently
> because of a registry rebuild. You'll need to provide some user feedback
> here (e.g. a splash screen), the user will think the app is broken and
> Linux (or GNOME, for that matter) sucks.

That's a long time indeed (even if it only happens once). A better
solution though would be if packages with plugins ran gst-inspect-0.10
in their post-install, which should eliminate the need to automatically
re-build the plugin registry when starting a GStreamer application after
that, as gst-inspect has already caused it to be rebuilt after package
installation. I consider this a packaging problem in the end. We should
celebrate the fact that there is no more gst-register in 0.10 by the
way, after it caused quite a few problems in 0.8.


> Video playback:
> - mpeg, vob: basic playback works, slider is wrong after seek, hangs on
>              eos. No metadata provided
> - avi: appears to work. seeking is very slow.
> - matroska, mkv: appears to work, hangs on eos.
> - quicktime, mp4: some movies work, some movies don't. Seeking is
>                   broken. Hangs on eos.
> - ogg, theora: appears to work, no video metadata provided.
> - asf, wmv: doesn't work.

I don't get these hangs at EOS, but I think it can be assumed that they
are all related.

The missing metadata should be trivial to add back. For the theora case
that's already fixed in CVS.

The mpeg demuxer in -ugly still has some issues, that's true, but here
are multiple people working on that. Don't know if you've also tried the
(open source and free) fluendo mpegdemuxer; it works really well for me.

asf/wmv: doesn't work because the required demuxer (asfdemux) hasn't
been ported yet. Or rather: I haven't committed the port yet, but I
intend to do so soon. I believe Fluendo is also working on an asf
demuxer of their own, so there will actually be two demuxers to choose
from in the not so distant future.


> - gaming formats (quake, final fantasy etc.): doesn't work.

Don't know much about those, but I suspect there are decoders for those
in ffmpeg and it's just gstffmpegdec that needs fixing up?


> - vcd, dvd: doesn't work.

The required sources haven't been ported yet, and there are probably
other issues in addition to that. Don't think DVD playback is going to
be fixed in time for the feature freeze, but it will be fixed
eventually, no question about that (can't really say I have ever been
able to get DVD stuff working at all in totem-0.8 on any of my three
machines though, so I'm not sure how much of a regression that is).


> - multi-language/language-selection (audio in mkv, ogm): the files I
> tested didn't work, but that may be related to other problems... I
> haven't been able to actually get the language selection menu item
> sensitive for any multi-language file I tried... Unsure if this works or
> not. Probably doesn't [1].
> - subtitles (subtitles embedded in mkv, ogm, or separate text files):
> doesn't work [1] [2].

Both are related and both don't work yet as far as I'm aware, but the
infrastructure is mostly in place now as far as I know, so I'd expect
this to be working by the time 2.14 is released.


> Audio playback:
> - wma (Windows Audio): doesn't work.

asfdemux, see above.

> - mp3: appears to work. Unsure about non-utf8 tags.

non-utf8 tags work just like in 0.8.


> - ogg, vorbis: works, except for minor metadata twitch.

Already fixed in CVS.


> - m4a: wish I could say, can't find sample files. :-( I apparently
> removed all of them from my HD...

Plays, but seeking is broken (same issue as the quicktime/mp4 one
above).


> Nautilus properties page:
> - blank for all file types above. Doesn't work.
>
> Firefox plugin:
> - doesn't work for the few file types I could find in websites (wmv,
> mov).

As I said above, I hadn't looked at these two yet, so I'd been surprised
if they worked. Will fix.


> Part 2, gnome-media

> gstreamer-properties:
> - should be moved back to some behind screen, since gst now contains
> autodetection elements and most, if not all, users will no longer need
> gstreamer-properties.

True, especially if input selection is added to the sound recorder.


> gnome-sound-recorder:
> - playback is broken for all formats.

Used to work fine for me just the other day, but doesn't seem to now.
Not sure what's happening there. Can't be anything big.


> I'd like to suggest that relevant maintainers & co give ideas on
> how those will be fixed and which regressions will be left open for
> 2.14, from there on we can decide if it's worth it for GNOME to jump
> ship and ship 0.10 rather than 0.8.

First of all, let me say that as far as I can tell many of the issues
you found are related:
  - all the 'hangs at EOS' are most likely related
  - asfdemux port is not in CVS yet, hence no asf/wmv/wma
  - qtdemux needs more love, incl. seeking fix.
  - stream selection doesn't work right yet (can't choose between
         languages, matroska films with subtitles don't start)

 - totem-video-thumbnailer needs some love, files that don't start up to
         play won't work here either (plus a video format
         conversion issue in the thumbnailer itself).
 - nautilus property page and mozilla plugin need work
 - gnome-sound-recorder: playback doesn't work at the moment, but used 
         to before (for me)


When will all this be fixed? That's hard to say. As soon as we possibly
can fix it. I am already working on it, and so are others.

There is no doubt there are regressions compared to 0.8 at the moment.
These are being worked on though, and they will continue to be worked
until all the indicators in the media test suite are green, whatever
GNOME decides to go for in the end.

Having said that, I don't think everything was quite as peachy in 0.8 as
one might be led to believe from your comments. There's whole bunch of
0.8 media playback issues that have been fixed in 0.10 (and many more
non-playback issues).

The two issues that require a bit more than just a few fixes in 0.10
are:
 - DVD playback
 - stream selection/multiple languages/subtitles

Playing back specific titles of a DVD doesn't require a lot of extra
work (and I'm quite keen on making that work for thoggen as well). That
will probably be possible before 2.14 comes out.

DVD menus are another thing. That will probably take a bit longer and I
don't know anyone who's got plans to work on that at the moment. 

There are plans to fix stream selection and subtitles, but I don't know
when that will be. Stream selection requires a bit of design work from
what I understand, but it's being in the works. Totem should
automatically be able to make use it once this is fixed in GStreamer.


> 0.8 is not unmaintained, btw, I intend to do a new release containing
> all fixes I provided to make the totem firefox plugin work fine. There's
> also a crasher fix in there, IIRC. If anything else pops up, I'll
> consider including that, also. If there's serious bugs, I may even have
> a look at them.

That's great, and I'm sure the 0.10 backends/plugins will be able to
benefit from your work on that as well in the end.

And while there's little doubt in my mind that if there is someone who
is able to single-handedly maintain the 0.8 series on his own then it'd
be you, there's also a whole bunch of issues that are simply completely
impossible to fix in 0.8, and even you won't be able to fix them. This
includes all those crashers deep down in the guts of the 0.8 scheduler
and all kinds of other threading-related issues, as documented in
various bug reports.

0.10 may not be as good as 0.8 yet in every respect, but we're talking
about a timeframe of 6-9 months from March here (and it is already
better in many other respects). A lot of progress will be made until
March, and it will continue to be made after that. 0.8 on the other hand
will more or less stagnate minus a few crasher fixes here and there, and
its progress will depend entirely on how much free time and motivation
you have besides your other committments.

As a GNOME user as well as a developer I really hope that GNOME will
decide to go for the GStreamer version that is actively being worked on
and constantly being improved, even if that means temporary feature
regression.

Enough rambling from my side. I'll better go back to working on some of
the issues that have been brought up :)

 Cheers
  -Tim