Re: designing a wrapper lib

[Jorn Baayen <jorn nl linux org>]

On Wed, 2002-09-18 at 22:04, Thomas Vander Stichele wrote:
> I've been largely silent on the issue, I wanted to wait and see what 
> everyone was thinking here ;)
> 
> > Assuming everyone agrees we need to use one wrapper library for all
> > gnome core apps, I want to start up a brainstorming session on what we
> > want to have in the lib (let's just call it gmedialib for now, to avoid
> > any references to either mm or gstplay). When we have a list of things
> > we need we all agree on, we should start to design an API, implement it,
> > and push it as new standard lib.
> 
> This is indeed exactly what we need.  At this point it's not really 
> relevant which lib is going to be used.  What is important however is that 
> we find a good set of features that our library should offer to Gnome 
> applications.  The main thing here, is that we want to get multi-media 
> complexity out of the way of app developers.
> 
> Now, one thing that I think is very important, is that we shouln't 
> abstract all of GStreamer away.  It should still be possible to get at 
> GStreamer itself.

Yes, I agree.

> 
> Now, I first wanted to discuss how we can find out about these features we 
> want to provide.
> 
> It is REALLY important that we succeed in getting good feedback from 
> NON-multimedia people.  This is not as easy as it sounds.  If we just ask 
> them, "What do you want", they won't know.  We need to help them find out 
> what they want and need.  As to how we do that, not sure yet ;) Providing 
> examples might be a start, but it will also limit their vision.  For 
> example, I hadn't thought of orph's cool nautilus hack.
> 
> It is also REALLY important we get feedback from experienced hackers.  I 
> am not one of them ;) We need them to go over our API, and help us out 
> with some questions.  But, bear in mind that these guys are busy ;) we 
> want to maximize their input by only bringing them in on specific issues 
> and when we know what we want to do.
> 
> 
> Next, on to some features.
> 
> Obviously, jorn made a good starting list -
> 
> > - simple audio/video playback
>   is important; we must also figure out a way for people to get at the 
>   actual output.  This is harder than it sounds, and I don't know enough
>   about the underlying X mechanisms to say much here, but I'm sure we can
>   find some GDK people to let us know what we need to provide.

Yeah, there's already a gst video widget isn't there? I'd guess we could
use that, although I've never tried it.

> 
> > - gapless transition/crossfading (Not sure about this one, at least RB
> > will need it, but i can imagine other apps needing it. It's just a very
> > nice feature that would ideally Just Work on any mediaplayer.)
> 
> This is a very tricky one.  I've worked on stuff like this in all sorts of 
> projects.  If you do this, you also need a way of defining how to do this 
> crossfade.  Which amounts to timing, which leads to synchronization, and 
> so on.  This isn't even done in monkeymedia yet - I keep trying to get 
> round to it but you know how to-do lists are.

Crossfading might not even be necessary, well maybe, but personally i
only foudn it annoying. The thing i really want though is gapless
transition, which is also cool for less complicated playlisted apps,
imho.

> 
> > - relative volume
> 
> Not sure what you mean here.

Ah, I jsut meant setting a volume on streams.

> 
> > - equalizer backend, perhaps a widget as well?
> 
> Personally, I've never found this really important, because everyone uses 
> this sort of thing wrongly ;) For most apps, this is totally unnecessary.

I don't care all that much either, I've never used it since xmms eq is
crap... 

> 
> > - visualization
> 
> Would be nice to have.  Maybe not in a simple API though.

Yeah, well I guess it would be nice. Esp. since sometimes you need to
detect whether to use stereo2mono or not, it would be nice to have this
easy to use.

> 
> > - good audio cd support (with things like eject and cd availability
> > signals as well)
> > - metadata reading and writing
> 
> Yes, I agree.  I hope we can get this tied easily to GStreamer, where it 
> makes the most sense since that way it doesn't force dependencies on our 
> lib.  Wim put a patch into GStreamer as a proof of concept for mp3.

Cool, could you give a quick explanation how it works? I'm curious :)

> 
> > - internet radio support
> 
> Not sure if this a good idea at this point.  Especially since there's not 
> that much to support anyway - we can only provide mp3 listening and ogg 
> listening (which is still mainly unused).

We have it in monkeymedia ATM, it's trivial to do, since all we need is
a signal on the stream object emitting new metadata when it's available.
(gnomevfssrc already cares about forwarding metadata (with that patch),
and already plays from http and stuff).

> 
> 
> Here are some other things that might be important:
> 
> - thumbnailing of video
>   Orph from ximian made a really cool nautilus hack, which thumbnails
>   videos using GStreamer, then plays the video instead of the thumbnail
>   on mouseover. The thumbnailing or framegrabbing makes a lot of sense to
>   get into this lib

Yeah, that would be very nice.

> - statistics on audio/video files
>   for audio, that would mean, the fancy bits you can see in cooledit.
>   Things like RMS, max. value, stuff like that.  Not necessarily for 
>   end-user UI display, but for manipulating the audio.

Well, this might be a bit out of scope, since the lib is not meant for
audio manipulation, right? Also I doubt it would be used much.. except
for sound editing apps :)

> - the "ding" api (Christian's term)
>   We should provide a simple "play this sound" function (which is really
>   simple to do with gstreamer, would be a twenty line function), but
>   it'd be nice to have one that's preloaded somehow.  In GStreamer terms,
>   this would mean that the pipeline would already exist, part of the file
>   would be loaded, and stored in a buffer, and it would instantly play
>   when requested, then getting the rest of disk as necessary.

A function to preload a stream would suffice for this probably, since
you could just hold the preloaded stream and play it whenever needed.

> - there should be a way to terminate sound events as soon as possible.
>   This is important for accessibility (correct me if I'm wrong, Bill, 
>   because I got this idea from what you said in Boston), because you
>   want to be able to skip through e.g. a webpage real quickly.  The sound
>   should start and stop with things you do in the UI.

That should just work by stopping the stream, it would be odd to have a
special func for this :)

> - subtitle playing

Yeah, good one

> 
> Now, my take on this is that we should start with a few important core 
> features, and design the API in such a way that it can be easily used by 
> these apps, regardless of how it is implemented right now.  I'm not sure
> we'll succeed in setting up an API and getting it right at our first try.
> Which also makes me believe that we should focus on a small set of 
> necessary features.  IMO, we should punt at this point on things like 
> internet radio, equalizing, and so on.

Yeah.

Cheers
Jorn

> 
> But yes, the API should either not 
> define those functions yet or leave place to implement this later.
> 
> Hope all of this makes sense to you guys,
> Thomas
> 
> -- 
> 
> The Dave/Dina Project : future TV today ! - http://davedina.apestaart.org/
> <-*-                      -*->
> Cause I wake to find there's
> nothing left of me in her
> nothing more than a heart still at war
> <-*- thomas apestaart org -*->
> URGent, the best radio on the Internet - 24/7 ! - http://urgent.rug.ac.be/
> 
> _______________________________________________
> gnome-multimedia mailing list
> gnome-multimedia gnome org
> http://mail.gnome.org/mailman/listinfo/gnome-multimedia
>