Re: AudioServer Standard?

[Stefan Westerfeld <stefan space twc de>]

   Hi!

(I was asked to CC that discussion to linux-audio-devel... however
please make sure that the replies arrive still at the gnome-kde list, as
we're trying to discuss whether aRts is suitable as a common audio server
solution for KDE & Gnome).

On Mon, Sep 20, 1999 at 10:10:39AM -0400, Havoc Pennington wrote:
> On Sat, 18 Sep 1999, Stefan Westerfeld wrote:
> > 
> > Currently, there were some discussions on kde-core-devel regarding what
> > should be the audio server for KDE 2.0. I suggested aRts (well, I wrote
> > it, so I think it's a great thing... ;), another solution would be
> > KAudioServer2 and the third solution would be ESD.
> > 
> 
> We are looking for an ESD replacement or at least a rewrite, and
> apparently it has some technical shortcomings (don't ask me for details, I
> don't know anything about sound). I _definitely_ want to have KDE and
> GNOME using the same sound system.

Yes, that would be just perfect.

> If KDE is for sure going to use aRts and people-who-know say it is
> technically better than esound, I'm one of the C++ nerds over here in
> GNOME-land so I'll try to look at porting the client side to C for us and
> hooking it up via ORBit. I don't think we care if the application itself
> is in C++, though stranger things have happened. :-)

Well, that of course would actually be a nice solution, since it would
allow everyone to use the aRts capabilities fully in desktop applications.

It isn't possible right now to say "KDE is switching to aRts for sure",
because some issues that raised in the discussion on kde-core-devel are
not adressed right now. But I think it can be done, and I'll work on it.
Probably we can get some discussion (definite descision?) about that on
the upcoming KDE II developer meeting, that would be until 10. October.

> Can you tell us about the features of aRts or point us to a web page or
> something?

So, here again a listing of features that aRts has. You'll find some good
information on the aRts web page, in the manual, for instance at

  http://arts.linuxbox.com/doc/manual/index.html

(read section 11, which talks specifically about aRts vs. esd). The 
homepage URL is

  http://linux.twc.de/arts

Of course, the web pages are mainly focused on explaining how aRts is used
as synthesizer, since thats what most people will do with it right now. It
doesn't cover too technical details, too. 

So I tried to make a technical feature list here, hoping some insight in
why I think aRts is the right thing for an audio server (or if you like,
audio framework, audio middleware, multimedia subsystem or something like
that).


== Overview: aRts and its features

+ the core idea: signal flow

  aRts is a signal flow system. That means, that it is based on the assumption
  that multimedia tasks generally can be described as flow graphs with
  components. Arts manages the signal flow between the components.

  Currently the signal flow is assumed to happen always realtime. Further
  enhancements of aRts could also handle non-realtimed signal processing.

+ implementation as CORBA server

  On the other hand, aRts is designed to be a server. That is, it will run,
  and different applications will tell aRts what things to start for signal
  flow evaluation.

  That means that multiple applications share the same signal flow system.
  You could for instance play an mp3 (with a standard, arts compliant mp3
  player) and record the result with your hard disk recording system which
  also runs on the same aRts server.

  If you like, you could also add some midi synthesis.

+ modularity

  The components that are running inside the signal flow evaluation are
  implemented as small modules that are linked to the server (a shared-
  library interface would be no problem for that).

  So if you need a new wave form, a new filter, a new effect, a new sample
  format etc. you simply write a new module.

  Through the signal flow system, it is also possible to draw a signal
  flow (aRts calls that a structure), and reuse that as module in other
  signal flows.

  So you could for instance implement a reverb effect out of some delays
  and some arithmetic modules (and perhaps some filters).

+ fast scheduling

  aRts uses very small components, and schedules them with very little
  overhead. So you don't have some processes which do signal processing
  and connect them with TCP streams (which would really cosume lots of 
  CPU power).

  Instead, all modules are running in one adress space. They are connected
  by ring buffers, which allows you to run 300 modules at a time on a
  PII-350.

+ network transparency

  Since aRts uses CORBA for almost everything, it is network transparent.
  On the other hand, some audio server functionality that has been
  implemented in aRts use TCP to transfer the signal for instance from your
  external (non-arts-module-like) mp3 player to aRts. This TCP stuff is
  also network transparent.

+ gui suppprt

  aRts can build modular GUIs with the same component system (signal flows,
  structures, etc) that is used for building audio stuff. So the GUIs have
  the same features regarding modularity, flexibility, recombination,...
  as the core synthesis.

+ synthesis implemented GUI independant

  All the synthesis modules aRts uses (such as Synth_STD_EQUALIZER) don't
  know anything from the GUI at all. They use no Qt and no whatever. They
  are only connected to the GUI through the same signal flow mechanisms
  that are used for everthing else

+ distribution

  Arts is somewhat ready for distributing flow graphs across an arbitary
  number of servers. Currently, the standard aRts systems run with one
  server that handles audio signal flow (all Synth_* components running
  there), and one server that handles the gui stuff (all Gui_* components
  run there).

  That may be an option for further work on that, either allowing aRts to
  run for instance multiple servers together for signal flow, or add more
  servers for video flow, or add certain selected applications (such as
  your midi sequencer) as own server with own components.

+ availability

  Arts is under development since nearly two years now. It works all the
  time. The flow system is well tested, and the realtime performance is fine.
  It's not some "here I have specs that you might want to read"-project, but
  something that is implemented and known to work.

== Summary

Well, I'll try to end with a brief feature listing, while marking

+ is implemented, can be done with the current (or next) aRts release
? should be possible, but isn't in the code yet
- might be a problem

== features
+ signal flow system
  + realtime
  ? non-realtime
+ consequent server design
+ modularity
+ fast scheduling
+ distribution
+ gui support
+ synthesis is implemented gui & toolkit independant
+ network transparency


== usage scenarios
+ abstraction of /dev/dsp for all applications
  + mix outgoing streams
  + postprocess outgoing streams with effects
  ? get incoming streams (audio recording)
  + full duplex operation (record, process, write)
+ play wavs (for instance for window manager sound)
? hard disk recording
+ realtime midi (sequencer) -> audio conversion
  + timidity style sample based conversion
  + midi synthesis

== possibilities after extending to other media streams
? midi processing
? video processing
    
== problems
- incompatibilities with other solutions such as esd, GMF, NAS,
  KAudioServer1/2, and perhaps some ALSA concepts
- not really designed to be an audio server
- larger as esd for instance
  - code
  - memory
- more CPU load as esd for instance
- requires root (suid or similar) for realtime operation
- uses mico C++ bindings with BOA
- only flow graph editor/gui server for Qt available

Well, thats a brief summary - the best thing to do to see more is to
download a version and play a little with it. And of course - please ask
anything that isn't clear after reading this mail.

  Cu... Stefan
-- 
  -* Stefan Westerfeld, stefan@space.twc.de (PGP!), Hamburg/Germany
     KDE Developer, project infos at http://space.twc.de/~stefan/kde *-