Re: Problems with the Stream interface

[Maciej Stachowiak <mjs eazel com>]

Miguel de Icaza <miguel@helixcode.com> writes:

> > This is a problem for applications in general. If we want to be able
> > to handle large data sets, and have everything be network-transparent,
> > we need this kind of loading to be async. I've been told it's not a
> > problem for evolution because the relevant data sets are small, but I
> > imagine in the future someone might send a 300-page Word document
> > attachment, and the poor recipient will have to see the whole UI block
> > while it gets streamed into the view component.
> 
> We are now using threads in evolution to avoid similar problems. 
> 

Miguel, Nautilus uses threads inside gnome-vfs, but gnome-vfs hides
that threading behind a callback-based async interface.

> > If we don't want to force components to be bother with the complexity
> > of handling asynchronous I/O (an understandable goal), it should
> > actually be possible to emulate the current synchronous stream
> > interface in terms of the async calls (so the only IDL would be for
> > the async stream interface, but the bonobo client-side wrappers would
> > provide emulated synchronous calls). 
> 
> The problem with this approach is that it means that anyone
> implementing the Stream interface outside of the Bonobo C
> implementation will have to bother with the details.  For instance,
> Perl components or Python components which have nothing but the most
> minimal runtime available.

I proposed a solution in my rsponse to Michael that should be happy
for all languages.

> But what is exactly the problem you are facing here Maciej?  

I posted a link to a specific Nautilus bug report that describes the
user-visible symptom problem in my previous message.

In brief: when you click on a text or image file viewing a remote
directory via FTP or WebDAV (or other remote protocols), the whole
Nautilus UI blocks while the component is loading. We don't have this
problem for directory views or various other Nautilus components
because the Nautilus::View interface, unlike the Stream and
PersistStream interfaces, is fully asynchronous.

> I do like to some extent the idea of having a callback-based system to
> deliver the async features, but the Stream interface as it stands is
> just a simple file-IO like interface.  I would very much like to avoid
> replacing it to be asyncronous.

Unfortunately it is not possible to do that without forcing all
applications to use threads instead of a callback-based async
architecture. Since the latter is what we usually promote to GNOME
developers, it is important that we support that use model. I believe
(after long, hard thought) that this is only possible my making the
Stream interface asynchronous.

I will post IDL for my proposal shortly.

 - Maciej