Re: Thinking about GtkFileSystem

[Owen Taylor <otaylor redhat com>]

On Fri, 2003-03-14 at 08:44, Alexander Larsson wrote:

> One thing that is totally left out of this is Authentication. For any 
> non-trivial filesystem implementation that does remote shares you will 
> need to at least enter username/password, and often more (domain, key 
> phrases etc). Furthermore, it would be very painful for users if each app 
> had to authenticate, as the user would have to type username + password in 
> each app reading files from a protected share. Any sane system must have a 
> "daemon" process that remembers the entered passwords and pops up 
> authentication dialogs when needed.
> 
> Now, how will this be affect the API? Well. It means that any blocking I/O 
> call could potentially block for a very long time, waiting for user input. 
> If the file dialog just used blocking calls this could stop the whole app 
> from even repainting while the auth dialog is up. I think this means we 
> have to go with an async I/O model. This ties up with some of the things 
> discussed below.

The other thing that you'd like is the ability set the right transient
parent ... same thing as I mentioned for error dialogs.

> > It appears to me that the URI strings are going to need to be 
> > uninterpreted ... something like a URI encoding a windows 
> > filename can be non-obvious to traverse with string operations.

[...]

>  
> I agree that the uri approach is probably better. Although i know DV will 
> scream at us for calling these things uris.

Well, the code in GTK+ itself will probably only deal with actual
file: URL's... so I'll leave invented URI scheme, etc, to others :-)

> > Semantics of notification
> > =========================
> > 
> > At first glance, it might seem desirable to have strict 
> > consistency semantics:
> > 
> >  If a folder is being monitored, then if no change notifications
> >  are received:
> > 
> >   A) All calls to list_children(folder_uri) will succeed
> >   B) The results of two successsive calls to list_children(folder_uri) 
> >      will be identical
> >   C) If a child is listed by list_children(folder_uri), then 
> >      a call to get_info(child_uri) will succeed.
> >   D) The results of any two calls to get_info(child_uri) will
> >      be identical.
> > 
> >  (Would need to be elaborated to describe what happens when
> >  notifications _are_ received.)
> > 
> > This is the level of consistency needed by GtkListStore. But
> > such a high level of consistency isn't going to be found in
> > any actual file system API... files can disappear at any
> > point. So, implementing it would require the file system 
> > object to actually keep a mirror of all the information about
> > a monitored folder locally and only update it when sending
> > change notifications.
> 
> This looks really really hard to get, and even if you have it i'm not sure 
> its realistically useful. Take D for instance. Even if i do get a 
> notification between to calls to get_info there is no guaranteed that i 
> have entered the mainloop inbetween and gotten the change fam event. Or do 
> you mean it should cache the old values until the event has been 
> dispatched?

That's what I meant by "implementing it would require the file system 
object to actually keep a mirror of all the information about
a monitored folder locally and only update it when sending
change notifications" if you want stuff like:
 
 children = list_children()
 foreach child in (children):
   info = get_info

not to deal with the children you need a local cache of the data
and to only change that at controlled points. Since GtkTreeModel (and in
general, any sane code for displaying the GUI) does require this strong
sense of consistency, we are going  to have to cache the information
somewhere.

>  
> > Can incremental filling be piggybacked on top of notification?
> > ============================================================
> > 
> > For remote network filesystems, incremental filling of 
> > directories is interesting. Hopefully we can simply use the
> > change-notification mechanism to accomplish this. 
> > Considerations:
> > 
> >  - It means that we can only do incremental filling for
> >    directories that are currently being monitored.
> > 
> >  - Relevant to the above discussion of notification 
> >    semantics, in order to do incremental filling via
> >    notification, you have to keep a complete local copy
> >    of all 'interesting' information, just as you
> >    do to provide strong guarantees about consistency.
> > 
> >    So, this would be an argument for doing the 
> >    consistency creation in the file system rather than
> >    in a wrapper ... you dont' want to keep two entire
> >    copies of the information around.
> 
> Now, this is interesting. The way Fam works is that when you start 
> monitoring a directory you get a load of FAMExists events for each of the 
> files in that directory (or a FAMDeleted if the directory doesn't exist).
> I think the main reason for this is to avoid the various races you get 
> between stating/listing the directory and starting to monitor it. 
> 
> I think we could use a similar idea to get both async list_dir and to 
> piggyback on the file notification API.

Hmm, same suggestion as Michael. It makes GtkFileSystem pretty
unpleasant for _direct_ use, but we've already accepted that
we are going to have to wrap it to get sane consistency 
guarantees for the upper levels.

If we went this way, I don't think there is any reason to
distinguish between 'exists' and 'added' for the purposes
of a file selection.

[...]

> Due to the auth issues discussed above maybe we should have an async stat.
> I'm not sure how this would look. Perhaps the file_changed, file_added 
> (and file_exists if we add that) signals on the monitor object could pass 
> in a GtkFileInfo with the new data. 

I don't particularly see why authentication would require an 
async _stat_. Isn't authentication typically on a per-folder
basis?

> > typedef struct _GtkFileInfo        GtkFileInfo;
> > 
> > GtkFileInfo *gtk_file_info_new  (void);
> > GtkFileInfo *gtk_file_info_copy (GtkFileInfo *info);
> > void         gtk_file_info_free (GtkFileInfo *info);
> 
> 
> Maybe a way to get a GtkFileInfoType mask for the FileInfo?

I thought about that, but it didn't seem immediately useful.
The general thing would be to copy what I did for PangoFontDescrition:

PangoFontMask pango_font_description_get_set_fields (const PangoFontDescription *desc);
void          pango_font_description_unset_fields   (PangoFontDescription       *desc,
                                                     PangoFontMask               to_unset);
                                                                               

> > struct GtkFolderMonitorIface
> > {
> >   GTypeInterface base_iface;
> > 
> >   /* Signals
> >    */
> >   void (*deleted)      (GtkFolderMonitor *monitor);
> >   void (*file_added)   (GtkFolderMonitor *monitor,
> > 		        const gchar      *uri);
> >   void (*file_changed) (GtkFolderMonitor *monitor,
> > 			const gchar      *uri);
> >   void (*file_removed) (GtkFolderMonitor *monitor,
> > 			const gchar      *uri);
> > };
> 
> What about a "created" signal? If you start monitoring something that 
> doesn't exist? Fam goes into polled mode for active monitors on files that 
> doesn't exist (since dnotify need the file to exist to work).

What would be the use case for monitoring something that doesn't
exist? I would think:
 
 A) We wouldn't allow the file selector to be pointed at
    a non-existent location.
 B) If the directory the file selector was looking at vanished
    we'd change it to the "closest" existent directory.

> 
> > Miscellaneous questions about API
> > =================================
> > 
> > - Supporting operations on a non-monitored directory is
> >   going to add complexity to file system implementations.
> >   Perhaps we should rename GtkFolderMonitor to GtkFileFolder
> >   and move list_children() and get_info() to there. 
> 
> You might want to share folder data between several folder monitors 
> though. If we add a "file_exist" signal to the monitor that means the 
> monitor has "state" for the user, and each user must create a private 
> monitor object.

Hmm. That can, I think, be left as an implementation detail. For 
GtkFileSelector, we probably can just get away with taking the
overhead if there are multiple instances monitoring the same
directory.

But that does make clear to me why I'm a bit uncomfortable with the
everything-async model ... it means that the monitor object
becomes stateful and no longer has a meaning independent of
past history. 

Regards,
                                            Owen