Thinking about GtkFileSystem

[Owen Taylor <otaylor redhat com>]

I've started working on fleshing out the details of the 
GtkFileChooser API, and one are that is definitely
a bit less clear to me is the file-system backend
that provides the glue between the user interface
and different file system implementations.

This is going to be a semi-public API .. that is, the
headers will be installed, but you'll have to #define
something special to use them, and there won't be 
strong guarantees of future compability.

I looked some at EggFileSystem, and a little bit at 
GnomeVFS, but mostly this is just writing down stuff.

I'd really appreciate feedback from people with experience
with GnomeVFS as to whether this looks reasonable
to write a GnomeVFS backend for, and whether it will work
in the slow-filesystem case.

Thanks,
                                 Owen

Overview
========

The idea of the file system object is to present an abstract view
of the underlying filesystem to the GUI control. This allows
switching between, say, a native-Unix-filesystem and the 
gnome-vfs virtual file system via dynamically loaded modules.

Features that the file system object needs to have include:

 - Listing the roots of a multi-rooted filesystem. 
 - Listing/iterating through the children of a folder
 - Getting information about a file or directory, including
   size, modification time, mime type, and icon.
 - Watching for changes in the list of roots
 - Watching a directory for changes in the set of children
   or properties of a child
 - Creating new directories


How to represent a file/folder location
=======================================

A couple of possiblities exist:

 - The URI as a string

 - An object. (Like EggFileSystemItem)

My preference here is to avoid the opaque objects and just use
the URI strings. This could clearly lead to some inefficiency,
but not having to convert back-and-forth is going to simplify
the API a bunch.

It appears to me that the URI strings are going to need to be 
uninterpreted ... something like a URI encoding a windows 
filename can be non-obvious to traverse with string operations.

This means that the file system object will need a method
something like:

  get_parent (uri);

as well as a list_children (uri);


Filename encoding
=================

A subject that's simply incredibly difficult, as we've discovered
multiple times in the past.

Here's one possibility:

 - The URI strings are uninterpreted, except that they 
   must be valid UTF-8, to allow display and printing. Practically 
   speaking, they probably  should always be in the 
   ASCII subset of RFC 2396, but I don't think we'll have
   any reason to enforce that in the code.

 - Every component (defined by calling get_parent() on
   the URI repeatedly) has a display name that is UTF-8.
   This display name is _not_ guaranteed to be unique
   among children of the parent.

 - There is a function for creating an URI from a base
   directory and UTF-8 child part.

The difficult thing is interpreting things the user types
into the file selector entry 'C:foo' 'http:www.gnome.org'
'http://www.gnome.org/My File', and so forth. Not sure
what to do here except write some magic heuristics.


Error handling
==============

Almost every operation needs to allow for errors because
the underlying file system can change at any time.
In general, we want to have friendly user strings for
at least some of the errors, so using GError seems
appropriate.


The notification API
====================

Notification on individual files doesn't seem particular useful ...
if a folder is being displayed in the file selector, the system
will typically want notification of changes to the directory
or to any file in the directory. So, notification can be restricted
to a folder-level granularity.

At that point, there are basically two options for the API:

 - Global signals on the file system object for all directories,
   with methods to (ref-counted) monitor and unmonitor particular
   folders.

 - Explicit monitor objects with signals just for one folder..

I suspect the latter choice is going to be more convenient.


Semantics of notification
=========================

At first glance, it might seem desirable to have strict 
consistency semantics:

 If a folder is being monitored, then if no change notifications
 are received:

  A) All calls to list_children(folder_uri) will succeed
  B) The results of two successsive calls to list_children(folder_uri) 
     will be identical
  C) If a child is listed by list_children(folder_uri), then 
     a call to get_info(child_uri) will succeed.
  D) The results of any two calls to get_info(child_uri) will
     be identical.

 (Would need to be elaborated to describe what happens when
 notifications _are_ received.)

This is the level of consistency needed by GtkListStore. But
such a high level of consistency isn't going to be found in
any actual file system API... files can disappear at any
point. So, implementing it would require the file system 
object to actually keep a mirror of all the information about
a monitored folder locally and only update it when sending
change notifications.

I think it's probably better to do this detailed mirroring in
GUI code... one implementation possibility is that we might
want to wrap a strongly consistent file system object around
the real file system object to reduce the amount of error
checking that has to be scattered through the code.


Can incremental filling be piggybacked on top of notification?
============================================================

For remote network filesystems, incremental filling of 
directories is interesting. Hopefully we can simply use the
change-notification mechanism to accomplish this. 
Considerations:

 - It means that we can only do incremental filling for
   directories that are currently being monitored.

 - Relevant to the above discussion of notification 
   semantics, in order to do incremental filling via
   notification, you have to keep a complete local copy
   of all 'interesting' information, just as you
   do to provide strong guarantees about consistency.

   So, this would be an argument for doing the 
   consistency creation in the file system rather than
   in a wrapper ... you dont' want to keep two entire
   copies of the information around.


How to do the 'stat' operation?
===============================

There are various types of information that we want to get
about files:

 Display name
 Modification time (atime/ctime as well? I think not)
 Icon
 Mime Type

Getting them one by one if you need multiple items is expensive;
there are basically two options, expensive in different ways

 - Get everything, and cache them.
 - Retrieve each item separately.
 
It probably makes sense to have a single call similar to stat()
that can get any combination of items.



Draft API
=========


typedef gint64 GtkFileTime;

/* Mask of information about a file, for monitoring and
 * gtk_file_system_get_info()
 */
typedef enum {
  GTK_FILE_INFO_DISPLAY_NAME      = 1 << 0,
  GTK_FILE_INFO_IS_DIRECTORY      = 1 << 1,
  GTK_FILE_INFO_MIME_TYPE         = 1 << 2,
  GTK_FILE_INFO_MODIFICATION_TIME = 1 << 3,
  GTK_FILE_INFO_SIZE              = 1 << 4,
  GTK_FILE_INFO_ICON              = 1 << 5
} GtkFileInfoType;

/* GError enumeration for GtkFileSystem
 */

#define GTK_FILE_SYSTEM_ERROR g_file_system_error_quark ()

typedef enum
{
  GTK_FILE_SYSTEM_ERROR_NONEXISTANT,
  GTK_FILE_SYSTEM_ERROR_NOT_FOLDER,
  GTK_FILE_SYSTEM_ERROR_FAILED,
} GFileError;

GQuark     gtk_file_system_error_quark      (void);

/* Boxed-type for gtk_file_system_get_info() results
 */
typedef struct _GtkFileInfo        GtkFileInfo;

GtkFileInfo *gtk_file_info_new  (void);
GtkFileInfo *gtk_file_info_copy (GtkFileInfo *info);
void         gtk_file_info_free (GtkFileInfo *info);

G_CONST_RETURN gchar *gtk_file_info_get_display_name      (GtkFileInfo *info);
void                  gtk_file_info_set_display_name      (GtkFileInfo *info,
							   const gchar *display_name);
gboolean              gtk_file_info_get_is_directory      (GtkFileInfo *info);
void                  gtk_file_info_get_is_directory      (GtkFileInfo *info,
							   gboolean     is_directory);
G_CONST_RETURN gchar *gtk_file_info_get_mime_type         (GtkFileInfo *info);
void                  gtk_file_info_set_mime_type         (GtkFileInfo *info,
							   const gchar *mime_type);
GtkFileTime           gtk_file_info_get_modification_time (GtkFileInfo *info);
void                  gtk_file_info_set_modification_time (GtkFileInfo *info,
							   GtkFileTime  modification_time);
void                  gtk_file_info_set_size              (GtkFileInfo *info,
							   gint64       size);
gint64                gtk_file_info_get_size              (GtkFileInfo *info);
void                  gtk_file_info_set_icon              (GtkFileInfo *info,
							   GdkPixbuf   *icon);
GdkPixbuf *           gtk_file_info_get_icon              (GtkFileInfo *info);

/* The base GtkFileSystem interface
 */
#define GTK_TYPE_FILE_SYSTEM             (gtk_file_system_get_type ())
#define GTK_FILE_SYSTEM(obj)             (G_TYPE_CHECK_INSTANCE_CAST ((obj), GTK_TYPE_FILE_SYSTEM, GtkFileSystem))
#define GTK_FILE_SYSTEM_CLASS(vtable)    (G_TYPE_CHECK_CLASS_CAST ((vtable), GTK_TYPE_FILE_SYSTEM, GtkFileSystemIface))
#define GTK_IS_FILE_SYSTEM(obj)          (G_TYPE_CHECK_INSTANCE_TYPE ((obj), GTK_TYPE_FILE_SYSTEM))
#define GTK_IS_FILE_SYSTEM_CLASS(vtable) (G_TYPE_CHECK_CLASS_TYPE ((vtable), GTK_TYPE_FILE_SYSTEM))
#define GTK_FILE_SYSTEM_GET_CLASS(inst)  (G_TYPE_INSTANCE_GET_INTERFACE ((inst), GTK_TYPE_FILE_SYSTEM, GtkFileSystemIface))
typedef struct _GtkFileSystem      GtkFileSystem;
typedef struct _GtkFileSystemIface GtkFileSystemIface;

struct GtkFileSystemIface
{
  GTypeInterface base_iface;

  /* Methods
   */
  GSList *           (*list_roots)     (GtkFileSystem *file_system);
  gboolean           (*list_children)  (GtkFileSystem  *file_system,
				        const gchar    *uri,
				        GSList        **children,
				        GError        **error);
  gboolean           (*get_parent)     (GtkFileSystem  *file_system,
					const gchar    *uri,
					const gchar   **parent,
				        GError        **error);
  GtkFileInfo *      (*get_info)       (GtkFileSystem  *file_system,
					const gchar    *uri,
				        GtkFileInfoType types, 
				        GError        **error);
  GtkFolderMonitor * (*create_monitor) (GtkFileSystem  *file_system,
					GtkFileInfoType types,
					const gchar    *uri,
				        GError        **error);
  gboolean           (*create_folder)  (GtkFileSystem  *file_system,
					const gchar    *uri,
			                GError         *error);
  gchar *            (*make_uri)       (GtkFileSystem  *file_system,
					const gchar    *base_uri,
					const gchar     display_name);

  /* Signals
   */
  void (*roots_changed) (GtkFileSystem *file_system);
};

GSList *          gtk_file_system_list_roots     (GtkFileSystem    *file_system);
gboolean          gtk_file_system_list_children  (GtkFileSystem    *file_system,
						  const gchar      *uri,
						  GSList          **children,
						  GError          **error);
/* FALSE return indicates error, TRUE return and NULL stored in @parent
 * indicates that there is no parent.
 */
gboolean          gtk_file_system_get_parent     (GtkFileSystem    *file_system,
						  const gchar      *uri,
						  const gchar     **parent,
						  GError          **error);
GtkFileInfo *     gtk_file_system_get_info       (GtkFileSystem    *file_system,
						  const gchar      *uri,
						  GtkFileInfoType   types,
						  GError          **error);
GtkFolderMonitor *gtk_file_system_create_monitor (GtkFileSystem    *file_system,
						  const gchar      *uri,
						  GError          **error);
gboolean          gtk_file_system_create_folder  (GtkFileSystem    *file_system,
						  const gchar      *uri,
						  GError           *error);
gchar *           gtk_file_system_make_uri       (GtkFileSystem    *file_system,
						  const gchar      *base_uri,
						  const gchar       display_name);

/*
 * Foldering monitoring object interface
 */
#define GTK_TYPE_FOLDER_MONITOR             (gtk_folder_monitor_get_type ())
#define GTK_FOLDER_MONITOR(obj)             (G_TYPE_CHECK_INSTANCE_CAST ((obj), GTK_TYPE_FOLDER_MONITOR, GtkFolderMonitor))
#define GTK_FOLDER_MONITOR_CLASS(vtable)    (G_TYPE_CHECK_CLASS_CAST ((vtable), GTK_TYPE_FOLDER_MONITOR, GtkFolderMonitorIface))
#define GTK_IS_FOLDER_MONITOR(obj)          (G_TYPE_CHECK_INSTANCE_TYPE ((obj), GTK_TYPE_FOLDER_MONITOR))
#define GTK_IS_FOLDER_MONITOR_CLASS(vtable) (G_TYPE_CHECK_CLASS_TYPE ((vtable), GTK_TYPE_FOLDER_MONITOR))
#define GTK_FOLDER_MONITOR_GET_CLASS(inst)  (G_TYPE_INSTANCE_GET_INTERFACE ((inst), GTK_TYPE_FOLDER_MONITOR, GtkFolderMonitorIface))
typedef struct _GtkFolderMonitor      GtkFolderMonitor;
typedef struct _GtkFolderMonitorIface GtkFolderMonitorIface;

struct GtkFolderMonitorIface
{
  GTypeInterface base_iface;

  /* Signals
   */
  void (*deleted)      (GtkFolderMonitor *monitor);
  void (*file_added)   (GtkFolderMonitor *monitor,
		        const gchar      *uri);
  void (*file_changed) (GtkFolderMonitor *monitor,
			const gchar      *uri);
  void (*file_removed) (GtkFolderMonitor *monitor,
			const gchar      *uri);
};


Miscellaneous questions about API
=================================

- Supporting operations on a non-monitored directory is
  going to add complexity to file system implementations.
  Perhaps we should rename GtkFolderMonitor to GtkFileFolder
  and move list_children() and get_info() to there. 

- Is additional information needed in GtkFileInfo - 
  is-symlink? atime? ctime? permissions?