Re: [gfvs] cdda backend

[David Zeuthen <david fubar dk>]

Hi again,

Almost more questions than answers this time, sorry. Also no patch
either since I wanted to hear your thoughts before doing the work.

On Wed, 2007-12-19 at 11:43 +0100, Alexander Larsson wrote:
> I commited this with some small changes:
> Renamed fuse_name to stable_name
> Remove unused GVolumeMonitor *union_monitor added to GHalVolumeMonitor.

Great, thanks!

> > I'm thinking this can also be used for the "favorite" servers like we
> > have today in gnome-vfs; e.g. we just implement a volume monitor that
> > creates GVolume objects that matches a list in gconf. Then when the
> > mount is actually created the volume monitor adopts the DaemonMount in
> > it's adopt_orphan_mount() function.

Btw, is this how we should implement it? Or did you have anything else
in mind? I really miss my "public_html @ p.fd.o" link that comes up
automatically - I use it a lot when doing screen shots to share. 

I think if we do the feature this way (a gvfs module that links in
gconf) it can be written in a day or less.

> So, can there never be a situation where the GMount gets created before
> the corresponding GVolume object is? And in that case, how does the
> GVolume locate the mount?

That's a good question; it's not supported in the API yet because that
situation can never happen for the use I wrote it for (cdda:// backend).

How about a another vtable entry in the GVolumeMonitor

 gboolean (*request_adoption_of_orphan_mount) (GVolume *adopter,
                                               GMount *orphan);

that each implementation can implement? Alternatively we can put this
method on the GMount interface but that seems wrong since it's only
something volume monitors would ever want to do.

> > Other thoughts / questions about the API
> > 
> >  - Busy mounts. Right now the cdda:// backend refuses to unmount
> >    if there are open files on it. I think that's probably the right
> >    thing to do for any backend. 

Btw, what are you thoughts about this? I feel strongly this is the right
thing to do .. otherwise you can lose data.

I'd even go as far and say that the GVfsBackend base class should
automatically return G_IO_ERROR_BUSY on unmount() when there are pending
jobs. At least that would be the default unmount() operation unless you
override it.

> Probably means we need to add flags
> >    to the unmount() calls; the flags used in Linux, e.g.
> >     - force unmount
> >     - lazy unmount
> >    comes to mind.
> 
> While I can sort of see the usecases for these, aren't we sort of
> overcomplicating the API here? Is this really required? When and how
> would the user force unmount or lazy unmount something? (Or the reverse,
> if there is a case for one of these when should we not do it.)
>
> >  - Further, if a mount can't be unmounted because it's busy (and
> >    we can't avoid that since we support mounts backed by kernel
> >    drivers), we probably want something like lsof(8) that Nautilus
> >    and other stuff can use to put up a dialog showing what apps
> >    is blocking the unmount. How about a list_open_files() method
> >    on GMount() that returns an array of
> >     - process id (is that portable? maybe need an abstraction)
> >     - icon
> >     - name
> >     - etc.
> >    Would need backend support for this too.
> 
> Now we're getting into really lowlevel bizarro things. I don't think we
> should really expose this in a generic API. Can't this just be reported
> in the error message for the G_IO_ERROR_BUSY error when unmounting.
> 
> I.E. the error dialog would say "Cannot unmount, because firefox (pid
> 35) is keeping files open on the volume", or "Cannot unmount, because 13
> applications are keeping open files on the volume.". Maybe in the last
> case we should at least give the name (and pid?) of one application so
> that you can make progress on unmount by killing that.

Right, I definitely think we want the dialog to look something like this

 +-------------------------------------------------------+
 | Cannot unmount "public_html @ p.fd.o", the following  |
 | applications are still using it                       |
 |                                                       |
 | [icon] GEdit - Secret Stuff.txt                [kill] |
 | [icon] GEdit - My Diary.txt                    [kill] |
 | [icon] Tomboy                                  [kill] |
 | [icon] /bin/cat                                [kill] |
 |                                                       |
 |                                              [Cancel] |
 +-------------------------------------------------------+
 (dunno what icon / wording to use for [kill] though. Maybe
  there could also be a [switch to] button that dismisses
  the dialog (or not?) and takes you to the application)

Stuffing all these details into the GError doesn't seem at all
reasonable. So I think a simple g_mount_list_open_files() function is
what we need (need an async function too).

Anyway, In fact, I think we want such a dialog in the next gtk+
release: 

GtkWidget *gtk_io_unmount_busy_dialog_new (GtkWindow*parent,
                                           const char *markup, 
                                           GMount *mount);

or something. Then Nautilus or the drive applet would use that one.

Now, this dialog by itself _could_ offer the option of "Forcibly
unmount", "Unmount anyway" or whatever but I think that's bad UI. Maybe
some other desktop than GNOME using gvfs wants that though.

However, the main reason you want the moral equivalent of "umount -f" is
for the networking applet (e.g. nm-applet). Basically if a network
connection (including VPN) goes down you want to forcible unmount any
network shares that used that connection (and tell the user, maybe via a
notification, what applications are affected). That implies the
networking applet needs to know not only whether a GMount is network
connected but also the name of the endpoint. (See below for API
proposal.)

Now, you could propose that the backends themselves should deal with
networking connections going away (by listening to e.g. NetworkManager
events or whatever) but I don't think that's reasonable at all. You want
them to do one thing and do that thing good.

(Btw, another feature you want in the network applet is the ability to
warn the user that if he brings down network connections / VPN's where
we have mounts with open files. So nm-applet would also use the
proposed gtk_io_unmount_busy_dialog_new() function although passing in
slightly different markup.)

Thinking about it I'm not sure lazy unmount is needed. Historically,
IIRC, lazy unmount was added because force unmount on Linux didn't work
well. The reason force unmount didn't work was because we didn't (and
still don't) have revoke(2) on Linux. So on Linux without revoke(2), the
hal backend will lazy unmount when asked to force unmount. When
revoke(2) is made available, we'll use that instead.

So I think the conclusion is that we do want to pass GMountUnmountFlags
to unmount() on GMount. With a single G_MOUNT_UNMOUNT_FORCE defined in
that enumeration. Thoughts?

> I do believe we want some form of mount options. For instance, a very
> common request is to be able to specify the filename encoding of e.g. a
> ftp share. 

(These are exactly the same bugs I'm dealing with in gnome-mount)

> 
> However, I don't think the right place for this is as API arguments for
> the mount call. Instead i think these are more like preferences for each
> mount point, that we can store somewhere and that gvfs automatically
> picks up each time you mount that particular share.
> 
> I haven't sat down and worked out all the details about this though.

Sounds good to me; no need to complicate the API. However, I think we
want a remount() method on GMount() that either errors out if remounting
is not possible, or remounts the mount with the new options. Otherwise
you'll end up with

http://people.freedesktop.org/~david/gm-prop/gm-prop2.png

or similar to tell the user to remount.  That would be bad. Thoughts?

> Now, we could add something to GFileInputStream and GFileOutputStream to
> allow you to set options. This is clearly not something that is part of
> the requirements for loading and saving documents, and I'm sure it will
> be abused in weird ways. However some uses of it might be important
> enough that its worth having this wart on the API. I dunno. It seems a
> dangerously small step from ioctl()...

How about something simple like 

 gboolean g_input_stream_set_option (GInputStream *stream, 
                                     const char   *key, 
                                     const char   *value,
                                     GError       *error);

 char *   g_input_stream_get_option (GInputStream *stream, 
                                     const char   *key,
                                     GError       *error);

with async versions and ditto for output stream too? (Would it make
sense to have a GStream base class for this and other things?) 

(Oh, you could go more crazy and add API for querying what options a
stream can do or add full type support so it's more than string values.
Shrug.)

---

Another thought: we probably need a way to learn more about the GDrive,
GVolume and GMount objects than just their names and icons. First, cf.
the network applet use, we need to know if the mount is networked or not
and if it is, what hostname is it using. Simple proposal

 gboolean g_volume_is_local         (GVolume *volume);
 char *   g_volume_get_foreign_host (GVolume *volume);

 gboolean g_mount_is_local         (GMount *mount);
 char *   g_mount_get_foreign_host (GMount *mount);

 (not sure we need the same on GDrive; I think these are always local)

---

It would be good to know more precisely the "purpose" of a GDrive,
GVolume or GMount in question. Right now all we expose is the name and
icon and I think that not sufficient for applications like F-Spot,
Rhythmbox, Banshee etc. We ought to export API for this.

There are (at least) three sources from which we can get this
information

 a. if a GVolume stems from the gvfsd-gphoto2 backend chances are
    there's photos (or music) on the mount; similar if it stems
    from the cdda:// backend there will be music on it.

 b. Things like GPS readers, music players, cameras: we have this kind
    of information from HAL whitelists. The obex-ftp backend will know
    it's a phone. Things like that.

 c. One can sniff for things like DCIM/ on the file system itself to
    infer there's photos on it. Probably other things too (e.g. the
    iPod has a predictable file system layout).

So the question is how to export good API with this information and
where to put it. I think it would be good to separate "we know the
device is XYZ" from "looks like there is this XYZ kind of files on the
file system".

A simple proposal is this

 typedef enum {
  G_DRIVE_CLASSIFICATION_OPTICAL,         // optical drives
  G_DRIVE_CLASSIFICATION_CARD_READER,     // examples: any Nin1 card reader
  G_DRIVE_CLASSIFICATION_STILL_CAMERA,    // examples: any modern camera
  G_DRIVE_CLASSIFICATION_VIDEO_CAMERA,    // examples: modern digital video cameras
  G_DRIVE_CLASSIFICATION_LOCATION_DEVICE, // examples: Sony’s GPS-CS1 [1]
  G_DRIVE_CLASSIFICATION_MUSIC_PLAYER,    // examples: PSP, iPod
  G_DRIVE_CLASSIFICATION_VIDEO_PLAYER,    // examples: PSP, iPod
  G_DRIVE_CLASSIFICATION_BOOK_READER,     // examples: Amazon Kindle
  G_DRIVE_CLASSIFICATION_PHONE,           // examples: any phone with obex-ftp

  // Can't immediately think of more content types here

 } GDriveClassificationFlags;
 (for readability all the 1<<n stuff is omitted)

 [1] : http://www.dpreview.com/news/0608/06080202sonygpscs1.asp

which allows one to specify that a drive has more than one
classification; API to access it is

 GDriveClassificationFlags g_drive_get_classification (GDrive *drive);

where we don't need the async version as this is computed at init /
drive connection time and cached. 

Now for the content classification; this goes on GMount as we may need
to sniff the file system.

 typedef enum {
   G_MOUNT_CONTENT_PHOTOS,            // contains DCF (e.g. DCIM/ folder)
   G_MOUNT_CONTENT_MUSIC,             // contains music
   G_MOUNT_CONTENT_VIDEO,             // contains video files
   G_MOUNT_CONTENT_BOOKS,             // contains electronic books
   G_MOUNT_CONTENT_LOCATION_DATA,     // contains e.g. GPS traces
   G_MOUNT_CONTENT_OPERATING_SYSTEM,  // contains an operating system
   G_MOUNT_CONTENT_BACKUP,            // contains backup data (think timemachine-ish apps)

   // probably want more here; e.g. phones can store contacts, notes,
   // and other stuff. Bastien?

 } GMountContentFlags;
 (again, for readability all the 1<<n stuff is omitted)

for this we need both sync and async API since this is computed on
demand; the sync API would be something like

 GMountContentFlags g_mount_guess_content (GMount *mount);
 (note that this is a level above mime-types. Maybe we can just return
  a list of mime types? Shrug.)

A volume monitor implementation, such as the HAL one, would also map
results from the drive's classification into the content classification.
E.g. if the drive has G_DRIVE_CLASSIFICATION_STILL_CAMERA it will map
the content type G_MOUNT_CONTENT_PHOTOS even when no DCIM/ directory is
present. I think that's a feature.

To have a central location where we do sniffing we offer a general
utility function

 GMountContentFlags g_io_guess_content (GFile *mount_root);

that the various volume monitors can take advantage of and merge extra
results into it.

Then we want convenience API on the volume monitor to make it easy to
write applications.

 GList *  g_volume_monitor_find_drive_by_classification (GVolumeMonitor            *volume_monitor,
                                                         GDriveClassificationFlags  classification_flags);


 GList *  g_volume_monitor_find_mount_by_classification (GVolumeMonitor            *volume_monitor,
                                                         GMountContentFlags         content_flags);
          (and an async version as well)

Thus, in e.g. f-spot you'd do this for the import dialog

 mounts = g_volume_monitor_find_mount_by_classification (volume_monitor, G_MOUNT_CONTENT_PHOTOS);

or for Evolution

 mounts = g_volume_monitor_find_mount_by_classification (volume_monitor, G_MOUNT_CONTENT_CONTACTS);

or Rhythmbox

 mounts = g_volume_monitor_find_mount_by_classification (volume_monitor, G_MOUNT_CONTENT_MUSIC);

It would probably be good with input from applications developers if
this API is what they want. Thoughts?

(I idly wonder if 32 bits is enough for these flags; maybe use a
guint64?)

Oh, sorry for all this newly proposed API. But I strongly feel that
drive and content classification is necessary to make it easy to write
media applications.

       David