Re: File monitor rewrite: Solaris (and other) help wanted

[Morten Welinder <mortenw gnome org>]

> My plan is to make it a guarantee of the API that both CREATED
> and CHANGED events will always be followed by a CHANGES_DONE
> hint.

Great plan, but you cannot get that in a meaningful way.  Think

    creat
    write
    mmap
    close
    # Further writes by way of the mapped region

I don't think you can detect the end of that write.

Morten








On Thu, Jan 15, 2015 at 10:42 AM, Ryan Lortie <desrt desrt ca> wrote:
> hi Aleksander,
>
> On Thu, Jan 15, 2015, at 06:28, Aleksander Morgado wrote:
> > Currently GFileMonitor doesn't have a unique way to know whether a file
> > got closed. There is the changes-done-hint event, but that covers IIRC 2
> > things: files getting closed and also a "virtual" emission which happens
> > after some time even if the files were not closed (think of a log file
> > which never gets closed). The main issue is that if you want to get
> > notified of when a file was fully updated *and* closed, you need to
> > fallback to raw inotify. The rationale for wanting to get notified only
> > when the file got closed is e.g. Tracker monitoring the Downloads/
> > directory. We may not want to extract file info for an ongoing download,
> > just for when the download is fully finished (and destination file
> > closed). More background here:
> > https://bugzilla.gnome.org/show_bug.cgi?id=635765
>
> Short story: I want to add a flag to either disable or enable emission
> of virtual changes-done hints on monitor backends that can reliably
> handle it themselves.
>
> Even for fully-capable backends, I think virtual emissions are
> potentially important because, even if the file is not closed, someone
> watching it may want to update their opinion of its contents
> periodically.  The question is only about what the default should be.
> Those who favour a nice clean API would say that virtual emissions
> should be off by default.  Those who favour backwards compatibility
> would suggest that today's behaviour of virtual emissions should be kept
> as-is unless explicitly disabled.  I'm not sure what we will do.
>
> Unfortunately, there's a longer story: None of the backends support
> reliable emission of non-virtual changes-done.
>
> Here's why:
>
> My plan is to make it a guarantee of the API that both CREATED and
> CHANGED events will always be followed by a CHANGES_DONE hint.  That's
> already enforced in the state machine logic in GFileMonitorSource in the
> branch.  My reason for that is that apps like Tracker should not want to
> response to CREATED events until the file content is complete.
>
> The idea (taking your download example) is that a file is created
> something like so:
>
>   creat()
>   write()
>   write()
>   close()
>
> which sends us IN_CREATE, IN_MODIFY, IN_MODIFY, IN_CLOSE_WRITE.
>
> As you mention, it doesn't make sense for the app to respond to the
> empty (or maybe very slightly populated) download just because it saw a
> CREATED event from GFileMonitor -- it should wait for the CHANGES_DONE.
> Consider this case:
>
>   creat()
>   close()
>
> in that case, we'd see IN_CREATE, IN_CLOSE_WRITE, with no IN_MODIFY.  We
> still want to see a CHANGES_DONE event in that case, though so that the
> app knows that the empty file is the 'final result'.  This is the basis
> of my opinion that CREATED should always get a CHANGES_DONE after it,
> even without actual CHANGED events.
>
> With the new support for move and rename events, a file created by the
> "write to temp and mv into place" method will be reported either as
> MOVED_IN or RENAMED with no CHANGES_DONE.  That's okay, because you know
> that a file that was MOVED_IN or RENAMED into place is ready to be
> handled immediately.
>
> Unfortunately, there is another set of cases.  IN_CREATE is sent both
> for the creat() case (in which case it will be followed by
> IN_CLOSE_WRITE) but also for cases like mknod(), bind() on a unix
> socket, mkdir(), etc..  In those cases, we will receive no
> IN_CLOSE_WRITE.  We could detect that situation by looking at the
> filesystem and seeing that the newly-created file is of a special type,
> but link() also produces IN_CREATE without IN_CLOSE_WRITE.
>
> The only thing that saves us in this second case is that we get a
> virtual emission of CHANGES_DONE a couple of seconds later.  At least
> link() is uncommon, although it stands to become a more common way of
> creating files, via O_TMPFILE.
>
> In short, I believe that we currently don't have any backend for which
> we could safely disable emission of virtual CHANGES_DONE events.
> Ideally, in the future, we will gain a mechanism in inotify to tell the
> difference between "file created via open for writing, IN_CLOSE_WRITE
> coming soon" and "inode appeared in the file system in complete form".
> At that point we could expose a new event type in GFileMonitor
> (_APPEARED?) that doesn't need CHANGES_DONE to be emitted.
>
> Cheers
> _______________________________________________
> gtk-devel-list mailing list
> gtk-devel-list gnome org
> https://mail.gnome.org/mailman/listinfo/gtk-devel-list