Re: mnotify: mixing nonotify & simple mount watching (Was: Re: [RFC/PATCH] Nonotify...)

[John McCutchan <ttb tentacle dhs org>]

On Sun, Aug 15, 2004 at 07:48:58PM +0200, nf wrote:
> > 
> > Ahh, now I see your point. With inotify the gui probably does need to
> > 'buffer' events for a bit so they aren't updating on every event. This
> > behavior does not require a timer. 
> > 
> 
> "Buffer events for a bit": This smells very much like you need a timer.
> Cause how long does "a bit" last - one second, two seconds,...? ;-)
> 

Well it really is just an implementaiton detail.

> > When using inotify to watch a directory you can get events like
> > IN_CREATE with 'abc.txt' as the argument to the event. That requires
> > much less processing then dcontents_mtime just telling you a directory
> > has been changed. Inotify tells you the filename that was created /
> > deleted.
> 
> I do admit that the information inotify provides is quite luxurious. But
> it can get enormous in lots of cases - like writing a file with a
> shell-script into a monitored directory - which flushes the buffers with
> every line and generates tons of "write" events. But that's the good old
> discussion whether monitoring "the edge" (events) or "the level"
> (polling in fixed interval) is more efficient - I believe both have
> advantages... 

Inotify allows you to specify what you are interested in.. if you
really need to get every write to a file you need to be pretty
quick about getting those events from kernel driver. After some
playing around we should be able to come up with a queue
length that in 99% of the cases never overflow.

> 
> > > 
> > > I don't get it: Why is it a bug in the application if it doesn't get
> > > enough CPU-time to read all the events? As far as i understand inotify,
> > > there is simply NO guarantee that you receive all events. It's just
> > > "luck".
> > 
> > It's not luck but the kernel makes no guarantees that an app will get
> > all the events. The kernel just can't afford to spend the resources
> > trying to guarantee that all events are delivered to the client.
> > This is simply a luxury for the client. If you look at other operating
> > systems that deliver events such as these to user space they also don't
> > make any guarantee about the app getting all events.
> 
> Thats where i thought nonotify behaves a little more "deterministic". 
> 

If you get a QUEUE_OVERFLOW event, the client should just stat() 
everything it cares about. But with tweaking this should really only
happen in very rare cases. So even though it isn't 100% predictable..
in the real world I don't think it will matter.

> > In reality I don't think this will be much of an issue, in the kernel
> > we can make each /dev/inotify event queue arbitrarily long, and getting
> > events from the dev is VERY cheap. 
> 
> Well - but how long is "arbitrarily long", cause you have to limit those
> queues somehow? 

We can tweak the queue lengths until things work for most cases.

> 
> Btw, one idea i had to solve the congestion problem was to write the
> events in some kind of hash-table and therefore events of the same type
> like ("WRITE to file A") will overwrite and older ("WRITE to file A")
> event, but never overwrite ("DELETE file B"). But then i think you
> really need to attach timestamps to the events - to interpret them in
> the right order on the client side...

That would work but then each watcher would need it's own
event hash and when the device is being read the code
would have to:
	walk each watcher's hashes collecting all events
	sort all the events collected.
	then send the events to user space.

That would become a pretty expensive read() function.

It might be worth implementing on the user space side, which
is on my TODO list for GAMIN.

> 
> Another thing: Because you are an expert on kernel-vfs: Do you have an
> idea how to solve this issue: How to get "umount -f" working with open
> filedescriptors. Detaching the open fd's from the vfs somehow and
> disabling them...

That is possible to do (Just look at how inotify handles unmount in
generic_superblock_shutdown())., but that would break applications left, right
and center. Apps aren't designed to handle files disappearing out
from underneath them. I think a better way to handle that is to fix the apps.

John