Re: Building off Medusa

[Alexander Larsson <alexl redhat com>]

On 11 Apr 2003, Seth Nickell wrote:

> On Thu, 2003-04-10 at 07:02, Manuel Amador (Rudd-O) wrote:
> > >
> > >
> > > Just so you know... Incremental indexing won't be possible using libfam.
> > > FAM will not scale to monitoring over about 500 files, so you definitely
> > > will not be able to get change notification on all the files on a disk.
> > > I would love a way to register with the kernel to be notified whenever
> > > *any* file changes, but I don't believe there is such a mechanism.
> > >
> > Oh, but it does, and it does well (at least on Linux and Solaris).  I 
> > just got a command-line tool which reports on stdout all files changed.  
> > You run it with a directory or file as the sole argument, and it spits 
> > out all modified files' paths, half a second after they're modified.  
> > Fam does *not* monitor *each* file.  I don't know how it does, but it 
> > doesn't watch every file.  If it would, I would agree with you.
> 
> You can add FAM (with dnotify running underneath) monitors to every
> directory on your system without problems? On my system FAM starts
> having trouble around several hundred directories. 

Fam does indeed have problems scaling, however the several-hundred-dirs 
problem is another problem i think. You're requesting lots of monitors, 
while not reading the events generated from fam. At some point the pipe 
between fam and your app will be full, and to avoid deadlocking fam will 
then not read further requests from you. 

I put a fix for this in the gnome-vfs monitor wrappers recently.

> FAM does not support recursive monitoring, which means you have to add a
> monitor for every directory to receive notification of changes to those
> files. According to the FAM FAQ, you can only have 1024 active
> "requests", which means you can only monitor up to 1024 directories. My
> computer has more than 30000 directories... so I don't see how it would
> be possible to use FAM to receive notification of changes across my
> whole filesystem.

I'm not sure about the 1024 thing, but each monitor of a file or a 
directory means one directory is opened (the fd is shared between each 
monitor that opens the same directory), and the maximum amount of open 
file-descriptors does put a limit on how many files you can monitor.

-- 
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
 Alexander Larsson                                            Red Hat, Inc 
                   alexl redhat com    alla lysator liu se 
He's a war-weary moralistic grifter with no name. She's a cosmopolitan goth 
mermaid who don't take no shit from nobody. They fight crime!