Re: stuck in file crawl task loop



On Mon, 2007-01-22 at 12:36 -0500, Joe Shaw wrote:
> Hi,

Hi.

> Are you running 0.2.14?

Yep.  0.2.14-0ubuntu3 exactly.

> A bunch of looping issues were fixed there.

Ahhh.  Good to know then.

> Do you have any clock skew between your local machine and the NFS
> server?

Nope:

$ ssh nfsserver date; date
Mon Jan 22 16:44:40 EST 2007
Mon Jan 22 16:44:40 EST 2007

Use NTP here.

> If you touch a (previously non-existent) file there, and
> compare the timestamps between "stat <file>" and "date", what's the
> difference there?

I think I understand you:

$ rm /data/foo; touch /data/foo; ssh nfsserver stat /mnt/data/foo\; date
  File: `/mnt/data/foo'
  Size: 0               Blocks: 0          IO Block: 4096   regular empty file
Device: fc05h/64517d    Inode: 50197       Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1001/   brian)   Gid: ( 1001/   brian)
Access: 2007-01-22 16:47:07.000000000 -0500
Modify: 2007-01-22 16:47:07.000000000 -0500
Change: 2007-01-22 16:47:07.000000000 -0500
Mon Jan 22 16:47:07 EST 2007

A few other attempts produced a 1s differential... I wrote it off to
timing of the ssh and so on.

> It can't, no, but Beagle falls back to using an sqlite database for
> storing the same information if it can't be written to EAs for whatever
> reason, like lack of permissions or no EA support on the filesystem.

Is there a way to see what's in the database?  Just for the
interested?  :-)

> A while back most distributions switched to gamin,

gamin, that's the name I could not remember and had no Internet access
to dig up.

> which provided the
> FAM API but as a per-user daemon, which is good for security.  However,
> gamin didn't support remote notification either.

Right.

> Gamin then was more or less abandoned,

Any idea why?

> and a lot of applications just
> access the inotify APIs directly,

Blech.

> although some people I think are still
> using gamin (which can wrap any API, like dnotify or inotify).

And as it seems, provides a good layer of abstraction for the
application developer, no?

> The right thing to do is for these remote filesystems to implement file
> notification themselves.  CIFS (Samba) supports notification at a
> protocol level, and I know that Samba (although maybe only Samba 4) uses
> inotify for file notification.

Cool.  So a client of a CIFS Samba share can request inotify events just
like a local filesystem can?

> I don't know what the state of file
> notification is in NFS, although it's probably in NFSv4.  I don't know
> what the state of NFSv4 is on Linux.

Indeed.

> As for Beagle and FAM, the file system backend has pluggable event
> watchers, so there's no probably no inherent reason why it couldn't work
> with FAM.  Originally there was a backend against .NET's built-in file
> system watcher -- which on Linux used FAM on the backend -- but that API
> wasn't rich enough and so things were always broken.  I think that FAM's
> own API is rich enough for us to use, it's just a matter of someone
> sending a patch for it.

I see.  But by FAM of course given that most distros have dropped it for
gamin or at least disabled the remote file notification bits, it's
really quite moot, no?

Wasn't there a beagle SOC project to do proper remote indexing?  i.e.
running beagled on the (say, NFS) file server (and thus having access to
EAs and inotify) and having the client beagled talk to it?  What is the
status of this if any?

b.

-- 
My other computer is your Microsoft Windows server.

Brian J. Murrell

Attachment: signature.asc
Description: This is a digitally signed message part



[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]