Re: stuck in file crawl task loop

[Joe Shaw <joeshaw novell com>]

Hi,

On Mon, 2007-01-22 at 17:00 -0500, Brian J. Murrell wrote:
> On Mon, 2007-01-22 at 12:36 -0500, Joe Shaw wrote:
> > Are you running 0.2.14?
> 
> Yep.  0.2.14-0ubuntu3 exactly.

Are those the packages that Kevin Kubasik put up?  If so, those are
actually a CVS snapshot before 0.2.14 was release, and they probably
have looping bugs in them still.

(Kevin, can you please pull those packages offline, and/or build new
official 0.2.14 ones and put them up?)

The Ubuntu guys have given us pushback on putting new versions in to
backports; they want us to only backport specific fixes.  So what we're
going to do is build new packages and host an apt-get repository on
beagle-project.org instead.

> I think I understand you:
> 
> $ rm /data/foo; touch /data/foo; ssh nfsserver stat /mnt/data/foo\; date
>   File: `/mnt/data/foo'
>   Size: 0               Blocks: 0          IO Block: 4096   regular empty file
> Device: fc05h/64517d    Inode: 50197       Links: 1
> Access: (0644/-rw-r--r--)  Uid: ( 1001/   brian)   Gid: ( 1001/   brian)
> Access: 2007-01-22 16:47:07.000000000 -0500
> Modify: 2007-01-22 16:47:07.000000000 -0500
> Change: 2007-01-22 16:47:07.000000000 -0500
> Mon Jan 22 16:47:07 EST 2007

Correct.  So, this should work, but the pre-0.2.14 packages have me a
little nervous.

> > It can't, no, but Beagle falls back to using an sqlite database for
> > storing the same information if it can't be written to EAs for whatever
> > reason, like lack of permissions or no EA support on the filesystem.
> 
> Is there a way to see what's in the database?  Just for the
> interested?  :-)

Yep, it's ~/.beagle/Indexes/FileSystemIndex/FileAttributesStore.db.
Just open it up with sqlite or sqlite3 and you can poke around in there.

> > Gamin then was more or less abandoned,
> 
> Any idea why?

I think the developers got a mostly working solution and just left it to
do other things.  There wasn't a community built up around it.

Also note that the lack of remote file notification was a design
decision; I don't think there were ever plans to add it.

> > although some people I think are still
> > using gamin (which can wrap any API, like dnotify or inotify).
> 
> And as it seems, provides a good layer of abstraction for the
> application developer, no?

Well, it would if its API didn't suck. :)  The inotify API is really
simple and trivial to use.  FAM's is much more complex.

> Cool.  So a client of a CIFS Samba share can request inotify events just
> like a local filesystem can?

I believe so.  That was the plan, and I know that Robert did a lot of
work to inotify to handle Samba's use cases.  To get a definitive answer
you'd have to ask the Samba guys though. :/

> I see.  But by FAM of course given that most distros have dropped it for
> gamin or at least disabled the remote file notification bits, it's
> really quite moot, no?

Yeah, I think this is a big reason why no one has done it yet. :)

> Wasn't there a beagle SOC project to do proper remote indexing?  i.e.
> running beagled on the (say, NFS) file server (and thus having access to
> EAs and inotify) and having the client beagled talk to it?  What is the
> status of this if any?

The idea behind the SoC project was searching other running daemons,
yeah.  The use case in mind was less of having all your files on a
server, but more having your files scattered across a few different
desktops.

The work is pretty much stalled.  The two guys who worked on it pretty
much dropped off after the project was finished, and Fredrik began
integrating it over the summer, but he too has pretty much vanished.

If you have all of your files on an NFS server and they don't change
much, you could use the beagle-build-index tool to create a static index
of the files on the NFS server, and export that to Beagle for searching
it on the client.  The obvious downside to this is that you don't get
change notification.  Someone -- Alex Macdonald, I think -- wrote a
small daemon process that watched for inotify events on the backing file
system and updated the static index.  I believe it's in bugzilla
somewhere; it needs polish, but it might work pretty well.

Joe