Re: [Tracker] First time user on FreeBSD with some troubles

["Aron Stansvik" <elvstone gmail com>]

On 4/24/06, Jamie McCracken <jamiemcc blueyonder co uk> wrote:
> Aron Stansvik wrote:
> > On 4/23/06, Jamie McCracken <jamiemcc blueyonder co uk> wrote:
> > > Aron Stansvik wrote:
> > > > Duh! Wow now it actually works:
> > > >
> > > > In xterm nr. 1:
> > > >
> > > > $ mkdir /usr/home/elvis/test
> > > > $ echo "hej hej hej" > /usr/home/elvis/test/hello.txt
> > > > $ trackerd /usr/home/elvis/test
> > > > 060423 22:57:20 [Warning] Can't open and lock time zone table: Table
> > > > 'mysql.time_zone_leap_second' doesn't exist trying to live without
> > > > them
> > > > DB initialised
> > > > Watching directory /usr/home/elvis/test (total watches = 1)
> > > > waiting for file events...
> > > > processing /usr/home/elvis/test with event TRACKER_ACTION_CHECK
> > > > indexing file /usr/home/elvis/test
> > > > processing /usr/home/elvis/test/hello.txt with event TRACKER_ACTION_CHECK
> > > > processing /usr/home/elvis/test with event TRACKER_ACTION_CHECK
> > > > indexing file /usr/home/elvis/test/hello.txt
> > > > Extracting Metadata for file /usr/home/elvis/test/hello.txt with mime text/plain
> > > > text file is /usr/home/elvis/test/hello.txt
> > > > saving text to db with file_id 2
> > > > 24 bytes of text successfully inserted into file id 2
> > > >
> > > > In xterm nr. 2:
> > > >
> > > > $ tracker-search hej
> > > > /usr/home/elvis/test/hello.txt
> > > >
> > > > Hm. What does this give? Is it choking on something in my home directory?
> > > no its probably that you have a huge no of files in home (including
> > > subdirs) and its queuing them up to be indexed.
> > >
> > > I will definitely improve this in the next version as it looks like its
> > > hanging when it aint.
> > >
> > > glad it all works!
> > >
> > > Good night!
> >
> > And good morning!
> >
> > Well it's definately choking on something. Or maybe not choking per
> > se, but something is hindering the indexing. I've had trackerd running
> > all night and day with my entire home directory as target, and so far
> > no indexing messages have come up.
> >
> > Is there a limit to how many files I can reasonable expect tracker to
> > index, and if so; what is the limit? Is it dependant on the platform?
> > Does each monitored directory require an open file handle, each file
> > in the directory tree? If so, that might be the problem, since I have
> > some untarred source distributions and other whatnot in ~.
> >
> > I'm using Gamin as the monitoring backend, which in turn I think is
> > using kqueue's on BSD.
> >
> > I shall compile tracker with debugging symbols and see what gdb says.
> >
> > I tried running trackerd against various not-so-populated directories
> > in my home directory, with success.
> >
> > Best regards,
> > Aron
> >
> > PS. I don't have Poppler installed at the moment, so at first I
> > thought that maybe trackerd was halting trying to index PDF files. But
> > I tested running it against a directory with some PDFs in it and it
> > seemed to work fine. DS.
>
> unlikely to be the problem. File handles is a good possibility if you
> have very broad directories (it is designed to handle deep depths as we
> use a breadth first sweep rather than a depth first)

Okay, I think my directories are pretty moderately broad. I'm also
suspecting some file handle limit, but it's puzzling to not see any
"Too many open files" messages. I digged some into FreeBSD
documentation and found in tuning(7):

"The kern.maxfiles sysctl determines how many open files the system sup-
ports.  The default is typically a few thousand but you may need to bump
this up to ten or twenty thousand if you are running databases or large
descriptor-heavy daemons.  The read-only kern.openfiles sysctl may be
interrogated to determine the current number of open files on the system."

I shall do some investigation based on this information. I'll be back
with results/findings.

> I should have this sorted in the next version and I will test against
> very huge directories.

Great.

> Thanks for bringing this up.

Tracker is a too good idea to just leave in the rain because of some
initial hazzles with my somewhat unorthodox desktop platform :)

Aron