Re: [Tracker] 0.6.94 performance questions

[Jin Suh <jinssuh yahoo com>]

I use a USB 2.0 thumb drive to contain the tracker outputs by creating a link /root/.cache/tracker -> /mnt/out/cache/tracker. Will this make performance slow to write *.db to a thumbdrive? The size of the thumbdrive is 4GB and the current cache size is about 500MB. I used this way in 0.6.91 without problems (I think it took less than an hour back then for the same 90GB Linux /home files). 

It is still indexing for 3 days. Now the applet says "29 days and 11 hours to index, done: 30000 of 870000".

My only purpose is to use the tracker with a full speed to index data and nothing else and then I am going to use the tracker-search. So, is there a good way to tweak the src codes to index using highest performance? I already set the throttling to (0 fastest). It doesn't seem to work that way.

 

Does the development trunk contain the latest updates? Do you think this will improve indexing performance?

Thanks,

Jin

---

From: Martyn Russell <martyn imendio com>
To: Jin Suh <jinssuh yahoo com>
Cc: Tracker-list <tracker-list gnome org>
Sent: Thursday, May 21, 2009 12:04:11 PM
Subject: Re: [Tracker] 0.6.94 performance questions

On Thu, 2009-05-21 at 08:06 -0700, Jin Suh wrote:
> 1. I have about 90GB linux files - various files including scripts,
> data, src codes, tgz, bz2 and tar files under /home. The tracker
> applet says it will take 26 days to finish indexing. Windows seem much
> faster (11 min. for 2GB data folder).

Hi,

The 26 days is an estimation. In the last few days we have really
improved this too. We were finding that it was creating a database of
~25Mb just for 4k items. We now do approximately 5Mb for 10k items. In
the next release this will be greatly improved.

> 2. In the config files, I set the performance as 0 (fast),
> SmartPausing is off, no watch but just crawling /home and the memory
> for faster indexing. Let me know if you need the config files.

OK, again, we have been doing a lot of performance testing over the past
2 weeks and really tightened our belts here (so to speak).

> 3. I often see a message saying "indexer is paused due to low
> memory(disk) or high load".  Why is that?

We can't be more specific due to legacy APIs we don't want to break
during 0.6 releases. But it means either your disk space is low (you can
configure when it triggers this in the config) OR we pause because we
are monitoring the file system and a lot of events are being produced
(like when copying lots of files around).

> 4. I see a bunch of places in the src that sets the "nice -n19" which
> is the slowest. I want to run the tracker with a full performance all
> the time in indexing. Could you tell me how to speed it up for faster
> indexing?

Yea, this isn't the slowest and it will use your processor, but it won't
have *priority* over other processes, that's the point. Also if we
didn't add throttling of any sorts, it would bring your computer to its
knees ;)

> Thank you guys, as always,

No problem :)

--
Regards,
Martyn