Re: [Tracker] Controlling tracker priority and settings on tracked files

[Richard Shephard <Richard Shephard altech-multimedia com>]

Hi Martyn,

Thanks for the answers.

> What version are you using?
Tracker 0.10.35

I will need to dig into the application a bit to see if the store is being accessed by our app. I don't see 
it in the 'top' list so it did appear on the surface as doing anything with tracker.

I could not find documentation on the tracker extract-rule mentioned below. Can you perhaps provide a link?



Thanks.

-----Original Message-----
From: Martyn Russell [mailto:martyn lanedo com]
Sent: 07 September 2012 07:12 PM
To: Richard Shephard
Cc: tracker-list gnome org
Subject: Re: [Tracker] Controlling tracker priority and settings on tracked files

On 07/09/12 16:21, Richard Shephard wrote:
> Hi,

Hello Richard,

> I know that I have seen something like this before in the docs, I just
> can't find it again.
>
> Our application is using tracker in more or less the standard
> configuration. However, due to processor consumption, we have reduced
> the miner indexing speed to 19, however, tracker-store still consumes
> 90+% of CPU power. Tracker-miner is reduced in priority, but not store.

If you're using Tracker >= 0.14.0, we use SCHED_IDLE to help here. We
also have the throttle which is what you're referring to above. What
version are you using?

> Is there a nice way of reducing tracker-store's CPU usage? (other than
> renice-ing it from the command line)

Hmm, we could patch tracker-store if that's what you mean. Though it
shouldn't be necessary because the traffic is driven from the miners
unless you're adding to that with client requests (queries/updates)

We decided not to nice() the store because it is client driven and if
you need the information fast from application A you don't want it
restricted because of nice() due to application B being a constant hog.

> Also, I know that the miners can be told what directories and files to
> ignore, but is there a way of telling it what files only to include?

 From the config, you can specify directories yes, but not files.

You can specify files to index by the dbus interface, but that's for
applications usually. It sounds like this wouldn't be useful to you if
you want to index all files of a specific type.

> i.e. we only need multimedia file types, so we only want to index files
> like mp4,mp3, etc. - not the entire array of possible files.

I see, that's an interesting approach to indexing.

The closest mechanism we have to that is:

   $prefix/share/tracker/extract-rules/*.rule

For example:

   /usr/share/tracker/extract-rules/10-mp3.rule

This allows you to specify which mime types will be covered by the
extractor modules installed. This would certainly allow you to control
the data extracted and could improve performance. This would work
because only the MP3 extractor would ever be found for all files we
check. So you would have file data in the database on a basic level
(name, size, etc) but for MP3 files you would have full class
information too.

To be able to specify mime types to explicitly allow would need some
patching in Tracker but it wouldn't be too hard to add that.

> The "problem" we have is that one aspect of the project includes a SD
> card with a full Linux install on it. The code mounts this because it
> thinks it is media (and there well maybe), but tracker goes banana's
> trying to index the full system. This makes the database file grow past
> 600MB. There is not 600MB worth of multimedia files to store, but mostly
> Linux fs style files we have no need of.

I see. Not long ago we disabled (by default) indexing removable devices
when they're inserted. Depending on your version and config, this might
be happening and could easily be turned off.

The GNOLME docs application wanted something similar here, they wanted
to be able to index content on demand for removable devices because it's
just too hard to generalise about when you need that content and when
you don't.

If you always mount that media in the same place and you know where the
media files are located, you could add that directory specifically to
the config and remove indexing removable media in the preferences. That
should work.

HTH,

--
Regards,
Martyn

Founder and CEO of Lanedo GmbH.

E-Mail Disclaimer : http://www.altron.co.za/email.asp
Should you have any questions regarding this e-mail legal notice please contact us on
1. Telephone number : +27315082800
2. Fax number : +2731539 3370
3. E-mail address : info altech-multimedia com