Re: [Tracker] Tracker daemon/indexer responsibilities

[Jamie McCracken <jamie mccrack googlemail com>]

On Wed, 2008-06-25 at 19:31 +0100, Martyn Russell wrote:
> Jamie McCracken wrote:
> > On Wed, 2008-06-25 at 16:46 +0100, Martyn Russell wrote:
> > > Hi all,
> > >
> > > Thinks are all going well on the indexer-split branch, however, it
> > > occurred to me that daemon has its work duplicated in the indexer. We
> > > need to resolve where the responsibility lies for the daemon and the
> > > indexer.
> > >
> > >
> > > The Modules
> > > ===========
> > >
> > > First about the modules. So we have these modules, they all share a
> > > common API. This API includes functions to:
> > >
> > > - Index content
> > > - Get directories
> > > - Know if a file or directory should be ignored
> >
> >
> > actually this is wrong - only index content should be in the module and
> > only the indexe rneeds to access it
> >
> > a desktop file should specify what directories and/or regex for files to
> > be indexed/ignored 
> >
> > that way only the desktop file is shared
>
> This isn't exactly what I mean.
>
> Take the GAIM module, it should know:
>
> - Where chat logs are
> - How to index those chat logs.
> - Know what files (in the directory where the chat log are) to ignore.
> - Know what sub directories to ignore.
>
> Consider these factors for a module which some 3rd party writes for
> product foo. How will Tracker know how to do any of those things above?

the desktop/ini file will contain fields for :

directories to watch
recursive directories to watch
ignore directories regex
files to index regex
files to ignore regex
index executable
index module name
file mime types
service name

we may also need a user config file for each module/executable which the
index content will use

enabled
mod specific settings



> The idea here is that we have a shared module which can be loaded by the
> daemon (for example, to know which directories to monitor, like the chat
> log directory for the GAIM module) and the indexer can load the module
> to call the indexer function to files passed to it by the daemon are
> indexed correctly.
>
> In the case above, I meant that the same module API could be used by the
> daemon to know it should monitor the /usr/share/ directory the .desktop
> file changes (for applications). Typically /usr/share is not monitored
> because it isn't in $HOME (the default location). So any added or
> removed applications would be missed.
>
> > I imagine with xesam in the future some standard will be created for
> > specifying how to index content and its likely to use desktop files too
>
> Oh, have you heard this or is this you intuition? :)


we discussed this previously (mostly for executables rather than
modules)

things to pin down:

1) stdout xml output format of extractor/indexer
2) which directory to put these in
3) How to handle more complex things like crawlers (EG evolution emails)

jamie