Re: [Tracker] Roadmap for 0.12

["Florent Viard" <fviard lacie com>]

Le 28/07/2011 13:41, Adrien Bustany a Ãcrit :
> On Thu, 28 Jul 2011 11:26:10 +0200, Philip Van Hoof wrote:
> > On Wed, 2011-07-27 at 22:56 +0300, Adrien Bustany wrote:
> > > Le Wed, 27 Jul 2011 10:24:17 +0100,
> > > Martyn Russell <martyn lanedo com> a Ãcrit :
> >
> > > > â Deleting orphaned resources?
> > > >    This is not straight forward to do and we decided though we 
> > > want to
> > > >    do it, we just don't have time. This is useful in cases where (for
> > > >    example, you have artists in the database for music that no longer
> > > >    exists. What's hard about this is knowing what can be cleaned up).
> > >
> > > Additionally, this can easily be moved to an external process without
> > > too much overhead... What we have for contacts is Harmattan is a daemon
> > > that exports a DBus interface. With this interface you can:
> > > 1. Register a SPARQL query to delete orphaned resources, with a unique
> > >    identifier
> > > 2. Increase the "load" for the query
> > >    The load is an arbitrary double, when it reaches 1 the cleanup query
> > >    is ran by the daemon. For exemple, the qtcontacts-tracker backend
> > >    registers a contacts cleanup query, and increases the load everytime
> > >    you insert/delete a contact so that the cleanup query is ran every
> > >    100 updates.
> > > This is only one approach to the problem, which we chose because it was
> > > good enough and not too intrusive. There are surely other solutions,
> > > which might or not be better according to what your requirements are.
> >
> > Hmm, I think the solution should be agnostic about the SPARQL queries
> > and shouldn't require registering.
>
> Yes, the whole point of this solution is that it works with an
> unmodified Tracker... The refcounting solution is indeed nicer, but was
> not an option for Harmattan in our timeframe, and also has to be very
> carefully specified to avoid surprising behaviours :)
>
> <snip>

Hi,

Regarding this question I have a suggestion:
Doing a little like what does apt-get:
Adding an "autogenerated" flag to the resources and do extractors set 
this flag to "ON" if the resource is created by this extractor and that 
it is an autogenerated one (for example for an album or an artist when 
extracting songs).

This flag is never set when the resource already exists and is just updated.
This flag is cleared if the resource is updated "directly" by the user 
or by an extractor that has this resource as main subject.

Then, an external "cleaner" thread or process, regularly scan resources 
with this flag "ON" and:
1) use the ref count to determine if this resource is now useless and 
could be removed.
2) (needs no change in code but is certainly slower for the "cleaner" 
process): For each of these resources, search all the other resources 
having a link to it, and remove if none found.


++

Florent Viard
Software Development Engineer
........................................
fviard lacie com
www.lacie.com



LaCie CloudBox on YouTube http://youtu.be/FCrkN47CBr4