Re: [Tracker] Roadmap for 0.12

[Philip Van Hoof <philip codeminded be>]

On Wed, 2011-07-27 at 22:56 +0300, Adrien Bustany wrote:
> Le Wed, 27 Jul 2011 10:24:17 +0100,
> Martyn Russell <martyn lanedo com> a Ãcrit :
 
> > â Deleting orphaned resources?
> >    This is not straight forward to do and we decided though we want to
> >    do it, we just don't have time. This is useful in cases where (for
> >    example, you have artists in the database for music that no longer
> >    exists. What's hard about this is knowing what can be cleaned up).
>
> Additionally, this can easily be moved to an external process without
> too much overhead... What we have for contacts is Harmattan is a daemon
> that exports a DBus interface. With this interface you can:
> 1. Register a SPARQL query to delete orphaned resources, with a unique
>    identifier
> 2. Increase the "load" for the query
>    The load is an arbitrary double, when it reaches 1 the cleanup query
>    is ran by the daemon. For exemple, the qtcontacts-tracker backend
>    registers a contacts cleanup query, and increases the load everytime
>    you insert/delete a contact so that the cleanup query is ran every
>    100 updates.
> This is only one approach to the problem, which we chose because it was
> good enough and not too intrusive. There are surely other solutions,
> which might or not be better according to what your requirements are.

Hmm, I think the solution should be agnostic about the SPARQL queries
and shouldn't require registering.

What I have in mind is a fairly simple stored reference counting
solution that if object is a resource, adds a reference to the resource
in case of INSERT, and if it's a resource in case of DELETE deletes a
reference.

Then creation of a resource is floating in case of anonymous node as
reason for creation, and fixed or 1 in case of non-anonymous creation of
a resource.

So basically:

INSERT { <fixed> a nie:InformationElement . 
         <blah> a nie:InformationElement }

would create <fixed> and <blah> with each 1 as reference count

INSERT { <blah> nie:isRelatedTo <fixed> }

would add a reference to <fixed> because object is a resource. Giving us
<blah> has 1 and <fixed> has 2


DELETE { <blah> nie:isRelatedTo ?o } WHERE { <blah> nie:isRelatedTo ?o }

would take a reference from <fixed>, so puts us back in the original
situation.

Whereas

INSERT { _:anon a nie:InformationElement ; nie:url 'anon:test' . 
         <blah> a nie:InformationElement }

would create an anon and <blah> with the anon having 0 as reference
count and <blah> having 1 as reference count

INSERT { <blah> nie:isRelatedTo ?o } WHERE { ?o nie:url 'anon:test' }

would add a reference to the anon because object is a resource. Giving
us <blah> has 1 and the anon has 1


DELETE { <blah> nie:isRelatedTo ?o } WHERE { <blah> nie:isRelatedTo ?o }

would take a reference from the anon, and because it's now -1 the code
will remove the resource completely.

Maybe we could limit it to anonymous blank nodes instead of just
anonymous ones. Maybe we could require a keyword at INSERT and DELETE?

INSERT WREFCNT { ... } ...
DELETE WREFCNT { ... } ...

I think we should not let it happen then for INSERT OR REPLACE, but not
sure.

Etc

To me this sounds like doable to implement within one or two weeks.

Regards,

Philip
 

-- 


Philip Van Hoof
freelance software developer
Codeminded BVBA - http://codeminded.be