[Tracker] Ready for review - Was: Ontology change support upgrading nrl:maxCardinality 1 to multiple

[Philip Van Hoof <philip codeminded be>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi guys

It's done! I'm amazed that so soon I figured out my own old code again.

Available for review and test (please do test) here:

https://git.gnome.org/browse/tracker/log/?h=maxcardinality-change-support

I will of course squash all the commits together.

This was my own test:

> pvanhoof lenny:~$ cat
> /opt/tracker/share/tracker/ontologies/94-extra.ontology @prefix dc:
> <http://purl.org/dc/elements/1.1/> . @prefix nco:
> <http://www.semanticdesktop.org/ontologies/2007/03/22/nco#> . 
> @prefix nfo:
> <http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#> . 
> @prefix nie:
> <http://www.semanticdesktop.org/ontologies/2007/01/19/nie#> . 
> @prefix nrl:
> <http://www.semanticdesktop.org/ontologies/2007/08/15/nrl#> . 
> @prefix nao:
> <http://www.semanticdesktop.org/ontologies/2007/08/15/nao#> . 
> @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> . 
> @prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> . @prefix
> tracker: <http://www.tracker-project.org/ontologies/tracker#> . 
> @prefix xsd: <http://www.w3.org/2001/XMLSchema#> . @prefix nao:
> <http://www.semanticdesktop.org/ontologies/2007/08/15/nao#> . 
> @prefix extra: <http://www.tracker-project.org/ontologies/extra#>
> .
>
>
> extra: a tracker:Namespace, tracker:Ontology ; tracker:prefix
> "extra" ; nao:lastModified "2014-08-28T14:15:00Z" . # <- change
> this during testing
>
> extra:hasThing a rdf:Property ; rdfs:label "hasThing" ; 
> rdfs:comment "Connects a media container with a single media stream
> contained within." ; rdfs:subPropertyOf nie:hasPart ; rdfs:domain
> nfo:Media ; nrl:maxCardinality 1 ; # <- comment this out during
> testing rdfs:range nie:DataObject .
>
> pvanhoof lenny:~$ cat
> /opt/tracker/share/tracker/ontologies/94-extra.description @prefix
> extra: <http://www.tracker-project.org/temp/extra#> .
>
> <virtual-ontology-uri:93-extra.ontology> a dsc:Ontology ; dsc:title
> "extra-specific annotations Ontology" ; dsc:description "extra
> properties" ;
>
> dsc:author "Red Hat, Inc" ; dsc:editor "Red Hat, Inc" ; dsc:gitlog
> "http://git.gnome.org/cgit/tracker/log/data/ontologies/94-extra.ontology";;
>
>  dsc:localPrefix "extra" ; dsc:baseUrl
> "http://www.tracker-project.org/ontologies/extra#";; 
> dsc:relativePath "./94-extra.ontology" . pvanhoof lenny:~$
>
>
> pvanhoof lenny:~$ tracker-sparql -qu 'insert { <b> a
> nie:InformationElement . <a> a nfo:Media ; extra:hasThing <b> }' 
> Done pvanhoof lenny:~$ tracker-sparql -q 'select ?a ?b { ?a
> extra:hasThing ?b }
> > ^C
> pvanhoof lenny:~$ tracker-sparql -q 'select ?a ?b { ?a
> extra:hasThing ?b }"
> > ^C
> pvanhoof lenny:~$ tracker-sparql -q 'select ?a ?b { ?a
> extra:hasThing ?b }' Results: a, b

To be expected, it's a single value atm:

> pvanhoof lenny:~$ tracker-sparql -qu 'insert { <c> a
> nie:InformationElement . <a> a nfo:Media ; extra:hasThing <c> }' 
> Could not run update,
> GDBus.Error:org.freedesktop.Tracker1.SparqlError.Constraint: Unable
> to insert multiple values for subject `a' and single valued
> property `extra:hasThing' (old_value: '100001', new value:
> '100003') pvanhoof lenny:~$ tracker-sparql -q 'select ?a ?b { ?a
> extra:hasThing ?b }'Results: a, b

Apply then the ontology change, restart tracker-store, then:

> pvanhoof lenny:~$ tracker-sparql -qu 'insert { <c> a
> nie:InformationElement . <a> a nfo:Media ; extra:hasThing <c> }' 
> Done pvanhoof lenny:~$ tracker-sparql -q 'select ?a ?b { ?a
> extra:hasThing ?b }'Results: a, b a, c
>
> pvanhoof lenny:~$

No data loss, and it suddenly allows multiple values.


Kind regards,

Philip



On 24/09/2014 22:32, Philip Van Hoof wrote:
> Hi everyone,
>
> As many might have noticed in the changelog of 1.2.1 release we 
> decided to revert the nrl:maxCardinality change made on 08-28.
>
> That's because this ontology change force a reindex and removal of 
> your existing metadata. That's data loss for certain users, so not 
> acceptable for the majority of distributors distributing Tracker.
>
> A week or so ago I started with adding support for this kind of 
> ontology changes.
>
> Of course can't we ever in a reliable way support
> nrl:maxCardinality change from multiple to one: we'd lose type
> information if we'd only support it for CSV-like strings and for
> any other type we can't store the multiple old values in a single
> value solution. I don't aim to ever support this. I do aim to
> support the one to multiple use-case.
>
> People interested in this can go look here:
>
> https://git.gnome.org/browse/tracker/log/?h=maxcardinality-change-support
>
>  Right now it will change the ontology's introspection and it'll
> allow free passage of the change without aborts or critical
> warnings.
>
> What doesn't yet work is in the catacombes of cruelty, angst and 
> untold stories of brain torture, create_decomposed_metadata_tables,
> to detect that this particular change is taking place this time.
> And then do the rename of the table, create (without the columns of
> the old single value), ensure the multi-value table is already
> created and then finally copy the renamed table's single values to
> the multi-value table to end with dropping the renamed table.
>
> The code is there but it doesn't work.
>
> This I will probably debug until it works later this week. I
> silently hoping that somebody will beat me to it. I fear I will
> have to do this miserable create_decomposed_metadata_tables stuff
> one more time.
>
> Kind regards,
>
> Philip _______________________________________________ tracker-list
> mailing list tracker-list gnome org 
> https://mail.gnome.org/mailman/listinfo/tracker-list

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.20 (MingW32)

iQEcBAEBAgAGBQJUJX+mAAoJEEP2NSGEz4aDrUcH/iLZuSPwB89ZLRlv79VlNJk2
7jBZ1HEmJh4LIxJaI6YtaB0I3oZJboDZUtg5PHM7m2/K0JUlOY3toaicUq3biuBv
rvFwcT8xGUYpOYS11C3NZJncA/1p989xKyqGIprwxjJvrfFd2cgbfUHjw0/Rq23k
BsdrStqL/YS9zs2pFRO6mOgTos1n/TbY2pnr5rZT6aH/ama7AeQs9Mc22Sr5Sh7O
JDTZrxuqc8gabO/q5+23W4jqSlpBhjDz3pHP/DizZuPa8z8nJFMwX1876inNbsKY
V4jArZeH6i+h4ZwLdcAki1As8bExG9CDJimH5McXDj3nPj75RyorrEAA8X6wY9E=
=/tE5
-----END PGP SIGNATURE-----