Re: [Tracker] Fwd: tracker 1.11.2

[Carlos Garnacho <carlosg gnome org>]

Hey Philip,

On Sat, Dec 10, 2016 at 4:57 PM, Philip Van Hoof <philip codeminded be> wrote:
> On Sat, 2016-12-10 at 16:36 +0100, Carlos Garnacho wrote:
>
>
> [cut]
>
> > I however think going from "odd minor number is unstable" to semver
> > versioning with minor!=0 is going to be confusing...
>
> Yes, changing it in the middle of a major release number is perhaps not
> the brightest idea. True.
>
> >  I will suggest the following plan:
> >
> > - Finish this 1.11.x/1.12 cycle
> > - Sticking to 1.12.x for as long it's needed while
> > - Gearing to a Tracker 2.x that switches to using semver approach to
> > communicate API/ABI changes.
>
> This makes a lot of sense to me.

Cool :), seeing that you/Sam agree, sounds like a plan.

> > If communicated properly, that'd at least allow us to have a single
> > last/current stable release to care about most often, and the window
> > to accumulate backwards compatible changes can be made variable based
> > on urgency (situations like this don't come up often but you never
> > know...),
>
> Well, yes. Usually are security bugfixes just patch increments. But in
> this case you solved the situation by adding sandboxing as a feature.
>
> And then semver states that 'you added or changed functionality and or
> APIs but didn't break backwards API compatibility' = minor increment.

Right, this would indeed be a reason for breaking the schedule and
releasing, or data loss situations, etc. I however hope those don't
appear often :).

> > I wouldn't mind that this unstable window is still made to
> > roughly match the 6 months gnome schedule though, and maybe release
> > 2.[x+1] with whatever might accumulate in that period.
>
> Sure. I would however not withhold from doing interim releases, and
> assign one of those to the 6 month gnome release.
>
> Not being a monorepo or monolithic architecture liker, I also don't
> think that having a cadans dictated by something like gnome is
> necessarily a good idea. But that clearly is just an opinion.
>
> I think every project should be independent.

It is mostly down to convenience, I eg. remember when gtk+ had these 9
to 12 month cycles (in the 2.x days) which was quite messy for
applications, some new features would go virtually unused until the
next gnome cycle, and close enough to gtk+ freezes when they actually
were (and bugs were reported). This incoordination ended up being
unpleasant on both sides.

I get your point, Tracker is not a project used only by gnome, but
it's the community it's always worked closest with, and it's also a
community who's "invested" a lot in it (as in making Tracker a pillar
for several core apps). Its 6 months schedule might not be fit for
everyone, but it's IMHO reliable and short enough.

> > How does that sound to you?
>
> But yes, makes sense.
>
> > I'll also take the opportunity to introduce to the ML the "roadmap"
> > that's been shaping up in my head for 2.x:
> >
> > - Getting as close to supporting the full sparql 1.1 spec as possible
> > in libtracker-data:
> >   * property paths: last weekend got halfway with this \o/
> >   * graph management: for DROP GRAPH I think triggers will perform
>
> Did ever something happen to cleaning up anonymous nodes of deleted
> subjects/context, and or do reference counting on them (and clear them
> once they reach zero references)?
>
> If not then we are still leaking those in the db afaik. We always wanted
> to do something about that.

Yeah, we still do leak those... I remember you/Jürg suggested setting
a foreign key with ON DELETE RESTRICT action from the various tables'
ID rows to the Resource table, so the cleaning up the Resource table
would fail for the still referenced nodes. I got as far as seeing
that:

- Performance would be just fine for the common ops, the trigger would
only run when trying to delete the parent key in the Resource table,
which is the once-in-a-while operation, modifications on the cols
setting the foreign key would be just as fast as they're now.

- It however wouldn't fix alone the other issue I saw happening before
the revert (graph URNs being deleted). I had a patch around that added
a Graph table, so IDs in the Resource table were ensured to be in
either rdfs:Resource or the Graph table. That already helped with
identifying and not deleting the graph URNs during garbage collection,
and seems useful for graph management, but I think I can't just add
the same RESTRICT action as CLEAR/DROP GRAPH will want pretty much the
opposite.

- I also wondered if it's more desirable, or allowed by the sparql
spec, that we actually garbage collect the inconsistent nodes. IMO not
leaving this type of data coherence up to miners/apps being educated
when deleting would be a win, but I've only seen mentions in the
sparql spec about impls being free to drop empty graphs, nothing about
triples with no longer bound elements.

> > just fine, CREATE is also easy, for LOAD/MOVE/ADD it looks like we can
> > unroll into specific updates.
> >   * the VALUES clause
> >   * the MINUS filter
> >   * CONSTRUCT/ASK/DESCRIBE
> >   * Removing or limiting the extensions we've gained on the way and
> > are addressed in 1.1 (eg. accepting "AS var" as good, property paths
> > greatly reduce the need for our property functions,... other
> > extensions like FTS must stay of course)
> >
> > - Double checking ontology migration code, ensure it can handle weird
> > ontology changes more or less elegantly.
>
> You will have a lot, a lot of fun with that code :-)

Already visited it briefly :P.

Kinda replying here to your other email, there will be indeed
situations of precision or data loss, as long as 1) what is supported
and what is not is properly documented, 2) we do our best to ensure
the resulting database represents the current ontology or 3) error out
and rollback the ongoing changes, I think should be fine.

I think this will be mainly useful for apps using private
databases/ontologies, if they are in control of the both the ontology
and the data, they can also look for ways to preserve or re-extract
what's interesting while changing their ontology.

> > - Library-fying tracker-store, and separating ontology for good, so
> > eg. an irc client wanting to store conversation logs privately can eg.
> > do:
>
> Yes! :) Want!

Cool :), I think this will be a win for apps considering their data
precious, not all data is equally disposable. I already feel shivers
each time I have to recommend tracker reset -r ...

Cheers,
  Carlos