Re: [Tracker] The quadruple, team conversations on IRC

[Martyn Russell <martyn lanedo com>]

On 29/07/09 13:53, Philip Van Hoof wrote:
> This is a mail describing what the team has been discussing on IRC, so
> that people who aren't on IRC can follow up too.

Hi :)

> o. We want to start having a quadruple alongside the decomposed sqlite
>     tables.
>
>     o. Such a quadruple will allow us to implement backup
>
>     o. Such a quadruple will allow us to implement named graph support

I think we need some real world examples here. At least from what I have 
seen with 0.6. the problems come when we want to store data like 
LastPlayed or Tags and from multiple locations. For example, if there 
are 2 media players or perhaps a file can be used in a video player AND 
a music player and they both want to store LastPlayed independently or 
have Tags uniquely set by themselves not shown between all players.

Are there any others good examples I have missed here?

>        o. Makes it possible for synchronization software to make somewhat
>           intelligent decisions based on for example origin of the triple
>        o. Allows us to store the origin of metadata (local, flickr,
>           facebook, etc)
>        o. Would make the KDE people interested in using Tracker for their
>           mobile targets (according to discussions with them at the
>           desktop summit conference earlier this month)
>        o. Avoids the embedded/non-embedded metadata issue when updating
>           indexed files as we'd know the origin for each statement

I am quite interested to know technically what changes this means? 
Another table? another database?

Does it just stop the "location" that wrote the data (like an 
application name)?

I heard Jürg mention that we would have one big table with links to 
decomposed tables, is that right? i.e. we don't have quadstore in all 
tables? Is this a specific implementation detail we get to choose or is 
this generally how quadstore is done?

Are there any alternatives to the implementation here we should consider?

/braindump

> o. The quadruple will store all metadata, also embedded metadata. This
>     will create an overhead of about 10% during storage. It wont give any
>     overhead for read queries.

This should still be faster than the 0.6 branch as far as I understand.

> o. We will check if we can win back some of that 10% (some ideas are
>     floating around)

Well, we still have to spend some time polishing master ;)

> o. If the embedded metadata can be left out of the quadruple then we
>     should try to achieve that.
>
> o. We wont try to remove the transaction from the decomposed tables yet.
>
> People can followup development of the quadruple in the "quad" branch on
> GNOME's git. This branch will likely soon be merged to "master".
>
> The branch contains the support for the quadruple and, implemented on
> top of that, support for backup. The branch doesn't yet implement named
> graphs (the origin column is left empty for now).
>
> Known concerns: Jamie's concern is a performance issue if we store both
> embedded and non-embedded data in the quadruple. We will investigate
> this performance issue and see if we can mitigate it.

Jürg, what is stopping us from moving to this?

--
Regards,
Martyn