Re: [Tracker] nfo:hasMediaStream max cardinality change

[Martyn Russell <martyn lanedo com>]

On 08/08/14 10:55, Philip Van Hoof wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> How to implement this would go like this:

Hi Philip,

Thanks for the email here, allow me to respond :)

> - - Detect the change with nao:lastModified and the existing
>    introspection ontology (tooling for this is available because of the
>    existing support)
> - - Store the pending change (tooling available)
> - - Make the new table for the multi-value property
> - - SELECT the column out of the class's table and INSERT the values into
>    the new table
> - - Change the introspection ontology
> - - DROP the column out of the class's table. I think SQLite doesn't
>    support this so you will have to recreate the table. There is tooling
>    available for this. (recreate as tablename_tmp with the new column,
>    select into from the old to the tablename_tmp, leaving out the old
>    column, drop tablename, rename tablename_tmp to tablename).
> - - Remove the pending change
>
> As original developer of the ontology change coping I can tell you
> that this is not an easy piece of work. I can of course assist whoever
> wants to try this.
>
> But without it I think we should revert the cardinality change for
> nfo:hasMediaStream. Perhaps add a nfo:hasStream or
> nfo:hasMultiMediaStream property or something instead and mark the old
> nfo:hasMediaStream as deprecated?

I think warning people is enough. I would prefer an upgrade path of 
course, but it's not always possible. Let me explain why:

- We don't break the database schema much and to be fair and this comes 
in the next set of stable releases.

- This isn't a highly regular or common property and should affect few 
people.

- The actual Nepomuk standards are showing this with no max cardinality, 
which is why I had no problem updating it to be in line with the 
standards here.

- Using codec without this change is quite useless, since most media 
don't use just one codec, hence the cardinality change.

- The property can't be deprecated because the new name we choose would 
differ from the official standard¹ and create possible confusion.

¹ http://www.semanticdesktop.org/ontologies/2007/03/22/nfo/#hasMediaStream

> Reason to revert is loss of data to people who are not using the
> journal (most and/or everyone of the embedded users) and having to

Actually, this entirely depends on the packager and distribution. The 
default is to enable the journal. I have been asked by the RedHat folks 
to disable it by default, and I was going to, but this makes me think 
it's wise not to.

I have no idea who enables or disables the journal in each case.

> rebuild the entire meta.db using the journal for everybody else.

I think putting a strong warning into the release note for 1.2 that a 
reindex may be required is the right way forward here.

We could bump the DB version, but we've not done that for years not with 
the current nao:lastModified approach we've had for so long. The DB 
version change would force a reindex.

> Latter is not the worst thing in the world, former is bad. Evil. Wrong.

Sometimes it's required though. For a while now, I've wanted some way to 
backup "user-specific" data in the DB, e.g. tags or things that can not 
be retrieved back from files (that's not always true for tags of 
course). This is why.

> Note that the disable-journal feature exists for privacy reasons too:
>
> Without it deleting triples will just mean adding a delete command to
> the journal. The original data remains in the journal. That also means
> that all data ever accumulated can be read from the journal. Also data
> that once got deleted.
>
> This is not acceptable for everybody, and they therefore should
> disable the journal.

So you think we should disable the journal by default?

> Another reason to disable the journal is a serious performance
> improvement, a lot less I/O usage, and a lot less disk space usage.

Sounds like you think we should :)

> So removing the disable-journal option is ... not an option (and the
> feature is being used by the N9, Jolla and a few other users. I think
> Pelagicore too for example -- or they should be).

Yea.

--
Regards,
Martyn

Founder & Director @ Lanedo GmbH.
http://www.linkedin.com/in/martynrussell