tracker 1.11.0
- From: Carlos Garnacho <install-module master gnome org>
- To: FTP Releases <ftp-release-list gnome org>
- Subject: tracker 1.11.0
- Date: Mon, 21 Nov 2016 13:06:24 +0000 (UTC)
About Tracker
=============
Tracker is a semantic data storage for desktop and mobile devices.
Tracker uses W3C standards for RDF ontologies using Nepomuk with
SPARQL to query and update the data.
Tracker is a central repository of user information, that provides two
big benefits for the user; shared data between applications and
information which is relational to other information (for example:
mixing contacts with files, locations, activities and etc.).
ChangeLog
=========
2016-11-21 Carlos Garnacho <carlosg gnome org>
Release 1.11.0
libtracker-data: Wrap BIND argument with () in SQL
Constructing the query as "SELECT $bind * FROM (...)" may trigger
sqlite parser overflows if the BIND form turns out to be too complex,
Doing the query as "SELECT ($bind) * FROM (...)" seems to be more
friendly to the parser.
This error was seen in gnome-music search, where the BIND() argument
is something like:
BIND((IF(STRSTARTS(?title_lower, "the "), SUBSTR(?title_lower, 5),
IF(STRSTARTS(?title_lower, "a "), SUBSTR(?title_lower, 3),
IF(STRSTARTS(?title_lower, "an "), SUBSTR(?title_lower, 4),
fn:replace(fn:lower-case(?title_lower),
"^[ !\"#$%&'()*+,-./:;<=>?@[\\]^_`{|}~]+|[
!\"#$%&'()*+,-./:;<=>?@[\\]^_`{|}~]+$", "")
)
)
)
) AS ?title_collation)
Which, despite the fn:replace regex argument being passed as a host
parameter, seemed too much to sqlite.
libtracker-sparql: Distcheck fix
2016-11-21 Marek Cernocky <marek_cernocky conel cz>
Updated Czech translation
2016-11-20 Carlos Garnacho <carlosg gnome org>
libtracker-sparql: Use internal headers
That's not guaranteed to be there yet at compile time, so it
randomly breaks build.
libtracker-miner: Use TrackerNotifier on TrackerDecorator
Remove the GraphUpdated handler in favor of TrackerNotifier,
it's been made to be a good fit here, and the porting results
in a nice code removal.
Given that the RDF types list is readwrite in TrackerDecorator,
but construct-only in TrackerNotifier, this means we need to
cater for changes in the RDF types list, just by creating another
TrackerNotifier and dropping the older one.
https://bugzilla.gnome.org/show_bug.cgi?id=773028
libtracker-sparql: Add TrackerNotifier to subscribe to change notifications
This object abstracts GraphUpdated, and is now the recommended method
to receive notifications upon changes in Tracker data. After creation,
the caller may connect to the ::events signal. This signal packs all
events received in one go, so as to avoid signal emission overhead on
mass changes.
The TrackerNotifier behavior can be slightly modified through the
flags passed during instantiation. Eg. it can be requested to query
some data (urn, location) upfront, although the API is tracker:id centric
otherwise.
https://bugzilla.gnome.org/show_bug.cgi?id=773028
tracker: Add remote connection support to "tracker sparql" CLI tool
The -r/--remote switch can be used to specify the base url to be used
in queries.
https://bugzilla.gnome.org/show_bug.cgi?id=773031
libtracker-remote: Add support for application/sparql-results+xml
As specified in https://www.w3.org/TR/rdf-sparql-XMLres/. This cursor
implementation is able to read the XML expected under that content
type.
https://bugzilla.gnome.org/show_bug.cgi?id=773031
Add libtracker-remote
This is yet another libtracker-sparql backend to connect to remote
HTTP SPARQL endpoints. Connections are made explicitly through the
tracker_sparql_connection_remote_new() API call, passing a server
to connect to. This commit introduces support for
application/sparql-results+json as specified in
https://www.w3.org/TR/sparql11-results-json/. XML format will be
handled next.
Just readonly queries are supported, and provided there's no
authentication schemes.
https://bugzilla.gnome.org/show_bug.cgi?id=773031
tracker: Fix "tracker sql -f"
It was doing nothing, missing the actual execution of the query.
libtracker-fts: Ensure a sqlite3_stmt is finalized on FTS5 failure paths
It is an error condition if no FTS5 API is found, it's no excuse to leave
the sqlite3_stmt unfinalized though.
libtracker-data: Add explicit calls to grab/release an stmt
An stmt can just be grabbed once at a time, and will ref both
the stmt and its DB interface for as long as the stmt is grabbed.
After releasing both refs will be dropped (and stmt_is_used set
back to FALSE), so it will be ensured that a database connection
lives as long as there are active cursors grabbing an statement.
libtracker-data: Remove global interface
We no longer need to restrict to a single TrackerDBInterface for the
direct connection use. This also allows to maximize the throughput
in clients performing multiple simultaneous async queries, as we're
going from using a single TrackerDBInterface to satisfy all requests
(with the locking overhead this involves) to using multiple database
connections that can run different concurrent queries each.
This (and the last commits) translates to faster concurrent access,
which evolves close to linearly under much higher loads, instead of
exponentially. It is expected though that O(n^m) happens again if
the number of operations vastly exceed the number of interfaces
serving requests, but that's much farther.
As an example, the numbers when running multiple concurrent queries
for "SELECT ?u WHERE { ?u a rdfs:Resource }" (returning 12897
elements each in this computer) progress like this on an otherwise
idle session:
Simul. Queries With Without
5 0,399s 0,437s
10 0,700s 0,842s
50 3,621s 4,285s
100 7,315s 8,956s
500 36,012s 1m 24,952s
1000 1m 16,355s 8m 33,880s
So as expected the numbers really shine when the thread contention
is most noticeable, and quite close to linearly (eg. in the "with"
column, every row is roughly 10x higher than the one corresponding
to simultaneous queries/10)
libtracker-data: Remove TrackerDBManager locking functions
The TrackerDBManager mutex is used nowhere now, it's safe to
remove.
libtracker-direct: Replace TrackerDBManager lock usage with private one
We need to lock here not because of the TrackerDBInterface, but because
of other misc things needed at the time of constructing the SQL query
(like TrackerOntologies, which constructs data on demand from the gvdb,
this can't work across multiple threads).
But we don't need to use the TrackerDBManager lock, it can be just
a private one, and we remove another usage of this global lock.
libtracker-data: Protect stmt creation with the TrackerDBInterface mutex
Creating the sqlite3_stmt counts as "accessing the database handle", so
make it sure they're also protected by the TrackerDBInterface mutex.
libtracker-data: Use internal mutex in TrackerDBInterfaceSqlite
There is no reason to use a global mutex for multiple TrackerDBInterfaces,
the threading restriction in sqlite is "no multiple threads using the same
sqlite3* at the same time" (being cursor included "using" there). This
means we must restrict simultaneous threaded access to a single
TrackerDBInterface, but multiple threads are free to use a
TrackerDBInterface each without further interference.
So, use a per-TrackerDBInterface mutex instead of a global one, if multiple
cursors are created on the same interface, they'll backreference to it, so
will be blocking on the same mutex.
libtracker-data: Remove threadsafe argument
This was only ever used to make libtracker-direct use protected
cursors, which is now set by the TRACKER_DB_MANAGER_ENABLE_MUTEXES
flag, so avoid passing this all along.
libtracker-data: Add ENABLE_MUTEXES TrackerDBManager initialization flag
libtracker-direct: Drop mutex around TrackerDBManager initialization
It doesn't need to care about this anymore, TrackerDBManager uses an
internal mutex.
libtracker-data: Use internal mutex for TrackerDBManager (de)init
There's no reason to handle locking externally, and this is not a frequent
enough operation to care about thread contention times in eg.
tracker-store, so just add an internal mutex protecting initialization.
libtracker-data: Pass "readonly" argument to TrackerDBInterface constructor
Instead of having two almost identical call stacks to obtain rw/ro
interfaces, put it all together in one with a readonly argument.
libtracker-data: Rename confusing TrackerDBStatement field
Now that the TrackerDBStatement is a GInitiallyUnowned, the
stmt_is_sunk boolean field is somewhat confusing. It is used
to keep track of the stmts being currently in use by a
TrackerDBCursor, renaming it to stmt_is_used seems more obvious.
libtracker-data: Make TrackerDBStatement a GInitiallyUnowned
We actually deal with it as if it were (cached stmts have one ref
owned by the cache, and one by the caller. non-cached has just one
reference).
So make TrackerDBStatement actually an unowned type, and
g_object_ref_sink() in both places, if the stmt is cached the
ref will be sunk by the cache, and an extra ref added for the
caller, and if non-cached the only sunk reference will be the
callers'.
libtracker-data: Refactor LRU sqlite3_stmt management
There's 3 independent operations we do on it: lookup, insert, and
move to top. These have been moved to separate functions (although
the code is mostly as-is).
libtracker-data: Refactor sqlite3_stmt preparation
Add a helper function, and use it in the two places preparing
the statement and checking for errors.
libtracker-data: Cleanup function initialization
We create enough functions to make it worth putting those in
an array, so they can be nicely categorized too.
libtracker-data: Remove unused field
libtracker-data: Delete collation/locale change mechanisms
Those are entirely unused, we support no live locale nor collation
changes. The n_active_cursors field is now unused, although it's
been left as it's useful to keep track of the amount of active
statements/cursors in an interface.
libtracker-data: Remove sqlite3_stmt argument
It's already contained in the TrackerDBStatement for all callers,
and it makes no sense passing other sqlite3_stmt than the one
there.
libtracker-data: Indentation fix
tracker-extract: Drop string constness
This string is actually copied and freed, so definitely not const.
tracker-extract: Remove unused variable
tracker: Avoid deprecated GSettings API
And use GSettingsSchema/GSettingsSchemaSource. This code is still
disgusting.
libtracker-sparql-backend: Make methods possibly throw GLib.Error
It was missing according to the valac warnings.
tracker-extract: Deep copy GstToc
According to gst_discoverer_info_get_toc(), it must be deep copied
if you want to use it past the discoverer info lifetime.
libtracker-extract: Pass the right struct pointer
Confusion between TrackerExtractInfo and TrackerDecoratorInfo, might
induce crashes.
libtracker-miner: Handle extra GFileMonitorEvent values
We don't use G_FILE_MONITOR_WATCH_MOVES as it doesn't add much to us,
so make the extra derived event types a no-op.
libtracker-miner: Drop wimax handling in TrackerMinerOnline
NM 1.2 deprecated this type as it doesn't support wimax anymore, and
I don't even know what it is. So it seems something we can do without,
falling back to the "unknown" network type is just as safe.
libtracker-data: Explicitly cast string to char*
This fixes a valac warning.
libtracker-data: Use right constness in some internal calls
The string array is fully const (i.e. we deep copy it), so make
these functions have the right constness.
libtracker-data: Rename internal variable
We are actually requesting the context to resolve variables to the
binding sql expression, so rename to a more apt need_binding_expression.
libtracker-data: Handle BIND while in group graph pattern
Currently SQL construction fails while using the BIND form inside
a group graph pattern, eg. after OPTIONALs. When in that stage,
the OPTIONALs will be already resolved to SQL in the "FROM ..."
clause as nested SELECTs.
As such the generated SQL is wrong, make it handle this case
by wrapping the nested SELECTs under "SELECT $bind_value, * FROM (...)"
so the value is made available to the wrapping SELECT. The older
code still applies when building simple "{ ... . BIND (...) }" clauses.
https://bugzilla.gnome.org/show_bug.cgi?id=774251
libtracker-data: Return NULL SQL expression if the binding is not yet setup
Recent vala changed property setters so they compare with the previous
value before emitting anything, this triggers the property getter called
at a time when we just don't have a DataTable to construct the SQL
expression (and it's being set anyway).
So ensure the binding is in the right state before trying to construct a
SQL expression for it.
tracker-needle: Fix build
It seems recent vala got pickier with passing unsigned to a %d
parameter. It's still right nonetheless, and changing the string
is a needless translation change, so just cast to int.
2016-11-15 Marinus Schraal <mschraal src gnome org>
Use nie:title instead of nmm:albumTitle
nmm:albumTitle is deprecated
https://bugzilla.gnome.org/show_bug.cgi?id=773697
tracker-extract: Use date as album uri identifier
Use the album creation date as part of the album uri identifier if
available. This should make the separation of similar named albums even
better. Also port mp3 & gstreamer to use the resource helper functon for
exctracting album disc data.
https://bugzilla.gnome.org/show_bug.cgi?id=773697
tracker-extract: Use albumartist if available
Use albumartist in libav/vorbis/flac (resource-helpers), gstreamer & mp3
extractors if available to be part of the album(-disc) uri. This makes
the uri more unique and tracker better at distinguishing separate albums
and it makes all the extractors behave the same.
https://bugzilla.gnome.org/show_bug.cgi?id=773697
tracker-extract: Don't mix artist and albumartist
The 'album artist' is not the same thing as the 'performer' or the
'artist' metadata. In some extractors these tags however got mixed to
provide fallback values for certain tags. In the long run this is not a
viable approach: the tracker information should be a true representation
of what is in the tags.
https://bugzilla.gnome.org/show_bug.cgi?id=773697
2016-11-11 Marinus Schraal <mschraal src gnome org>
tracker-extract: Fix flac albumartistsort tag parsing
Albumartist tag can have a sort variant too.
https://bugzilla.gnome.org/show_bug.cgi?id=772596
2016-11-08 Marinus Schraal <mschraal src gnome org>
tracker-extract: Revert libflac as default rule
Keep gstreamer as the primary extractor as long as the libflac extractor
is not on-par with it.
2016-11-07 Marinus Schraal <mschraal src gnome org>
tracker-extract: Pass the flac metadata
Oversight from the move to TrackerResource, the flac extractor was not
passing the metadata on for further processing.
https://bugzilla.gnome.org/show_bug.cgi?id=774062
tracker-extract: Albumartist may be NULL in flac
album_artist is a TrackerResource, but may be NULL: do not just unref
it.
https://bugzilla.gnome.org/show_bug.cgi?id=774062
2016-11-04 Fabio Tomat <f t public gmail com>
Update Friulian translation
2016-11-03 Carlos Garnacho <carlosg gnome org>
libtracker-data: Handle overflows on libicu-based normalization
We allocate by default double the string size, which is a pretty
generous allotment most usually. If that is not enough for storing
the normalized string, just reallocate the buffer with the given
expected size and try again.
Also, refactor string normalization to a separate function, so
it's shared by both tracker:normalize and tracker:unaccent.
https://bugzilla.gnome.org/show_bug.cgi?id=769982
libtracker-miner: Start TrackerDecorator queries on TrackerMiner::started
This should not be done in the initable vfunc, as the miner should remain
idle until tracker_miner_start() is called on it.
This fixes a possible race condition in tracker-extract, where this query
is already in flight at the time TrackerPersistence handles blacklisting
of files that caused earlier crashes of tracker-extract. So the query might
contain the file that's being blacklisted right away. tracker_miner_start()
is called after TrackerPersistence did the blacklisting job, so the first
query will get the up-to-date info.
2016-11-03 Marinus Schraal <mschraal src gnome org>
tracker-extract: Fix album disc uri in gstreamer
It should build the album disc uri with the album_artist_name.
https://bugzilla.gnome.org/show_bug.cgi?id=773607
2016-11-02 Daniel Mustieles <daniel mustieles gmail com>
Update Spanish translation
2016-10-30 Aurimas Černius <aurisc4 gmail com>
Updated Lithuanian translaton
2016-10-29 Mario Blättermann <mario blaettermann gmail com>
Update German translation
2016-10-28 Marek Černocký <marek manet cz>
Updated Czech translation
2016-10-26 Marinus Schraal <mschraal src gnome org>
tracker-extract: Only use CUE info as last resort in gstreamer
CUE data was overriding file tags in the gstreamer extractor, however
CUE data is notoriously unreliable. Instead, only use the CUE data if we
have nothing else.
https://bugzilla.gnome.org/show_bug.cgi?id=773524
2016-10-23 Anders Jonsson <anders jonsson norsjovallen se>
Update Swedish translation
2016-10-23 Gábor Kelemen <kelemeng openscope org>
Update Hungarian translation
2016-10-19 Carlos Garnacho <carlosg gnome org>
libtracker-data: Propagate "locale mismatch" error from locale_changed()
And set the differing locales in the error message. Errors here are
mainly due to 1) inconsistent locales, specifically the client running
with a different locale than tracker-store, and 2) faulty apps that don't
call setlocale() as appropriate.
We already used to warn when locales differ, but the message wasn't
that useful. Including the differing locales helps narrow down the
issue. This error is mainly visible in the libtracker-direct backend,
tracker-store shall handle locale changes on (re)start.
configure.ac: Bump version to 1.11.0
Not much worth releasing yet, but having tracker-1.10 have a higher
soname than master is confusing.
2016-10-18 Piotr Drąg <piotrdrag gmail com>
Update Polish translation
2016-10-18 Dz Chen <wsxy162 gmail com>
Update zh_CN translation
2016-10-16 Carlos Garnacho <carlosg gnome org>
libtracker-sparql: Avoid C++ keyword in variable
"namespace" is a C++ keyword, better to avoid in public headers,
spotted by Christoph Cullmann <cullmann kde org>.
https://bugzilla.gnome.org/show_bug.cgi?id=772979
Download
========
https://download.gnome.org/sources/tracker/1.11/tracker-1.11.0.tar.xz (4.81M)
sha256sum: 342df7df1927a65d957fb244ac84f776a1cea794f366c25669e6fa4a735e2125
[
Date Prev][
Date Next] [
Thread Prev][
Thread Next]
[
Thread Index]
[
Date Index]
[
Author Index]
