[Tracker] Fwd: tracker 1.11.0



It seems ftpadmin didn't pick the news this time, just for completeness:

NEW in 1.11.0 - 2016-11-21
==========================

  * libtracker-sparql: Added TrackerNotifier, helper object to receive
    notifications of changes to the Tracker database. All users of the
    GraphUpdated DBus signal are recommended to switch to it.
  * libtracker-sparql: Added client-side support for HTTP SPARQL
    endpoints.
  * libtracker-direct: Much reduced mutex contention during threaded/async
    queries on the direct access backend.
  * libtracker-sparql: Using BIND() after OPTIONAL{} now works properly
  * tracker-extract: Many improvements to music extraction, better labeling
    of albums, nmm:albumArtist metadata is more faithful to the file
    metadata.
  * libtracker-data: Fixed possible overflows in tracker:normalize/unaccent
  * Other fixes and cleanups.

Translations: ca, cs, de, es, eu, hu, lt, fur, pl, sv, zh_CN


---------- Forwarded message ----------
From: Carlos Garnacho <install-module master gnome org>
Date: Mon, Nov 21, 2016 at 2:06 PM
Subject: tracker 1.11.0
To: FTP Releases <ftp-release-list gnome org>


About Tracker
=============

Tracker is a semantic data storage for desktop and mobile devices.
Tracker uses W3C standards for RDF ontologies using Nepomuk with
SPARQL to query and update the data.

Tracker is a central repository of user information, that provides two
big benefits for the user; shared data between applications and
information which is relational to other information (for example:
mixing contacts with files, locations, activities and etc.).

ChangeLog
=========

2016-11-21  Carlos Garnacho  <carlosg gnome org>

        Release 1.11.0

        libtracker-data: Wrap BIND argument with () in SQL
        Constructing the query as "SELECT $bind * FROM (...)" may trigger
        sqlite parser overflows if the BIND form turns out to be too complex,
        Doing the query as "SELECT ($bind) * FROM (...)" seems to be more
        friendly to the parser.

        This error was seen in gnome-music search, where the BIND() argument
        is something like:

        BIND((IF(STRSTARTS(?title_lower, "the "), SUBSTR(?title_lower, 5),
                 IF(STRSTARTS(?title_lower, "a "), SUBSTR(?title_lower, 3),
                    IF(STRSTARTS(?title_lower, "an "), SUBSTR(?title_lower, 4),
                       fn:replace(fn:lower-case(?title_lower),
                                  "^[
!\"#$%&'()*+,-./:;<=>?@[\\]^_`{|}~]+|[
!\"#$%&'()*+,-./:;<=>?@[\\]^_`{|}~]+$", "")
                      )
                   )
                )
             ) AS ?title_collation)

        Which, despite the fn:replace regex argument being passed as a host
        parameter, seemed too much to sqlite.

        libtracker-sparql: Distcheck fix

2016-11-21  Marek Cernocky  <marek_cernocky conel cz>

        Updated Czech translation

2016-11-20  Carlos Garnacho  <carlosg gnome org>

        libtracker-sparql: Use internal headers
        That's not guaranteed to be there yet at compile time, so it
        randomly breaks build.

        libtracker-miner: Use TrackerNotifier on TrackerDecorator
        Remove the GraphUpdated handler in favor of TrackerNotifier,
        it's been made to be a good fit here, and the porting results
        in a nice code removal.

        Given that the RDF types list is readwrite in TrackerDecorator,
        but construct-only in TrackerNotifier, this means we need to
        cater for changes in the RDF types list, just by creating another
        TrackerNotifier and dropping the older one.

        https://bugzilla.gnome.org/show_bug.cgi?id=773028

        libtracker-sparql: Add TrackerNotifier to subscribe to change
notifications
        This object abstracts GraphUpdated, and is now the recommended method
        to receive notifications upon changes in Tracker data. After creation,
        the caller may connect to the ::events signal. This signal packs all
        events received in one go, so as to avoid signal emission overhead on
        mass changes.

        The TrackerNotifier behavior can be slightly modified through the
        flags passed during instantiation. Eg. it can be requested to query
        some data (urn, location) upfront, although the API is
tracker:id centric
        otherwise.

        https://bugzilla.gnome.org/show_bug.cgi?id=773028

        tracker: Add remote connection support to "tracker sparql" CLI tool
        The -r/--remote switch can be used to specify the base url to be used
        in queries.

        https://bugzilla.gnome.org/show_bug.cgi?id=773031

        libtracker-remote: Add support for application/sparql-results+xml
        As specified in https://www.w3.org/TR/rdf-sparql-XMLres/. This cursor
        implementation is able to read the XML expected under that content
        type.

        https://bugzilla.gnome.org/show_bug.cgi?id=773031

        Add libtracker-remote
        This is yet another libtracker-sparql backend to connect to remote
        HTTP SPARQL endpoints. Connections are made explicitly through the
        tracker_sparql_connection_remote_new() API call, passing a server
        to connect to. This commit introduces support for
        application/sparql-results+json as specified in
        https://www.w3.org/TR/sparql11-results-json/. XML format will be
        handled next.

        Just readonly queries are supported, and provided there's no
        authentication schemes.

        https://bugzilla.gnome.org/show_bug.cgi?id=773031

        tracker: Fix "tracker sql -f"
        It was doing nothing, missing the actual execution of the query.

        libtracker-fts: Ensure a sqlite3_stmt is finalized on FTS5 failure paths
        It is an error condition if no FTS5 API is found, it's no
excuse to leave
        the sqlite3_stmt unfinalized though.

        libtracker-data: Add explicit calls to grab/release an stmt
        An stmt can just be grabbed once at a time, and will ref both
        the stmt and its DB interface for as long as the stmt is grabbed.
        After releasing both refs will be dropped (and stmt_is_used set
        back to FALSE), so it will be ensured that a database connection
        lives as long as there are active cursors grabbing an statement.

        libtracker-data: Remove global interface
        We no longer need to restrict to a single TrackerDBInterface for the
        direct connection use. This also allows to maximize the throughput
        in clients performing multiple simultaneous async queries, as we're
        going from using a single TrackerDBInterface to satisfy all requests
        (with the locking overhead this involves) to using multiple database
        connections that can run different concurrent queries each.

        This (and the last commits) translates to faster concurrent access,
        which evolves close to linearly under much higher loads, instead of
        exponentially. It is expected though that O(n^m) happens again if
        the number of operations vastly exceed the number of interfaces
        serving requests, but that's much farther.

        As an example, the numbers when running multiple concurrent queries
        for "SELECT ?u WHERE { ?u a rdfs:Resource }" (returning 12897
        elements each in this computer) progress like this on an otherwise
        idle session:

        Simul. Queries  With            Without
        5                   0,399s          0,437s
        10                  0,700s          0,842s
        50                  3,621s          4,285s
        100                 7,315s          8,956s
        500                36,012s      1m 24,952s
        1000            1m 16,355s      8m 33,880s

        So as expected the numbers really shine when the thread contention
        is most noticeable, and quite close to linearly (eg. in the "with"
        column, every row is roughly 10x higher than the one corresponding
        to simultaneous queries/10)

        libtracker-data: Remove TrackerDBManager locking functions
        The TrackerDBManager mutex is used nowhere now, it's safe to
        remove.

        libtracker-direct: Replace TrackerDBManager lock usage with private one
        We need to lock here not because of the TrackerDBInterface, but because
        of other misc things needed at the time of constructing the SQL query
        (like TrackerOntologies, which constructs data on demand from the gvdb,
        this can't work across multiple threads).

        But we don't need to use the TrackerDBManager lock, it can be just
        a private one, and we remove another usage of this global lock.

        libtracker-data: Protect stmt creation with the TrackerDBInterface mutex
        Creating the sqlite3_stmt counts as "accessing the database handle", so
        make it sure they're also protected by the TrackerDBInterface mutex.

        libtracker-data: Use internal mutex in TrackerDBInterfaceSqlite
        There is no reason to use a global mutex for multiple
TrackerDBInterfaces,
        the threading restriction in sqlite is "no multiple threads
using the same
        sqlite3* at the same time" (being cursor included "using" there). This
        means we must restrict simultaneous threaded access to a single
        TrackerDBInterface, but multiple threads are free to use a
        TrackerDBInterface each without further interference.

        So, use a per-TrackerDBInterface mutex instead of a global
one, if multiple
        cursors are created on the same interface, they'll
backreference to it, so
        will be blocking on the same mutex.

        libtracker-data: Remove threadsafe argument
        This was only ever used to make libtracker-direct use protected
        cursors, which is now set by the TRACKER_DB_MANAGER_ENABLE_MUTEXES
        flag, so avoid passing this all along.

        libtracker-data: Add ENABLE_MUTEXES TrackerDBManager initialization flag

        libtracker-direct: Drop mutex around TrackerDBManager initialization
        It doesn't need to care about this anymore, TrackerDBManager uses an
        internal mutex.

        libtracker-data: Use internal mutex for TrackerDBManager (de)init
        There's no reason to handle locking externally, and this is
not a frequent
        enough operation to care about thread contention times in eg.
        tracker-store, so just add an internal mutex protecting initialization.

        libtracker-data: Pass "readonly" argument to
TrackerDBInterface constructor
        Instead of having two almost identical call stacks to obtain rw/ro
        interfaces, put it all together in one with a readonly argument.

        libtracker-data: Rename confusing TrackerDBStatement field
        Now that the TrackerDBStatement is a GInitiallyUnowned, the
        stmt_is_sunk boolean field is somewhat confusing. It is used
        to keep track of the stmts being currently in use by a
        TrackerDBCursor, renaming it to stmt_is_used seems more obvious.

        libtracker-data: Make TrackerDBStatement a GInitiallyUnowned
        We actually deal with it as if it were (cached stmts have one ref
        owned by the cache, and one by the caller. non-cached has just one
        reference).

        So make TrackerDBStatement actually an unowned type, and
        g_object_ref_sink() in both places, if the stmt is cached the
        ref will be sunk by the cache, and an extra ref added for the
        caller, and if non-cached the only sunk reference will be the
        callers'.

        libtracker-data: Refactor LRU sqlite3_stmt management
        There's 3 independent operations we do on it: lookup, insert, and
        move to top. These have been moved to separate functions (although
        the code is mostly as-is).

        libtracker-data: Refactor sqlite3_stmt preparation
        Add a helper function, and use it in the two places preparing
        the statement and checking for errors.

        libtracker-data: Cleanup function initialization
        We create enough functions to make it worth putting those in
        an array, so they can be nicely categorized too.

        libtracker-data: Remove unused field

        libtracker-data: Delete collation/locale change mechanisms
        Those are entirely unused, we support no live locale nor collation
        changes. The n_active_cursors field is now unused, although it's
        been left as it's useful to keep track of the amount of active
        statements/cursors in an interface.

        libtracker-data: Remove sqlite3_stmt argument
        It's already contained in the TrackerDBStatement for all callers,
        and it makes no sense passing other sqlite3_stmt than the one
        there.

        libtracker-data: Indentation fix

        tracker-extract: Drop string constness
        This string is actually copied and freed, so definitely not const.

        tracker-extract: Remove unused variable

        tracker: Avoid deprecated GSettings API
        And use GSettingsSchema/GSettingsSchemaSource. This code is still
        disgusting.

        libtracker-sparql-backend: Make methods possibly throw GLib.Error
        It was missing according to the valac warnings.

        tracker-extract: Deep copy GstToc
        According to gst_discoverer_info_get_toc(), it must be deep copied
        if you want to use it past the discoverer info lifetime.

        libtracker-extract: Pass the right struct pointer
        Confusion between TrackerExtractInfo and TrackerDecoratorInfo, might
        induce crashes.

        libtracker-miner: Handle extra GFileMonitorEvent values
        We don't use G_FILE_MONITOR_WATCH_MOVES as it doesn't add much to us,
        so make the extra derived event types a no-op.

        libtracker-miner: Drop wimax handling in TrackerMinerOnline
        NM 1.2 deprecated this type as it doesn't support wimax anymore, and
        I don't even know what it is. So it seems something we can do without,
        falling back to the "unknown" network type is just as safe.

        libtracker-data: Explicitly cast string to char*
        This fixes a valac warning.

        libtracker-data: Use right constness in some internal calls
        The string array is fully const (i.e. we deep copy it), so make
        these functions have the right constness.

        libtracker-data: Rename internal variable
        We are actually requesting the context to resolve variables to the
        binding sql expression, so rename to a more apt need_binding_expression.

        libtracker-data: Handle BIND while in group graph pattern
        Currently SQL construction fails while using the BIND form inside
        a group graph pattern, eg. after OPTIONALs. When in that stage,
        the OPTIONALs will be already resolved to SQL in the "FROM ..."
        clause as nested SELECTs.

        As such the generated SQL is wrong, make it handle this case
        by wrapping the nested SELECTs under "SELECT $bind_value, * FROM (...)"
        so the value is made available to the wrapping SELECT. The older
        code still applies when building simple "{ ... . BIND (...) }" clauses.

        https://bugzilla.gnome.org/show_bug.cgi?id=774251

        libtracker-data: Return NULL SQL expression if the binding is
not yet setup
        Recent vala changed property setters so they compare with the previous
        value before emitting anything, this triggers the property getter called
        at a time when we just don't have a DataTable to construct the SQL
        expression (and it's being set anyway).

        So ensure the binding is in the right state before trying to construct a
        SQL expression for it.

        tracker-needle: Fix build
        It seems recent vala got pickier with passing unsigned to a %d
        parameter. It's still right nonetheless, and changing the string
        is a needless translation change, so just cast to int.

2016-11-15  Marinus Schraal  <mschraal src gnome org>

        Use nie:title instead of nmm:albumTitle
        nmm:albumTitle is deprecated

        https://bugzilla.gnome.org/show_bug.cgi?id=773697

        tracker-extract: Use date as album uri identifier
        Use the album creation date as part of the album uri identifier if
        available. This should make the separation of similar named albums even
        better. Also port mp3 & gstreamer to use the resource helper functon for
        exctracting album disc data.

        https://bugzilla.gnome.org/show_bug.cgi?id=773697

        tracker-extract: Use albumartist if available
        Use albumartist in libav/vorbis/flac (resource-helpers), gstreamer & mp3
        extractors if available to be part of the album(-disc) uri. This makes
        the uri more unique and tracker better at distinguishing separate albums
        and it makes all the extractors behave the same.

        https://bugzilla.gnome.org/show_bug.cgi?id=773697

        tracker-extract: Don't mix artist and albumartist
        The 'album artist' is not the same thing as the 'performer' or the
        'artist' metadata. In some extractors these tags however got mixed to
        provide fallback values for certain tags. In the long run this is not a
        viable approach: the tracker information should be a true representation
        of what is in the tags.

        https://bugzilla.gnome.org/show_bug.cgi?id=773697

2016-11-11  Marinus Schraal  <mschraal src gnome org>

        tracker-extract: Fix flac albumartistsort tag parsing
        Albumartist tag can have a sort variant too.

        https://bugzilla.gnome.org/show_bug.cgi?id=772596

2016-11-08  Marinus Schraal  <mschraal src gnome org>

        tracker-extract: Revert libflac as default rule
        Keep gstreamer as the primary extractor as long as the libflac extractor
        is not on-par with it.

2016-11-07  Marinus Schraal  <mschraal src gnome org>

        tracker-extract: Pass the flac metadata
        Oversight from the move to TrackerResource, the flac extractor was not
        passing the metadata on for further processing.

        https://bugzilla.gnome.org/show_bug.cgi?id=774062

        tracker-extract: Albumartist may be NULL in flac
        album_artist is a TrackerResource, but may be NULL: do not just unref
        it.

        https://bugzilla.gnome.org/show_bug.cgi?id=774062

2016-11-04  Fabio Tomat  <f t public gmail com>

        Update Friulian translation

2016-11-03  Carlos Garnacho  <carlosg gnome org>

        libtracker-data: Handle overflows on libicu-based normalization
        We allocate by default double the string size, which is a pretty
        generous allotment most usually. If that is not enough for storing
        the normalized string, just reallocate the buffer with the given
        expected size and try again.

        Also, refactor string normalization to a separate function, so
        it's shared by both tracker:normalize and tracker:unaccent.

        https://bugzilla.gnome.org/show_bug.cgi?id=769982

        libtracker-miner: Start TrackerDecorator queries on
TrackerMiner::started
        This should not be done in the initable vfunc, as the miner
should remain
        idle until tracker_miner_start() is called on it.

        This fixes a possible race condition in tracker-extract, where
this query
        is already in flight at the time TrackerPersistence handles blacklisting
        of files that caused earlier crashes of tracker-extract. So
the query might
        contain the file that's being blacklisted right away.
tracker_miner_start()
        is called after TrackerPersistence did the blacklisting job,
so the first
        query will get the up-to-date info.

2016-11-03  Marinus Schraal  <mschraal src gnome org>

        tracker-extract: Fix album disc uri in gstreamer
        It should build the album disc uri with the album_artist_name.

        https://bugzilla.gnome.org/show_bug.cgi?id=773607

2016-11-02  Daniel Mustieles  <daniel mustieles gmail com>

        Update Spanish translation

2016-10-30  Aurimas Černius  <aurisc4 gmail com>

        Updated Lithuanian translaton

2016-10-29  Mario Blättermann  <mario blaettermann gmail com>

        Update German translation

2016-10-28  Marek Černocký  <marek manet cz>

        Updated Czech translation

2016-10-26  Marinus Schraal  <mschraal src gnome org>

        tracker-extract: Only use CUE info as last resort in gstreamer
        CUE data was overriding file tags in the gstreamer extractor, however
        CUE data is notoriously unreliable. Instead, only use the CUE data if we
        have nothing else.

        https://bugzilla.gnome.org/show_bug.cgi?id=773524

2016-10-23  Anders Jonsson  <anders jonsson norsjovallen se>

        Update Swedish translation

2016-10-23  Gábor Kelemen  <kelemeng openscope org>

        Update Hungarian translation

2016-10-19  Carlos Garnacho  <carlosg gnome org>

        libtracker-data: Propagate "locale mismatch" error from locale_changed()
        And set the differing locales in the error message. Errors here are
        mainly due to 1) inconsistent locales, specifically the client running
        with a different locale than tracker-store, and 2) faulty apps
that don't
        call setlocale() as appropriate.

        We already used to warn when locales differ, but the message wasn't
        that useful. Including the differing locales helps narrow down the
        issue. This error is mainly visible in the libtracker-direct backend,
        tracker-store shall handle locale changes on (re)start.

        configure.ac: Bump version to 1.11.0
        Not much worth releasing yet, but having tracker-1.10 have a higher
        soname than master is confusing.

2016-10-18  Piotr Drąg  <piotrdrag gmail com>

        Update Polish translation

2016-10-18  Dz Chen  <wsxy162 gmail com>

        Update zh_CN translation

2016-10-16  Carlos Garnacho  <carlosg gnome org>

        libtracker-sparql: Avoid C++ keyword in variable
        "namespace" is a C++ keyword, better to avoid in public headers,
        spotted by Christoph Cullmann <cullmann kde org>.

        https://bugzilla.gnome.org/show_bug.cgi?id=772979


Download
========
https://download.gnome.org/sources/tracker/1.11/tracker-1.11.0.tar.xz (4.81M)
  sha256sum: 342df7df1927a65d957fb244ac84f776a1cea794f366c25669e6fa4a735e2125


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]