Zeitgeist status update

[Mikkel Kamstrup Erlandsen <mikkel kamstrup gmail com>]

Hi,

With the 2.30 module deadline passed it seems appropriate that we give
a status report from the Zeitgeist team.

Since there have been a good deal of confusion about what Zeitgeist
is, and isn't, about I will try clear this up in this mail as well. I
will try to stay low on the buzz word factor and leave some of the
more exotic use cases out to avoid too much speculation.


    * Zeitgeist in 1 sentence

Zeitgeist is an event logging framework used to keep a log of user
activity in a structured way.


    * What new services do we provide for UIs and applications

Zeitgeist provides a DBus API to query and update the activity log.
Clients can query on time ranges, the acting applications, mimetypes,
and Nepomuk classifications of the subjects and events. Sorting can be
done on various criteria such as usage frequency and recent usage.

Concrete examples could be "Get me most used files of mimetypes x,y or
z between the months January till March"

One can also query for documents that are used in context with others.
As in "Which documents/websites are used with http://youtube.com
within the last week".

It is also possible for the applications to get notified when the log
is updated. This is for instance used by the Parental Control
application as well as the GNOME Activity Journal.


    * What Problems can we solve

The straight forward use case is as a GtkRecentManager on drugs.
Zeitgeist removes the need for each application to parse a big XML
file to retrieve recently used documents. It also removes the need to
ever truncate your usage history, our database format is compact and
can easily contain years of history. My estimation is that 1M log
entries will take up about 80mb (give or take 20mb).

Open up for a range of query capabilities that GtkRecentManager
doesn't provide. Instead of simply storing the most recent usage event
on a resource we store all usage events. This way we can not only
answer when the most recent use case was, but also account for the
entire usage history.

One use case that is already in the works is having the most used
resources within the last 3 weeks for an app in the context menu in a
window list. This is for example done in Docky.

Looking past just logging resource usage we will also start monitoring
window and document focus times. This opens up to a whole new world of
contextual relevancy that I wont elaborate on here. I am trying to
stick to the more down to earth aspects of Zeitgeist.


    * Which processes/daemons do we run

Zeitgeist itself is a single DBus daemon. Where the picture gets a
little more fuzzy is how we collect events. The long term goal is for
apps to submit events, maybe hooking directly into GtkRecentManager,
or in any case provide a very convenient way for apps to do this. Apps
like Pidgin or Empathy would probably need some plugin for logging
usage statistics of your contacts.

Right now we resort to less elegant ways of collecting events, like
running a separate daemon harvesting Firefox's history,
GtkRecentlyUsed's and other applications' history (this daemon is also
known as the datahub). The datahub is already on its way to becoming
redundant now that a Firefox extension is in the works (and one for
Epiphany already exist). It is our intent that the datahub should
eventually go away as application support becomes widespread, but it
may eventually still prove useful for usage together with online
service.


    * How resource hungry are we

Normal memory usage is around 5-10mb for the core Zeitgeist daemon.
The datahub process (and I repeat; we want to get rid of this) is
about 12mb.


    * What dependencies

Right now the daemon depends on SQLite, Python 2.5, python-gobject,
python-xdg, and python-dbus. For the datahub we additionally need
python-gconf and python-gtk2, but the datahub is optional.


    * Future plans

We have spend a lot of time planning and designing lately. When we
have a stable reference implementation of our design in Python we plan
to use that as a template for a C implementation. To be clear - the C
version will be log-format and API compatible with the Python version.

We plan to make good use of the upcoming Zeitgeist hackfest and should
have a 0.3 development release ready shortly after. If we are happy
about the 0.3 series we will rename it to 0.9 and go for a 1.0.

Regarding Gnome 3.0 I think we are in a situation much like Owen
Taylor recently outlined for Gnome Shell on the release-team mailing
list[1]. If we are desperate for Zeitgeist to be included in a Gnome
3.0 this March I believe it would be doable. It will require that we
really bust our backs and cut some corners, but it's doable.
Personally (not speaking for the Zeitgeist team here) I am not sure it
would be a very good idea for the same reasons Owen mention.


    * Relation to Tracker and Other Semantic Technologies

The very short version of this is that Tracker and Zeitgeist does not
depend on each other in any way. The catch however is that either one
becomes a whole lot more powerful when working together. To take an
example consider tagging. Zeitgeist is just a log so we don't manage
your tags, we are however fully equipped to understand events
concerning your tags. So you manage the tags via Tracker and track
their usage in Zeitgeist. The combined power enables one to reason
about what tags relate to resources in a temporal manner, even with
resources that are not tagged.

In the Zeitgeist world we call an application like Tracker a
Repository. Nepomuk or Desktop-CouchDB might work as other
Repositories. If there is some confusion in this area it is
understandable, since we do have some Repository-like features in our
0.2 series. This is however removed from the 0.3 series. It is still
undecided if we want to define a minimal Repostiory DBus API for
Zeitgeist and then ship a reference impl. of this API (which would run
in a separate process). Any full fledged Repository would be able own
the Repository service on the bus and Zeitgeist would not run its own.
But again let me stress that a Repository is not needed for the
Zeitgeist Log daemon to be useful.

Cheers,
Mikkel

[1]: See the "Time considerations" section on
http://mail.gnome.org/archives/release-team/2009-November/msg00019.html