Beagle and Gaim Logs

[Richard Laager <rlaager wiktel com>]

I'm the Gaim developer currently "in charge" of the logging code. I have
a few questions about Beagle's searching of Gaim logs. Please bear with
me... I'm new to Beagle. :) It looks really nice so far!

1. Old Logs

Really old versions of Gaim (pre 0.70, I think) stored conversations in
what we now call "old logs". These files are flat files, which contain
all the conversations for a given buddy. They are named things like:
~/.gaim/logs/rlaager.log (for a buddy with the screenname "rlaager").

Beagle doesn't seem to index these. In the interest of completeness*,
I'd like to see these files be indexed, and I'm willing to help in
whatever way I can. I never used C# before, so I would need a little
guidance if I was to try to create a patch.

* Despite all the flaws in the old logging format and the major
advantages of the new logging format, some users want to continue to use
the old logging format. (I have no idea why they want this, but they
do...) There is a third-party plugin that enables this. Thus, supporting
this format in Beagle would be useful to this subset of users as well.

2. Log file naming

I changed the format of Gaim log files recently (first released with
2.0.0beta3). They now include timezone information. This was reported to
you in Bug #333950: http://bugzilla.gnome.org/show_bug.cgi?id=333950

First, let me say I'm sorry I didn't notify you about that. I took care
to notify the people who package Gaim for distros, since this broke
compatibility with Gaim 1.x, but I didn't think to notify the Beagle
project. (I wasn't using Beagle at the time, though I had heard of it.)
It simply slipped my mind, and I apologize. I will keep Beagle in mind
in the future.

Moving forward, it is my intention to name Gaim log files like this:

`date +%Y-%m-%d.%H%M%S%z%Z.EXT`

(where EXT can currently be .txt or .html ... other types may
theoretically be added in the future, but I don't currently anticipate
that happening)

However, I would like to reserve the ability to add additional data
before .EXT. So, basically, a filename such as this would be valid:

2006-04-04.131115-0500CDT.rlaager-wants-to-make.life.difficult.html

The timezone abbreviation ends at the first period. So, I won't be
making filenames like this:

2006-04-04.131115-0500CDTWhere_does_the_tz_end_and_this_start.html

I haven't currently thought of any data I'd like to save, but I want to
be prepared for the future, so I don't need to break backwards
compatibility like I did with Gaim 1.5.0 -> Gaim 2.0.0beta3.

Gaim is coded to handle logs according to the I've just described and it
would be best if applications which worked with Gaim logs did so as
well. The easiest way to do this is to just read as much format as you
understand, stopping the TZ at the first period. Then, search for the
extension by finding the LAST period (which, in Gaim is done by finding
the end of the filename and looking backwards to find a period) and
reading the data from there to the end of the filename.

3. Timezones

You might wonder why on earth I added the timezone information, if there
was backwards compatibility breakage, etc.

We display log times like this:
$ date +%c
Sat 08 Apr 2006 04:23:11 PM CDT

I spent a couple months in another timezone. When I came back, all of
the logs were displayed as being CDT or CST. This was technically
incorrect and my obsessive nature drove me to find a solution.

Now that we save timezone information, we use the timezone abbreviation
in the date formatting and we use the GMT offset to sort things
appropriately. Thus, if I have a conversation in my timezone at 2:15
CDT, then jump across a timezone border (going backwards an hour) and
have another conversation (at, say, 1:20 MDT), the two conversations
will display like this:

1:20 MDT
2:15 CDT
older conversations display here, for example: 1:30 CDT

Previously, they would display as (assuming I'm still in MDT):
2:15 MDT (This really happened first, and in CDT.)
1:20 MDT
older conversations display here, for example: 1:30 MDT

If I then went back to CDT, all of those times would stay the same, but
MDT would change to CDT.

I don't know if this is of any relevance to Beagle, but I wanted to
point out the meaning of that data just in case.

4. Logger Plugins

Second, Gaim plugins can add additional logging methods, which may not
even be filesystem based. Is there any way that we could make those
searchable via Beagle?

There are two examples I'm concerned with. They are, in order of
importance (and likely ease of implementation):

The log reader plugin:

Over a year ago, when I first started playing with the Gaim source code,
I started on a log reader plugin to read logs from IM clients other than
Gaim. For example, I had some IM logs from MSN Messenger from back when
I dual-booted. When I choose to view logs in Gaim, it will automatically
load those into Gaim.

I haven't gotten this plugin release ready yet, but I intend to look
into this again real soon.

I'd like to see these other logs indexed by Beagle as well. One way to
do this would be to write a separate Beagle indexing module for them...

SQL Logging:

We get a fair number of questions about aggregating logs from multiple
Gaim instances. While doing synchronization of your ~/.gaim or
~/.gaim/logs directory is one way of doing this, there has been some
interest in having Gaim log to a SQL database. A Gaim plugin would add
an additional logger to accomplish this.

How could we make Beagle index logs in this case?

It would be nice to have some way that would just work with any new Gaim
logger that a user had. Perhaps we could make a Beagle indexer that
worked with Gaim somehow (either a plugin itself, or via DBUS maybe).
I'd love to hear people's thoughts on this.

5. This one isn't directly related to logs, but... Right now, if I
search for someone's name, I see a "Contact". If I double-click it,
nothing happens. Has anyone looked at using gaim-remote in Gaim 1.x
and/or the DBUS stuff in Gaim 2.0.0 to start a new conversation with
that user?

This seems to be pending the release of our DBUS support:
http://bugzilla.gnome.org/show_bug.cgi?id=158094

Alternatively, is this already handled if I had Galago installed?

I notice that there are fields for "Mobile Phone" and "Home Phone" when
I select a contact in Beagle. Does this mean Beagle is doing some sort
of Gaim/Evolution integration?

6. Searching from Gaim

Gaim currently has its own searching of logs, and it sucks majorly. I'd
love to integrate the Beagle search functionality... to reuse the
indexing that's already done by Beagle, etc. Any thoughts on that? How
hard would that be? Is it a good idea? How might I start?

The biggest problem I can think of is that Beagle doesn't support Gaim
logging plugins (as discussed above). However, it would be nice to use
the Beagle index when possible and then fallback to doing searching
ourselves.

Thanks,
Richard