Re: [Tracker] Releasing Tracker 0.6.90

[Martyn Russell <martyn imendio com>]

Michael Biebl wrote:
> 2009/2/5 Martyn Russell <martyn imendio com>:
> > Jamie McCracken wrote:
> > > On Thu, 2009-02-05 at 16:26 +0000, Martyn Russell wrote:
> > > > Jamie McCracken wrote:
> > > > > we are not deleting from qdbm! If you have changed stuff to do this then
> > > > > please revert. O.6.6 never deleted from qdbm and indeed never should as
> > > > > its prohibitively expensive
> > > > Carlos is working on a fix. If we fix this and get it done in time would
> > > > you be happy enough to release tomorrow?
> > > I would like to just quickly check things once you say you are ready -
> > > if the fix can be done today then i will look at it tonight and give you
> > > the go ahead for tomorrow if its ok
> > >
> > > I promise not to delay things further unless I find a showstopper
> > OK, I think it will have to wait until tomorrow (the fix) so if you look at
> > it tomorrow or over the weekend, we can release Monday.
>
> Martyn has been working hard on re-doing  tracker-extract.
> After some initial compilation troubles, I got r2900 up and running
> (more or less):
> My kern.log is literally filled with those entries
> Feb  7 00:43:42 pluto kernel: [52319.901584] tracker-extract[31461]:
> segfault at bf622fec ip b6f6ebec sp bf622ff0 error 6 in
> libextract-gstreamer.so[b6f69000+7000]
> Feb  7 00:47:29 pluto kernel: [52547.350109] tracker-extract[730]:
> segfault at bf3defdc ip b79a872c sp bf3defe0 error 6 in
> libextract-png.so[b79a6000+3000]
> Feb  7 00:47:47 pluto kernel: [52565.264967] tracker-extract[892]:
> segfault at bf3e1fdc ip b79e1b7c sp bf3e1fe0 error 6 in
> libextract-html.so[b79e1000+1000]
> ...
> Indexing and searching seems to work fine though, at least
> tracker-search yields results if I look for something.

Yes, I noticed this last night. There are still some files (like a JPEG 
I have) that cause the tracker-extract process to die. I didn't even 
notice until I looked at the tracker-extract.log we now have and saw the 
request ID start from 0 again during a reindex. I think there are some 
files which probably have always caused crashes and now are just more 
noticeable.

Next week, I will be try to fix as many of these that I can find with 
Mikael Ottela. Remember, this is why we have an extractor as a separate 
process, because the libraries and applications we depend on to get this 
information *may* crash. The JPEG issue I refer to was crashing in a 
jpeg library call, not our code. We may be using the library there, 
incorrectly but it still illustrates that we can not expect our 
dependencies to be perfect.

The reason the changes were made, is because previously Urho had a 5000 
MP3 test data set. I was unable to extract the data from these without a 
crash evert *10* MP3s. This was down to the communication between the 
indexer and extractor. Now we are using DBus, this is no longer the 
issue. Any issues with crashing now, are purely related to the content 
type and the specific extractor. It was very hard before to see these 
problems because the extractor has always just been restarted.

I am not sure why you are seeing logs in kern.log?

> I'm back to r2892 for the moment as this version does not have this problem.

Are you able to confirm that the files which are causing the crash are 
actually visible with TST then? I just want to make sure that this isn't 
a simple case of not being reported in kern.log before.

> I also found this in trackerd.log
> (trackerd:18833): Tracker-WARNING **: Error loading
> query:'sqlite-fulltext.sql' #0, Cannot use virtual tables in
> shared-cache mode

This has been this way since Jamie added FTS support to TRUNK as a 
compilable option. It needs fixing, but we are not using FTS yet, so the 
warning is irrelevant for now.

--
Regards,
Martyn