Re: [Tracker] Tracker to do list

[Laurent Aguerreche <laurent aguerreche free fr>]

Le jeudi 07 septembre 2006 Ã 22:22 +0200, Laurent Aguerreche a Ãcrit :
> Le jeudi 07 septembre 2006 Ã 20:03 +0100, Jamie McCracken a Ãcrit :
> > Laurent Aguerreche wrote:
> > > Le jeudi 07 septembre 2006 Ã 17:05 +0100, Jamie McCracken a Ãcrit :
> > > > Jamie McCracken wrote:
> > > > > Laurent Aguerreche wrote:
> > > > > > I wonder whether the use of strlen() on UTF-8 is correct, it
> > > > > > shouldn't... If I remember correctly, unicode can use arrays filled that
> > > > > > way:
> > > > > > '\0' 'H' '\0' 'E' '\0' 'L' '\0' L '\0' 'O'      ("HELLO")
> > > > > > where a '\0' can be replaced by a value to stock characters on 2 bytes.
> > > > > > But I don't remember if it happens with UTF-8. I'll have to check what
> > > > > > happen with strlen() and funky characters.
> > > > > utf-8 is not unicode.
> > > > >
> > > > > utf-8 if ascii is always 1 byte per character and is indistinguishable 
> > > > > from plain text/ascii
> > > > >
> > > > > Non-ascii is always 2-4 bytes per character (mostly 2 bytes though).
> > > > Also non-ascii bytes cannot contain an ascii character within its 
> > > > multibyte sequence. (multibyte characters in utf-8 always have bytes 
> > > > with most significant bit of 1 whereas ascii is always less than 128 so 
> > > > has msb of 0)
> > > >
> > > > for ref: http://en.wikipedia.org/wiki/UTF-8
> > >
> > > Ok, thank you.
> > >
> > > So I introduced a bug in tracker-utils.c during my work on UTF8. :-)
> > >
> > > In is_text_file(), I wrote:
> > >
> > >   if (data_read) {
> > >           char *s;
> > >
> > >           s = g_locale_to_utf8 (buffer, 65565, NULL, NULL, NULL);
> > >
> > > I propose this replacement:
> > >
> > >   if (data_read) {
> > >           char *s;
> > >
> > >           s = g_locale_to_utf8 (buffer, -1, NULL, NULL, NULL);
> >
> > yes thanks - seems I missed that one when reviewing your work. The 
> > buffer would need to be at most 4x the size of the input string to be 
> > fully utf-8 safe.
> >
> > It might be worth checking if thats the case elsewhere in tracker.
>
>
> There is another bug now: tracker_db_save_file_contents() is called with
> directory as file_name... So, of course, fgets() blocks on it.
> It seems I've found the reason: text_filter_file was sometimes wrongly
> set to a non-NULL value (because no initialisation happened) in
> tracker_metadata_get_text_file().
>
> I provide a patch.

I am sorry. This patch doesn't correct the bug, but it still needs to be
applied. I continue to investigate.


Laurent.