Re: [gedit-list] Fix Bug 342918 and even support universal encoding detection

[Jesse van den Kieboom <jessevdk gnome org>]

2012/10/16 Ma Xiaojun <damage3025 gmail com>

On Tue, Oct 16, 2012 at 1:27 AM, Jesse van den Kieboom
<jessevdk gnome org> wrote:
> This is just an implementation detail. You can at the moment only _Save_ a
> document with a new encoding (at which moment the new encoding will be set
> in the doc). Note that encodings are _only_ used when loading and saving a
> document from/to disk. The internal representation of the document is always
> in UTF-8. I don't think there is anything particularly difficult about
> allowing to reopen a document with a different encoding.

OK, let me look into it.

> Actually, we have had this kind of "universal encoding detection" for a very
> long time now, so I don't really see what you mean (see
> http://git.gnome.org/browse/gedit/tree/gedit/gedit-document-output-stream.c)

Your detection is just try a list of encodings, one-by-one.
Such naive approach is broken for all regions whose major local
encoding is not ISO-8859-15.
Mozilla, Chromium, Kate are using statistics to help detection.
That's why they can display stuff with random encoding correctly in most cases.
Gedit simply can't.

Well, that's not what it says in the blog post you refer to (there it just mentions the list), but sure, I didn't look further into it. Maybe you can clarify what "using statistics" means?

 

> gedit can't link against random libraries, unless it's something properly
> licenced, released and packaged. In any case, we already do this.

Do you want reinvent a wheel or link against KEncodingProber?

gedit is not going to link to a KDE library (just as KDE apps are not going to link to gobject libs) because it pulls in too many dependencies, not to mention that I'm guessing it's written in C++. With regards to uchardet, it would be very interesting to use it, but we'll have to see if it's possible. For one, is the MPL compatible with GPL? I see that there is a uchardet package in ubuntu at least, I don't know about other distros (also the version of the package is 0.0.1). Traditionally, gedit does not depend on much software outside of the GNOME core libraries, but I don't see anyone reimplementing something like this any time soon.