Text editing and UTF-8 conversion

[Keith Maika <keithm aoeex com>]

 Hello everyone,

I've recently decided to fix up / continue some work on an app I started 
a while go to learn GTK+.  It is a simple text editor like notepad.  One 
of the things I wanted to try and allow for the new version is large 
file support, but I'm having some issues in this area.

I'm using GIO to load the file data, and GtkTextView to display/edit 
it.   I discovered originally that GtkTextView needs utf8 data so in the 
load process I call g_utf8_validate and if necessary, g_locale_to_utf8 
but this conversion process seems to take a long time and I'm wondering 
if there is any way to speed it up.

I wrote a small application to test the loading code with this 
conversion, the source can be found here: http://pastebin.com/dhQhubFt
I have two text files I am testing with, same content, 37.0 megs of text 
exported from a db.  I created using edit+, and saved one w/ it's ansi 
encoding setting, and the other with UTF-8 setting.  My test program is 
able to process the UTF8 version in less than a second as the data is 
already valid.  The other file has characters that require a conversion 
and this bumps processing time up to about 5 seconds.

Is there anything that I could do differently to increase the loading 
speed or is this just something I need to just deal with and move on to 
the next item on the list?

My dev. environment is Visual C++ Express/Windows 7 if that information 
is of use.

Thanks,
Keith