Re: Fwd: wide char string literals to Glib ustring

["Onur Tugcu" <onur tugcu gmail com>]

Thank you,

You're right. Most of the confusion was from my failed test on ucs4.
I thought I wrote that code below and it threw an exception on linux.
But apparently I was wrong, or missed something in the code.

Now it is simply:

#include <iostream>
#include <ostream>
#include <locale>
#include <glibmm/ustring.h>
#include <glibmm/convert.h>
#include <glib/gmem.h>

template <int= sizeof(wchar_t)>
struct w2ustring_select;

template <>
struct w2ustring_select<2>
{
  typedef gunichar2 const* gunistr_t;
  gchar* (*fun)(gunistr_t, glong, glong*, glong*, GError**);
  w2ustring_select(): fun(g_utf16_to_utf8){}
};

template <>
struct w2ustring_select<4>
{
  typedef gunichar const * gunistr_t;
  gchar* (*fun)(gunistr_t, glong, glong*, glong*, GError**);
  w2ustring_select(): fun(g_ucs4_to_utf8){}
};

Glib::ustring w2ustring(std::wstring const &w)
{
  static w2ustring_select<> which;
  typedef w2ustring_select<>::gunistr_t whichstr_t;
  whichstr_t whatever= reinterpret_cast<whichstr_t>(w.c_str());
  gchar* utf8= which.fun(whatever, -1, 0, 0, 0);
  Glib::ustring u(utf8); g_free(utf8);
  return u;
}

int main()
{
  std::locale::global(std::locale("")); // phew, I didn't know!
  std::wstring w(L"üö");
  Glib::ustring u(w2ustring(w));
}

Which works on the few target platforms including windows, so the
issue is resolved.
(sorry if that specialization is worse than an "if" or could be done better.)
Now with my editors I can write unicode directly into string literals.

On Dec 9, 2007 10:13 AM, Chris Vine <chris cvine freeserve co uk> wrote:
> On Sat, 2007-12-08 at 23:53 -0500, Onur Tugcu wrote:
> > Well, code and words failed me, so I decided to show my testcase this way:
> > http://img147.imageshack.us/img147/8769/extrakv5.jpg
> > This shows the test case I made before typing here, and the one I showed here
> > (for linux) about 70 kb and work-safe.
>
> In the first (working case) the call to Glib::locale_to_utf8() does
> nothing, for the reasons I have explained.
>
> In the second (not working case) you have a conversion exception.  As in
> that case you have not set the global locale with std::locale::global
> (it is commented out), the program will use the "C" locale and the call
> to Glib::locale_to_utf8() will attempt to convert the string literal
> from ASCII to UTF-8 (normally a null operation) but as the string
> literal is by definition not valid ASCII an exception will be thrown.
>
> Were you to call std::locale::global(std::locale("")) and a conversion
> exception to be thrown, that would mean that the narrow charset used by
> your editor is different from the narrow charset of your machine locale.
> However, it looks as if your machine locale narrow character codeset is
> UTF-8 so a call to std::locale::global(std::locale("")) would probably
> work correctly but be non-portable.
>
> No wide characters are involved in this.  If you want to use wide
> character string literals in the code you would need to use
> g_usc4_to_utf8() instead of g_utf16_to_utf8 as in your original first
> code example.  That should work irrespective of locales if you are using
> a unix-like OS.  See code below which works.
>
> For maximum portability, as I have explained it is best to write string
> literals directly in UTF-8 (or to use gettext() with
> bind_textdomain_codeset() having set the bound codeset to UTF-8) and
> only use conversions for the purposes of displaying to a terminal (ie in
> a call to cout).  glib::ustring::operator<<() will perform a conversion
> to the locale codset automatically.  (I do not agree it ought to do so,
> but nontheless it does).
>
> Chris
>
> Wide characters ----
>
> #include <iostream>
> #include <locale>
> #include <glibmm/ustring.h>
> #include <glibmm/convert.h>
> #include <glib.h>
>
> Glib::ustring w2ustring(std::wstring const &w)
> {
>   gunichar const* utf32 = reinterpret_cast<gunichar const*>(w.c_str());
>   gchar* utf8= g_ucs4_to_utf8(utf32, -1, 0, 0, 0);
>   Glib::ustring u(utf8); g_free(utf8);
>   return u;
> }
>
> int main() {
>   std::locale::global(std::locale(""));
>   std::wstring w(L"üö");
>   Glib::ustring u(w2ustring(w));
>
>   std::cout << u << std::endl;
> }
>
>
>