Re: [Easytag-mailing] Japanese and english text = problems

[Jérôme COUDERC <easytag gmail com>]

Hi,

   In fact, there is a bug in id3lib to write some characters (as 
Latin-1, Japanese, ...) to UTF-16. There are information on the page : 
http://sourceforge.net/tracker/index.php?func=detail&aid=1016290&group_id=979&atid=300979.

According to the description, I've built a patch that can be applied to 
the id3lib 3.8.3 sources.
It is attached to this mail and works correctly for me.

Regards,
Jerome


Dullin Panuru wrote, the 05.02.2006 07:44 :

> It seems that the way EasyTag handles field when there is both English
> and Japanese in a field is not top notch. I was happily tagging away and
> everything seemed fine. I was finding the albums automatically with the
> CD database search which returned me with Japanese track listing and
> applied to the directory the music was in and everything was dandy when
> checking the files. 
>
> Unfortunately, problems popped-up when I tried using those file in my
> favourite music player Amarok. The album I tried to tag not only didn't
> display things properly but misplaced information. First of all I will
> display the track listing of the album :
> album name 
> ファイナルファンタジー  V マンボdeチョコボ
> Artist
> セニョール植松とマンボ・パンチョス
>
> Track listing
>
> http://gamerseden.kir.jp/soundtrack/ff/ff5manbo.html (in case something
> goes wrong in the mailing-list encoding.
>
> マンボ de チョコボ
> 飛竜の山　（ゲーム未収録曲）
> オープニング・アイディア Version2　（ゲーム未収録曲）
> 飛空艇 Version2　（ゲーム未収録曲）
> FINAL FANTASY MEGAMIX
>
> Now here is what the tags looked like after saving, changing directory
> and coming back in it
> Album name
> ￕﾡﾤￊ￫ￕﾡ￳﾿ﾸ?　￞￳ￜde￁￧ﾳￜ
>
> Artist
> ﾻￋ￧￫植松と￞￳ￜ￑￳￁￧ﾹ
>
> Track listing
> ￞￳ￜde￁￧ﾳￜ
> ￛￜの山（ﾲ￠未ￎ録￲）
> ﾪￗￋ￳ﾰﾢﾤￇﾢVersion2（ﾲ￠未ￎ録￲）
> ￛ空艇Version2（ﾲ￠未ￎ録￲）
> FINAL FANTASY MEGAMIX
>
> Now even weirder is the fact that under Amarok, all the titles are
> displayed correctly but the first 4 tracks have their album name
> replaced with their track number with a 0 in front of it (like 01, 02,
> 03 and 04)and all their album names are blank but the year,genre and
> track number are all correct.
>
> I tried fiddling with the encoding settings but I did not have any
> success (tried putting everything in Unicode mostly).
>
> One last thing, in the album CD data base search the album name was
> reported incorectly to the tags also as ファイナルファンタジー?　マンボ
> deチョコボ (notice the ? in the middle).
>
> Could anyone report on what is happening and maybe shed light on how to
> make things work.
>
> Thanks a million for this program it saved me countless hours of boring
> renaming.
>  

--
EasyTAG - Tag editor for MP3 and Ogg Vorbis files
http://easytag.sourceforge.net
--
Jerome COUDERC <easytag gmail com>

diff -ruN id3lib-3.8.3.orig/ChangeLog id3lib-3.8.3/ChangeLog
--- id3lib-3.8.3.orig/ChangeLog	2003-03-02 01:23:00.000000000 +0100
+++ id3lib-3.8.3/ChangeLog	2006-02-17 22:21:30.683249296 +0100
@@ -1,3 +1,8 @@
+2006-02-17  Jerome Couderc
+
+    * Patch from Spoon to fix UTF-16 writing bug
+      http://sourceforge.net/tracker/index.php?func=detail&aid=1016290&group_id=979&atid=300979
+
 2003-03-02 Sunday 17:38   Thijmen Klok <thijmen id3lib org>
 
 	* THANKS (1.20): added more people 
diff -ruN id3lib-3.8.3.orig/src/io_helpers.cpp id3lib-3.8.3/src/io_helpers.cpp
--- id3lib-3.8.3.orig/src/io_helpers.cpp	2003-03-02 01:23:00.000000000 +0100
+++ id3lib-3.8.3/src/io_helpers.cpp	2006-02-17 22:18:57.000000000 +0100
@@ -363,11 +363,23 @@
     // Write the BOM: 0xFEFF
     unicode_t BOM = 0xFEFF;
     writer.writeChars((const unsigned char*) &BOM, 2);
+    // Patch from Spoon : 2004-08-25 14:17
+    //   http://sourceforge.net/tracker/index.php?func=detail&aid=1016290&group_id=979&atid=300979
+    // Wrong code
+    //for (size_t i = 0; i < size; i += 2)
+    //{
+    //  unicode_t ch = (data[i] << 8) | data[i+1];
+    //  writer.writeChars((const unsigned char*) &ch, 2);
+    //}
+    // Right code
+    unsigned char *pdata = (unsigned char *)
+    data.c_str();
     for (size_t i = 0; i < size; i += 2)
     {
-      unicode_t ch = (data[i] << 8) | data[i+1];
+      unicode_t ch = (pdata[i] << 8) | pdata[i+1];
       writer.writeChars((const unsigned char*) &ch, 2);
     }
+    // End patch
   }
   return writer.getCur() - beg;
 }