[vte/wip/issue-233] icu: glue: Set error callback on the from-unicode conversion

[Christian Persch <chpe src gnome org>]

commit db6f2c1b4949e51669a0bb4a5e45728022919512
Author: Christian Persch <chpe src gnome org>
Date:   Sat Apr 25 10:37:43 2020 +0200

    icu: glue: Set error callback on the from-unicode conversion
    
    The from-unicode side of the converter is only used on user input
    (keyboard, IM, clipboard paste). When it receives input not representable
    in the target encoding, it inserts the substitution character of the
    target encoding, which in many cases is U+001A. However, when sent to a
    PTY, that is interpreted as Ctrl-Z.
    
    Instead, set the from-unicode converter to error out when encountering
    such unrepresentable input.
    
    https://gitlab.gnome.org/GNOME/vte/issues/233

 src/icu-glue.cc | 27 ++++++++++++++++++++++++++-
 1 file changed, 26 insertions(+), 1 deletion(-)
---
diff --git a/src/icu-glue.cc b/src/icu-glue.cc
index d86f23c3..6350386f 100644
--- a/src/icu-glue.cc
+++ b/src/icu-glue.cc
@@ -111,10 +111,35 @@ make_icu_converter(char const* charset,
 {
         auto err = icu::ErrorCode{};
         auto converter = std::shared_ptr<UConverter>{ucnv_open(charset, err), &ucnv_close};
-        if (err.isFailure())
+        if (err.isFailure()) {
                 g_set_error(error, G_CONVERT_ERROR, G_CONVERT_ERROR_NO_CONVERSION,
                             "Failed to open converter for charset \"%s\": %s",
                             charset, err.errorName());
+                return {};
+        }
+
+        /* The unicode->target conversion is only used when converting
+         * user input (keyboard, clipboard) to be sent to the PTY, and
+         * we don't want the ucnv_fromUChars to substitute the SUB character
+         * for illegal input, since SUB is U+001A which is Ctrl-Z, which
+         * the default UCNV_FROM_U_CALLBACK_SUBSTITUTE callback does.
+         * Use UCNV_FROM_U_CALLBACK_STOP to stop conversion when encountering
+         * illegal input.
+         */
+        err.reset();
+        ucnv_setFromUCallBack(converter.get(),
+                              UCNV_FROM_U_CALLBACK_STOP,
+                              nullptr,
+                              nullptr,
+                              nullptr,
+                              err);
+        if (err.isFailure()) {
+                g_set_error(error, G_CONVERT_ERROR, G_CONVERT_ERROR_NO_CONVERSION,
+                            "Failed ucnv_setFromUCallBack for charset \"%s\": %s",
+                            charset, err.errorName());
+                return {};
+        }
+
         return converter;
 }