[gtk-vnc] Add workaround to avoid hangs when connecting to SPICE

[Daniel P. Berrange <dberrange src gnome org>]

commit 7f4f2fe8da72ed9fef5dd4319e19feb2b4f3d62e
Author: Daniel P. Berrange <berrange redhat com>
Date:   Thu Jan 26 09:31:40 2017 +0000

    Add workaround to avoid hangs when connecting to SPICE
    
    When used with QEMU at least, both SPICE and VNC live in the same port
    range (5900+). It is thus not entirely uncommon for a user to mistakenly
    connect to a SPICE server with a VNC client, or vica-verca.
    
    When connecting to VNC server with a SPICE client, you quickly get an
    error. This is because the VNC server sends its short greeting and then
    sees the RedLinkMess from the SPICE client, realizes its garbage and
    drops the connection.
    
    When connecting to a SPICE server with a VNC client though, you get an
    indefinite hang. The VNC client is waiting for the VNC greeting, which
    the SPICE server will never send. The SPICE server is waiting for the
    RedLinkMess which the VNC client will never send.
    
    In VNC the protocol starts with the follow data sent:
    
     Server: "RFB 003.008\n"
     Client: "RFB 003.008\n"
    
    This can be tweaked so the client proactively sends the first four
    bytes
    
     Client: "RFB "
     Server: "RFB 003.008\n"
     Client: "003.008\n"
    
    From the VNC server POV, it'll still be receiving the same 12 bytes from
    the client - it just happens that 4 of them might arrive a tiny bit
    earlier than the other 8 of them. IOW nothing should break in the VNC
    server from this change.
    
    The SPICE server, waiting for its RedLinkMess message will see these
    four bytes "RFB " and interpret them as the SPICE magic number. This
    will be invalid and so the SPICE server will drop the connection.
    This avoids the VNC client hanging forever when connecting to a SPICE
    server by mistake.
    
    Signed-off-by: Daniel P. Berrange <berrange redhat com>

 src/vncconnection.c |   22 ++++++++++++++++++++--
 1 files changed, 20 insertions(+), 2 deletions(-)
---
diff --git a/src/vncconnection.c b/src/vncconnection.c
index 89f07d0..95a615d 100644
--- a/src/vncconnection.c
+++ b/src/vncconnection.c
@@ -5198,6 +5198,24 @@ static gboolean vnc_connection_initialize(VncConnection *conn)
 
     priv->absPointer = TRUE;
 
+    /* We should technically read the server greeting first.
+     * If the user mistakenly connects to a SPICE server
+     * though, we'll never see the greeting, because with
+     * SPICE the client starts first.
+     *
+     * By sending these 4 bytes first, if the user has
+     * accidentally connected to a SPICE server, it will
+     * notice this garbage and close the connection, avoiding
+     * us waiting forever for a VNC greeting that'll never
+     * come.
+     *
+     * This is harmless for real VNC servers, since the
+     * early send will just be queued until they've sent
+     * their greeting
+     */
+    vnc_connection_write(conn, "RFB ", 4);
+    vnc_connection_flush(conn);
+
     vnc_connection_read(conn, version, 12);
     if (vnc_connection_has_error(conn)) {
         VNC_DEBUG("Error while reading server version");
@@ -5226,8 +5244,8 @@ static gboolean vnc_connection_initialize(VncConnection *conn)
         priv->minor = 8;
     }
 
-    snprintf(version, 13, "RFB %03d.%03d\n", priv->major, priv->minor);
-    vnc_connection_write(conn, version, 12);
+    snprintf(version, 13, "%03d.%03d\n", priv->major, priv->minor);
+    vnc_connection_write(conn, version, 8);
     vnc_connection_flush(conn);
     VNC_DEBUG("Using version: %d.%d", priv->major, priv->minor);