Re: [gtk-vnc-devel] PATCH: Fix crash in OpenGL scaling code

[Rob Stoddard <rstoddard telanetix com>]

Yes, it would fix that problem. 

I ran into another problem which this widget brings to light, a problem 
with gtkglext which has to do with colormaps.  It happens rarely, but a 
quick and dirty fix for the problem is to set the environment variable 
(either before calling the program,  or in the program with a putenv() 
call) GDK_GL_NO+STANDARD_COLORMAP=1.  The problem is a BadColor error 
that causes X to stop working with any gtkglext program until it is 
restarted.  Like I said before, this is not a problem with gtk-vnc, it's 
a problem with gtkglext.  The first time I have run into it was with 
this widget, however, so there is something that this widget does to 
trigger this issue.  What it might be, I have no idea.

Rob Stoddard



Anthony Liguori wrote:
> Daniel P. Berrange wrote:
> > While getting GTK-VNC to use scaling inside virt-manager, I hit one 
> > annoying
> > crash bug. virt-manager will show the widget before the VNC 
> > connection has
> > neccessarily connected. This results in a crash at
> >   
>
> This looks very similar to the bug Rob reported.
>
> Rob: could you check to see if this patch fixes your problem too?
>
> Regards,
>
> Anthony Liguori
>
> > Program received signal SIGSEGV, Segmentation fault.
> > [Switching to Thread 46912496259456 (LWP 2816)]
> > rescale_display (obj=0x1486570, width=640, height=392) at 
> > vncdisplay.c:959
> > 959             for (j = 0; j < image->height; j++) {
> > (gdb) bt
> > #0  rescale_display (obj=0x1486570, width=640, height=392) at 
> > vncdisplay.c:959
> > #1  0x00002aaab541e75c in configure_event (widget=0x1486570, 
> > configure=<value optimized out>, data=<value optimized out>)
> >     at vncdisplay.c:1087
> > #2  0x0000003eeafa06e5 in _gtk_marshal_BOOLEAN__BOXED 
> > (closure=0x13a3e90, return_value=0x7fff3ac9fc30, n_param_values=2, 
> >     param_values=0x7fff3ac9fd20, invocation_hint=0x7fff3ac9fbf0, 
> > marshal_data=0x0) at gtkmarshalers.c:84
> > #3  0x0000003ee720b1a9 in g_closure_invoke () from 
> > /lib64/libgobject-2.0.so.0
> > #4  0x0000003ee721a831 in ?? () from /lib64/libgobject-2.0.so.0
> > #5  0x0000003ee721ba47 in g_signal_emit_valist () from 
> > /lib64/libgobject-2.0.so.0
> >
> >
> > THe patch is simple - skip doing anything in the configure_event method
> > if we've not got a connection yet
> >
> >
> > diff -r 02bc30c3e8f6 src/vncdisplay.c
> > --- a/src/vncdisplay.c    Wed Mar 05 19:21:36 2008 -0600
> > +++ b/src/vncdisplay.c    Thu Mar 06 17:50:12 2008 -0500
> > @@ -1084,6 +1084,12 @@ static gboolean configure_event(GtkWidge
> >  static gboolean configure_event(GtkWidget *widget, GdkEventConfigure 
> > *configure,
> >                  gpointer data G_GNUC_UNUSED)
> >  {
> > +    VncDisplay *obj = VNC_DISPLAY(widget);
> > +    VncDisplayPrivate *priv = obj->priv;
> > +
> > +    if (priv->fb.data == NULL)
> > +        return FALSE;
> > +
> >      rescale_display(VNC_DISPLAY(widget),
> >              configure->width, configure->height);
> >     
> >
> > Dan.
> >