Re: [gtk-vnc-devel] PATCH: Yet more endianness fixes

[Anthony Liguori <anthony codemonkey ws>]

Daniel P. Berrange wrote:
> On Thu, Mar 27, 2008 at 10:53:08PM +0100, Peter Rosin wrote:
>   
> > Daniel P. Berrange skrev:
> >     
> > > The changes to gvnc.c then deal with a couple of other issues. First of
> > > all the RealVNC server does not follow the RFB spec for CPIXEL (or the
> > > RFB spec is badly written). Basically when looking to see if a CPIXEL is
> > > 3-bytes, we just examine the VNC pixel format depth field & compare to
> > > 24. This is not what RealVNC does when encoding the data. It look at each
> > > individual colour component to see if they all fit in the most or least
> > > significant 3 bytes. So I change the cpixel decoding to follow this logic.
> > >       
> > Err, the following is pretty clear to me:
> >
> > "ZRLE makes use of a new type CPIXEL (compressed pixel). This is the 
> > same as a PIXEL for the agreed pixel format, except where 
> > true-colour-flag is non-zero, bits-per-pixel is 32, depth is 24 or less 
> > and all of the bits making up the red, green and blue intensities fit in 
> > either the least significant 3 bytes or the most significant 3 bytes. In 
> > this case a CPIXEL is only 3 bytes long, ..."
> >     
>
> The problem I hit is that I ran a server with:
>
>    "vncserver -depth 32 -geometry 1024x1024"
>
> And the pixel format sent by the server is:
>
>   Pixel format BPP: 32,  Depth: 32, Byte order: 4321, True color: 1
>                Mask  red: 255, green: 255, blue: 255
>                Shift red:  24, green:  16, blue:   8
>
> So, note that 'depth is 32' here, even though the shifts/masks clearly
> fit in 3 pixels. And empirically the server sends me ZRLE updates with
> a CPIXEL size of 3, not 4.
>
> So when the spec says 
>
>     "depth is 24 or less"
>
> This is clearly not corresponding to the 'depth' value sent in the pixel 
> format. So basically we need to ignore explicitly declared  pixel format
> depth completely and just look at the pixel masks/shifts to calculate
> the true depth, and thus whether CPIXEL is 3 or 4 bytes.
>   

Have you informed the realvnc folks of this?  Either the spec or realvnc 
should be fixed.  I don't like the idea of violating the spec to fix a 
bug with one server since we're then breaking another server that isn't 
violating the spec.

Let's try to get some official resolution of what the right thing to do 
is before we change our CPIXEL handling.

Regards,

Anthony Liguori

> Regards,
> Dan.
>