Re: [gtk-vnc-devel] PATCH: Yet more endianness fixes

["Daniel P. Berrange" <berrange redhat com>]

On Fri, Mar 28, 2008 at 08:50:36AM -0500, Anthony Liguori wrote:
> Daniel P. Berrange wrote:
> >The problem I hit is that I ran a server with:
> >
> >   "vncserver -depth 32 -geometry 1024x1024"
> >
> >And the pixel format sent by the server is:
> >
> >  Pixel format BPP: 32,  Depth: 32, Byte order: 4321, True color: 1
> >               Mask  red: 255, green: 255, blue: 255
> >               Shift red:  24, green:  16, blue:   8
> >
> >So, note that 'depth is 32' here, even though the shifts/masks clearly
> >fit in 3 pixels. And empirically the server sends me ZRLE updates with
> >a CPIXEL size of 3, not 4.
> >
> >So when the spec says 
> >
> >    "depth is 24 or less"
> >
> >This is clearly not corresponding to the 'depth' value sent in the pixel 
> >format. So basically we need to ignore explicitly declared  pixel format
> >depth completely and just look at the pixel masks/shifts to calculate
> >the true depth, and thus whether CPIXEL is 3 or 4 bytes.
> >  
> 
> Have you informed the realvnc folks of this?  Either the spec or realvnc 
> should be fixed.  I don't like the idea of violating the spec to fix a 
> bug with one server since we're then breaking another server that isn't 
> violating the spec.

I've looked at 3 different VNC server impls which support ZRLE now and
they all work in the way I described / implemented, so I'm inclined to
commit this

Dan.
-- 
|: Red Hat, Engineering, Boston   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|