Re: GVariant for prez!

[Havoc Pennington <havoc pennington gmail com>]

Hi,

On Thu, Apr 9, 2009 at 11:38 AM, Dan Winship <danw gnome org> wrote:
> So, my feedback is, if the type system is explicitly just D-Bus and
> nothing else, shouldn't it be called "GDBusType" or something? (I'm not
> saying it's *bad* that it's D-Bus-specific, just that it should be more
> explicit about it.)

I think as proposed it is not dbus-specific; the idea really is to
have a binary data format that has *two* use-cases, dbus and mmap'd
files (for dconf). (And possibly others as people find it useful.)
AIUI the concept is to generalize dbus, which is a common-used piece
of open desktop tech, to the mmap'd files case, rather than having a
different binary serialization format for the files case. And by
splitting out the serializer, encourage other uses of this format.

Perhaps, however, it should be clearer that this is one serialization
format, rather than a fully generic type system in the GType sense
(see the other email I just sent). The analogy is to GKeyFile, perhaps
thinking of GVariant as GBinaryFile almost (except it isn't strictly
for files only)

> (Alternate possibility: add a way to include type tags, so that callers
> and callees can agree that a particular "x" is to be treated as a time_t
> and not a generic int64. Eg, a signature of ":x" would mean a (string)
> type tag followed by an int64, but it would be distinct from "sx" or
> "(sx)".)

This was considered for dbus; for the original rationale on why it was
not done, see
http://lists.freedesktop.org/archives/dbus/2004-June/001169.html

Basically the dbus signature type system only describes the binary
data, not the interpretation. So it has integers but not enumerations,
it has heterogeneous-type tuples but not named structs, it has arrays
but does not distinguish list vs. slist vs. array, it has
"dictionaries" but doesn't guarantee the keys are unique since the
key-value pairs are serialized, not hashed. The assumption is that
there is implicit or explicit interface description; either an IDL, or
at least some code that assumes stuff about the signatures of the
functions it calls.

Where this falls over is a variant, which is supposed to describe its
own type; a dbus variant really only describes its binary layout,
rather than its type. But variants by definition have a type
unspecified by the IDL. To describe the type such that you can
round-trip arbitrary language-specific types through a dbus variant,
you suddenly need a LOT more code and it greatly complicates
interoperability and language binding implementation - you introduce
the need for a common type repository, something that dbus doesn't
really have (on purpose).

One of the tensions that's tough to deal with is: is the dbus type
system supposed to be the intersection of all other type systems or
the union?
  http://lists.freedesktop.org/archives/dbus/2007-August/008290.html
  http://lists.freedesktop.org/archives/dbus/2007-July/008154.html
This whole thread:
  http://lists.freedesktop.org/archives/dbus/2007-August/008300.html

If you want to pass other type systems through dbus variants without
losing type annotation, then dbus variants have to be tagged with the
union of all possible type systems. For most other purposes, e.g.
interoperability and ease of coding a binding and less code in
general, you probably want the intersection (roughly) of type systems.
Intersection = much much simpler. Union = allows full transparency
when interoperating only with yourself, i.e. round-trip any given type
system through dbus variants unmodified.

For the xml-rpc datetime type, I'd be tempted to give Joel's answer
about C++ strings.
http://www.joelonsoftware.com/articles/LeakyAbstractions.html
"Amusingly, the history of the evolution of C++ over time can be
described as a history of trying to plug the leaks in the string
abstraction. Why they couldn't just add a native string class to the
language itself eludes me at the moment."

In other words, all language frameworks I can think of do have a Date
type. The simple solution is to just hardcode a Date type in dbus,
rather than adding a super-generic support-any-type-at-all framework.
However, on the downside, we can't extend what dbus supports at all,
without adding a whole feature-negotiation thing to the protocol, and
so far no limitations of dbus have been annoying enough that anyone
was motivated to code that ;-)

It is plausible to have the GVariant type system be a superset of the
dbus type system, though. I don't think that actively hurts anything,
just means some GVariant will fail if you try to push them over dbus,
which is fine imo. The gjs dbus bindings for example throw an
exception if you try to send 'null' over dbus, and that's fine. It's a
real bug, to be fixed in the app.

Just to emphasize it again, the problem with something like the
XML-RPC date type (which is really the same as trying to push any
language-specific or protocol-specific type through dbus) only comes
up with variants, say for example if you have a string-to-variant
dictionary. If you just have a method
"Date getDate()" then the implicit or explicit interface contract, aka
IDL, deals with the problem; both sides can know that something
serialized as int64 is really a date. Because the problem is limited
in scope in this way, people have been able to work around it and it
doesn't come up that much in dbus usage. There are always hacks too,
like sending your own type annotations via some side channel such as
an additional arg to a method.


Anyway, just a download of background information.

My opinion/conclusion, as opposed to the download: I would tend to
avoid arbitrary custom type tags in the serialization format. I think
extending the serialization format a bit to be a superset of dbus (add
Date, add null, add float, one of the links above has a list of common
requests) is a reasonable thing to do. Maybe someday we add the same
extensions to dbus, so it would be nice to discuss these extensions on
the dbus list. The other good choice besides extending the format, I
think, is to just work around it case-by-case, e.g. by passing extra
args that contain type info, or whatever, which is what people already
do with dbus.

But I don't think it's the right goal that any arbitrary type system
can be registered in some kind of implicit or explicit dbus type
repository and round-tripped through the dbus variant. (Or the variant
used for mmap'd files, if those files are supposed to be in any way
language-independent or framework-independent, as they probably should
be.) To me a serialization format should be an interoperable
intersection of type systems, not something capable of expressing an
arbitrary type system. There are many advantages to this.

Havoc