Re: D-Bus AFL/GPL issues (was Re: GLib plans for the next cycle)

[Robert McQueen <robert mcqueen collabora co uk>]

Havoc Pennington wrote:
> Hi,
> 
> On Thu, Apr 23, 2009 at 1:24 PM, Robert McQueen 
> <robert mcqueen collabora co uk> wrote:
>> My belief is that the problem is that under certain implementations
>> of LGPL, the stuff you link the LGPL library to must also be LGPL
>> compatible, and that the AFL patent clause is not.
> 
> This doesn't make much sense to me (the whole point of LGPL over GPL
> is that there isn't a compatibility thing; LGPL should not 
> "contaminate" and require AFL to change).

The LGPL makes a clear distinction for a "work that uses the Library"
and allows all manner of things which the GPL does not. Other than that,
the terms of the LGPL are broadly similar to those of the GPL. In the
case of an LGPL library that relies on libdbus to do its job, libdbus is
definitely /not/ a work that uses the library.

In their definition of a derivative work, they say that you are allowed
to rely on external functions or lookup tables, but that if they are
missing, the library must still operate and its purpose remain
meaningful. In the case of EggDBus (or telepathy-glib), the library is
rendered totally useless by the lack of libdbus, so I think this doesn't
count.

So - if you're linking libdbus into Glib and distributing the result as
LGPL, you could be violating the GPL-incompatible terms of the AFL which
require additional actions that the LGPL does not. It's not open and
shut by any stretch of the imagination, and furthermore I'm not even
certain that this is even the line of reasoning behind our clients'
concerns.

However, I'm not a lawyer, but have definitely have come across clients
(who have presumably asked their lawyers) who have decided this
combination of LGPL + GPL/AFL is unsound. Hence I'd like to better
understand their rationale and check with our own lawyers before
dismissing it.

>> Yes, this is handwavy and I'm not sure what I'm talking about, but 
>> I'm sure of one thing: I have run into clients who due to this 
>> issue have either a) applied a handwavy "system library" defence 
>> for this problem,
> 
> Not sure system library is handwavy... the GPL is (relatively) clear
> on the system library point. However, the problem with system
> library exception is the text "unless that component itself
> accompanies the executable" i.e. it can't be used by someone shipping
> both the app and the library.

Yeah, that's why I said it was a handwave in the context. :)

>> * seeking legal advice from Collabora's advisors on whether the 
>>   above is a genuine issue, or what other concerns might be
>> * endeavouring to find out from our would-be clients above what their
>>   specific concern was
>> * trying to track down the Codefactory code (again) so we can actually
>>   complete the license change and move on with our lives
> 
> Certainly seems sensible to ask what people are worried about. Note,
> there was a lot of uproar about AFL 1.0 patent clause, but we aren't
> using that, this is AFL 2.1. I don't think AFL 2.1 requires anything
> more than LGPL/GPL, with respect to patents. And I think AFL phrases
> the requirements in a clearer way than LGPL/GPL.

The AFL (all versions through v3.0) is incompatible with the GPL
according to the FSF due to requiring distributors to seek the assent of
the recipient:
http://www.fsf.org/licensing/licenses/index_html#GPLIncompatibleLicenses
http://opensource.org/licenses/academic.php (clause 9)

>> Pending the outcome of my enquiries, if if *is* a real concern, and
>> we're unable to find where the Codefactory stuff really resides, 
>> I'd much rather get someone at Collabora to rewrite the code than 
>> handwave and pretend its fine.
> 
> Certainly open to this, if anyone wants to invest the effort.

If it proves necessary, I think we'll try and do this. In the case of
dbus-glib and the daemon itself, there's quite a lot of code there, but
in the restricted case of libdbus (assuming EggDBus) it seems more
tractable. We can just port telepathy-glib to sit atop EggDBus then
(hopefully very thinly), and be shot of the problem after an ABI bump.

>> (Its not an unthinkable amount of code, though of course if the 
>> effort to rewrite it is more than the effort to port EggDBus to 
>> GVariant/GBus, that might make more sense anyway. Although now Qt 
>> is LGPL rather than GPL+proprietary, they now have the same 
>> potential issue.)
> 
> Not to go on a tangent, but I think there's real value to a shared 
> "least common denominator" implementation of dbus.

>>> stop reading here if you just care about the legal issue <<<

This /is/ a big tangent. :) I'll bite and try and give you a little more
understanding based on the past few years at Collabora, but don't be too
offended if I'm not able to reply much beyond this, as I'm pretty busy
at the moment.

> One of the big reasons is precisely the protocol extensions people 
> are talking about. e.g. the "protocol v2" work in gbus/gvariant; it's
> great and all, but basically useless because dbus-daemon does not 
> speak this new protocol v2. To make it useful someone has to extend 
> libdbus to speak v2 anyway, and then what was the value of creating a
> non-libdbus implementation?

Protocol versions aside, I believe Ryan wished to propose new types, and
nobody has really worked out how you're meant to achieve this. Even
given a bus daemon which can speak two versions of the protocol and
convert between them, we'd need to do some kind of pairwise capability
negotiation. The client on the other side can't even parse the
introspect of the v2 client, let alone deciding what you do if you get
messages which use contain types that the daemon knows the other end
doesn't understand. So yeah, it's useless not because of libdbus, but
because the boat has well and truly set sail on changing the type
system, and the protocol is good enough that nobody sees fit to bother
changing it for its own sake.

> Most of the objections to libdbus seem to be around how it's 
> low-level and least-common-denominator, which makes it sort of 
> annoying, but, that was the whole intentional tradeoff when writing 
> libdbus, and I'm not sure people are considering the _value_ of being
> able to change one implementation and suddenly have a new auth 
> mechanism or new protocol feature that _everything_ (all apps, all 
> desktops, bus daemon, etc.) has support for...

In my experience this isn't why people are annoyed with libdbus and so
quick to suggest/endorse alternatives. Yes it's low-level and incredibly
verbose in use, partly due to API design and partly due to the OOM
checking, but I'm really not that convinced that it actually represents
the lowest common denominator, nor that it's a very good API for writing
bindings.

It's fine for the daemon and for apps that directly implement/consume
/really/ simple D-Bus APIs, but pretty tedious if you have any
substantial type recursion going on. (I'm of the opinion (as you can
tell from Telepathy, I guess :D) that you should just have good bindings
and let rip with the types you really want, otherwise what's the point
of having a recursive type system?)

One major issue of the libdbus API for bindings is the aborting stuff.
In general I agree with the principle that the application developer
should know what they're doing, and the library is well within its
rights to stab them in the face if they don't. We actually take this
approach in Telepathy, and most internal functions use return_if_fail
and our Telepathy applications enable fatal criticals.

However, in the case of a binding, the binding is forced to do a lot of
ahead-of-time checking to make sure the application hasn't screwed up,
and then report the error in the appropriate way for that language. In
general that isn't abort(), and nor is the Java/C#/Python/Vala/etc
author going to know what to do with that error anyway. Strictly
speaking this is a binding bug, but it's one that libdbus makes it
really hard to avoid.

dbus-python has had to duplicate a lot of the checking that libdbus does
to validate calls before calling methods in libdbus, because whilst
libdbus requires the application programmer gets stuff right at all
times, dbus-python can be tricked by the python programmer into causing
libdbus to abort. These checks are not in general exported as public
API, and even with many checks in the Python code, its still possible
for the Python programmer to assert libdbus from Python code in various
corner cases.

There are silly cases like half way through packing a struct where the
application has provided the D-Bus type, but later a value that doesn't
fit that type. You can't close the struct because you'll abort libdbus.
Unless you implement two-pass validation and check the types before
building the message, there's no way out of this other than to fill the
struct with nonsenense, /then/ close the iter, /then/ discard the
message. If it was Python code you could just throw an exception back to
the app author and get on with life.

Most bindings /except/ dbus-glib are forced to not use the shared
connection for similar reasons, because if you try and register the same
object path component from two bindings you just get aborted, so it's
not trivial to co-habit two service-exporting bindings on the same
libdbus. It was deemed too much of a hassle in both the Python and Qt
bindings so they just open private connections.

Further to this, the OOM stuff is simply not of interest to (m)any other
apps other than the daemon and the X server, and indeed if you write a
D-Bus implementation in your native language/VM/style then you almost
certainly get this for free, rather than wiring it up every other line
of code.

It seems that if you're not using libdbus directly, or you're not the
bus daemon, its a pretty hostile library to write bindings with,
especially if the language is dynamically typed. People hacking on both
bindings at Collabora have lost hair and screamed and sworn they would
rewrite them to not use libdbus given half a chance, and I honestly
don't think we're alone in this sentiment.

> Switching dbus-daemon away from libdbus seems like a pipe dream at 
> best, and deprecating libdbus is a herculean task, and I just am not
> sure people are thinking pragmatically and realistically on this 
> subject.

Sure, in general I think we're stuck with it for stuff that currently
uses it, but the wire protocol is small, so the effort of writing a
library that speaks/parses the protocol is really quite small. I'm
really not that averse to implementing the protocol for new bindings.
I'm almost certain it will be a net reduction in code size, enhancement
in maintainability, and performance benefit (no double-allocator dance,
dbus-glib *REALLY SUCKS* because of this) in every case.

> To add a Windows transport or SASL authentication or protocol v2 do 
> we really want to run around having to fix the Python implementation,
> the Qt implementation, the GLib implementation, etc.? The pain seems
> mildly worth it with the managed languages (C#/Java) but largely
> pure pain for the other languages that rely on C modules anyway.

I don't think the D-Bus wire protocol is large or complex enough to make
this argument for long. Once you've learnt the secret of the ASCII to
decimal and back again authentication, and been admitted to the secret
D-Bus implementor cabal (the spec /really/ should explain that), you can
write a working implementation in a few days. I'm pretty sure I've seen
finished or at least partly started: one other C reimplementation, two
Python ones, Ruby, Java, C#, and some guys in Collabora just did a
Haskell one too.

Given that, and the rate of change of the D-Bus protocol (~0?), it's
really not that hard to go and patch your D-Bus protocol implementation
with anything new that comes up. Frankly for the past 3-4 years, nothing
new has come up in a way that excited anyone into changing the protocol
or solving issues with pairwise feature negotiation / message
reformatting in the daemon.

I'm also quite certain that dbus-python has way more code to not trigger
hidden landmines in libdbus than it would if it actually spoke the wire
protocol itself. It's only a matter of time availability that has caused
us not to excise the final hidden abort-able code paths by just ripping
libdbus out.

> Not to mention that libdbus and dbus-daemon are quite well tested, 
> with probably the highest unit test coverage of any library in the 
> open source desktop stack that I can think of, and years of 
> real-world testing. This is not a benefit to sneeze at.

Fair, but the ongoing maintenance effort and grey hairs of people who
work and maintain/implement bindings isn't insignificant either, from
the perspective of people at Collabora who've done it at least. Bindings
need just as much test coverage anyway.

If GVariant is on track for Glib inclusion anyway, and if making EggDBus
use GBus is less effort than rewriting bits of libdbus if it proves
necessary, it might not be such a bad option. Any new users of EggDBus
will quickly run into any bugs, and if we port telepathy-glib to it, the
not insubstantial test coverage in that and various Telepathy CMs will
smoke out issues pretty rapidly.

Anyway, I'm not necessarily saying we're going to run around porting
stuff away from libdbus just for the sake of it, just that I don't think
the value of sticking with libdbus is as great as you make out. There
are of course arguments in both directions, but the legal issue is my
current focus.

> Havoc

Regards,
Rob

-- 
Robert McQueen                                 +44 7876 562 564
Director, Collabora Ltd.             http://www.collabora.co.uk