Re: Matrix as a replacement for Telepathy

[Robert McQueen <rob endlessm com>]

tl;dr - don't write multi-protocol messengers or gateways unless you
have a hugely compelling reason, I did it for 10 years and all I got was
this lousy t-shirt ;)

On 25/08/17 17:43, Matthew Hodgson wrote:
> Personally, I would really like to have a local comms API abstraction,
> similar to the one that telepathy provided, but able to support modern
> protocols - i.e. some daemon which exposes a simple API for multiple
> apps to leverage when they want to send/receive messages/calls/etc.  As
> I mentioned earlier, we need it for the Librem 5 phone if nothing else
> (just as Maemo drove the development of telepathy, iirc).

Having, er, done this already, for desktops as well as cellphones, I
would really seriously encourage you to... not adopt this architecture.
Most of the original goals underlying the Telepathy architecture turned
out to be false. You can't actually/realistically unload the UI of the
phone and have it be sufficiently responsive/reliable for things like
emergency calls, and you can't (sensibly) implement reliable local
logging of messages after you've split the state of whether messages
have been received, stored and shown into three processes, and you can't
efficiently implement a contact list that aggregates presence and other
metadata for contacts unless you have all of the data in process, or
copy it round repeatedly (I can only apologise for libfolks).

You lose a huge amount by not being able to have your protocol code in
the same process as the UI, unless you write an API which basically _is_
your protocol, or an API which is the UI, but the bigger the gap between
the two, the more developers fall into that pit and don't get any UI or
protocol work done. :)

Essentially, adding the D-Bus API to _entirely_ indirect the protocol
was, in the end, what made Telepathy so slow, expensive and frustrating
to develop, and discouraged any new contributions. As you've rightly
pointed out elsewhere in the thread, there is a _lot_ of protocol
functionality that appeared even in XMPP at the time, as well as other
protocols since, which Telepathy never gained - part of the problem is
that there are now a long list of components from spec, logger, client
lib, UI, backend, etc that must all be developed and tested together to
add one vertical feature (eg - message read receipts).

Doing it over again, I'd just make "GNOME Chat" or whatever based on a
UI process which could optionally unload/hide it's UI. Then you have
single place on the system to do the common stuff like logging, contact
aggregation, protocol configuration, managing presence, etc. If you
_need_ to export API to integrate to the destkop, you can just implement
and export that. But that's really not all that different to
Pidgin/libpurple any more, maybe with a nicer GNOME-y UI - or a
Matrix-only G* client which implemented everything you needed and had a
few touch points.

The only thing you lose vs current Telepathy in this scenario is the
ostensible security benefits of having your protocols able to be
sandboxed off into neat little walled gardens, although Telepathy was in
theory architected this way I'm not entirely sure who outside of
Nokia/Maemo managed to take advantage of this - possibly SELinux on
Fedora/RH?

However: with modern protocols and encryption approaches (just look at
Telepathy's chequered history with OTR) this falls down _really_ quickly
when you need to do things like logging that must be either trusted or
not to access/store encrypted messages, potentially a centralised key
store, S/ZRTP where the media streaming code needs the key anyway, etc
etc - so in a sense I think some of the IM protocols are too "high
touch" to be segmented away like that - and then the security is comeing
at a high cost of functionality.

But in your "Matrix megadaemon" scenario, where you've added a huge
complexity/ABI/IPC cost to changing UI <- IPC -> daemon, on top of the
existing cost of gateway <- protocol -> client, so you've still got all
of these downsides, but none of the upside.

> Neither Matrix nor libpurple provides this API out of the box currently.

When I started IM client development in whenever it was - 2000 maybe -
the hot new thing in Gaim (now Pidgin) was the "core/UI split" where by
adding _magic IPC_ to Pidgin, a daemon would appear with all the gnarly
protocol/core/etc logic and then a plethora of UIs would arrive on top
of it. My recollection is that even without the IPC (which I think
didn't get past a "send this message to someone" remote), the effort of
cleaning up the internal interfaces to make this even remotely possible
burnt out the lead developer at the time, although this did incidentally
make libpurple a possibility.

Before starting Telepathy we also looked again at this, with "ok, now we
have D-Bus, can we just complete that rather than starting our own
spec?" and the answer was pretty clearly no, because there is a lot of
UI which is dependent on code in the protocol backends. With the benefit
of hindsight, that should've told us something - but we weren't
listening because we were busy trying to solve the (imaginary) benefits
of unloadable/componentised UIs, centralised logging, etc. :)

>  So some replacement for telepathy probably is required, and if it spoke
> both Matrix & libpurple & some native protocols on the backend then so
> much the better.  Meanwhile, apps themselves can link against libpurple
> matrix-*-sdk today according to taste, if they want to use communication
> behaviour without relying on anything provided by the OS.

I'd do it this way round: Make a list of use-cases which can't be met by
having _a_ client which speaks the protocol(s) you need, and then figure
out the places elsewhere in the desktop that need to access it, and
write API to be the "bottom of that UI" on a case by case basis, rather
than coming at it from the top of the protocol stack.

Contributors will appreciate being able to develop on one component and
not 12, in one language, with less strict ABI/API concerns, rapid, etc
prototyping. David Woodhouse speaks good sense with his linc/sipe battle
stories, and from the UI side, in the end that actually really killed
off the ability for people to contribute to Empathy even when they
wanted to. "Hey can I add OTR support? There is libotr!" well... bla bla
bla new spec patch this patch that bla bla security keys bla etc etc ok bye.

However... at the end of the day, after thinking about this for 15
years, I am now just massively skeptical about the multiple-protocol
endeavour because it's trying to square a circle which is not a
real-world problem, and it's antagonistic to the specific goals of every
single IM/comms network provider, so you can burn resources to win each
battle but in the end, you will lose the war. It's appealing for free
software people because we're "the outsider" and we need "our client" to
access the "other people". But if you have one system you prefer to use,
and you are familiar with and aiming for that specific system, you can
_ALWAYS_ write a better client by ejecting the other protocols and
showing the semantics of that system correctly to the user.

Protocols differ, and the optimal UI for that differs too.
Pidgin/libpurple has captured/retained the mindshare for people who want
to add a protocol or need to develop UI along with their protocol
because the API allows them to do this without forcing everything
through a semantic abstraction. Any other user of libpurple who isn't
Pidgin has a commensurately (or significantly) worse experience (just
see telepathy-haze).

I'd wager that your Purism SMS and phone UI will be better at SMS and
phone if it simply doesn't try and do any of the other stuff - the
buttons you need in the UI are pretty much just set out in the GSM spec.
Android uses other apps to do the other stuff. The exception is LTE SIP
where it's literally GSM-semantic calls over a different transport.
Skype has a significantly different call UI for PSTN calls than it does
for Skype calls, because for usability's sake, it's easier to represent
different semantics to the user with a different UI.

I think Polari is another excellent experiment which also proves the
point - the best experience for IRC is an IRC client, which is as good
an experience as we've written for people who want to use the full range
of functionality that IRC has to offer - but due to Telepathy
underneath, it doesn't _have_ all of that IRC functionality, so it
plateaus at a certain level of usefulness for IRC users.

Matrix might provide a better UI experience for people who wished they
weren't using IRC (as I believe many people here do), but the gateway is
definitely inferior for non-Matrix users as it fails to correctly follow
IRC norms. Matrix lets you redact a message even if IRC has no way to
send that, everyone has a [m] from the IRC and when Matrix users speak
to each other, the tab completion is for a Matrix-derived full name
rather than the IRC nick, making it hard for IRC natives to follow a
conversation, etc.

Not to pick on Matrix (or IRC), I just don't believe any longer that
there's a silver bullet here. With hindsight, on the assumption that we
want to design/control a certain level of experience, and multi-protocol
is a need for interoprtability even at the cost of limiting
functionality, I think we'd be better off investing as a community into
a GNOME Chat that used libpurple but knocked some of the more weird API
corners into submission, and exposed specific well-designed integration
points to the rest of the desktop - quick chat/reply box in Shell,
presence/etc for Contacts, file transfer in Nautilus, etc.

This is in a sense a stylistic choice, but the Matrix protocol itself in
the end _must_ be an indirection/abstraction and hence developer
friction and impedance mismatch point, the same as Telepathy's D-Bus API
did, so although Matrix is well aligned with the IM/etc systems of
today, and I'm sure has avoided some of the conceptual mistakes that got
Telepathy tied up in latter days, I'm sure there are some other
different/new mistakes that mean the indirection will become
increasingly expensive, difficult and frustrating as the world moves on.

And, same as XMPP - I was always a strong believer in the federation
aspect and having an open/free protocol, but for two things, a) nobody
uses it any more for IM and the big players cut their portals/gateways
off due to SPIM - after you abstract away identity you can't model trust
properly at scale any more, and b) I don't really like centralised
gateways. :) You're essentially just adding a centralised infrastructure
problem to the semantic/abstraction one - once XMPP became popular
enough, the public servers and gateways became a significant abuse
target and the reliability totally evaporated, because in the end every
operator cares more about "their" users than backing any 3rd party. I
also don't really presume that just because anyone can run a server that
this is credible / desirable for general users.

If you're not trying to promote one network in favour of others, you
don't really need to take on the challenge gateways to relegate certain
protocols to 2nd tier - GNOME has a big enough fight on our hands with
incumbents and externalities frustrating us in the desktop space without
compounding the risk and joining a fight about the winning IM protocol
as well.

> M

Cheers,
Rob

........................................................................

Robert McQueen  |  +1.415.413.4159  |  Endless <http://endlessm.com/>