Re: How to do backend-per-process with GDBus properly

[Tristan Van Berkom <tristan upstairslabs com>]

Hi Milan,

On Fri, 2014-05-30 at 17:34 +0200, Milan Crha wrote:
[...]
In case of another client trying to open a different backend, the steps
> above will look the same, only the factory will open a new 
subprocess.
> The subprocess part makes it complicated, because:
> a) the needed D-Bus interface moved from the factory process to the
> subprocess
> b) there can be multiple subprocesses running at the same time, while
> the old behaviour made sure that only one factory was running all the
> time.
> 
> Fidencio (he's working on this) managed to write some basic code
> changes, but they do not fully work. GDBus currently claims that the
> required interface is not available. We may probably miss something
> obvious to someone more knowledgeable in GDBus, thus I do not want 
to go
> too deep into detail now, I'd rather appreciate an advice, a 
direction
> where to look and how to cover the above described scenario properly.
> 
I think we're missing a bit of detail here, when you say "another client
trying to open a different backend", do you really mean a different
backend ? or the same backend on a different persistence (i.e. in EDS,
the same backend for a separate ESource) ?

From what I understand, the same interface can be exported on the same
D-Bus 'bus' as many times as you want, provided that they are exported
on different paths - however I may be missing something as well, there
may be a rule about one process being the 'provider' of a given
interface and perhaps in a D-Bus session environment, there can only be
one 'provider' of a given entity, regardless of how many paths it's
exported on (This might be a logical restriction in the D-Bus design, as
it would reduce the possibility of deadlocking processes which make
D-Bus method calls to eachother). And indeed, the way that the D-Bus
"bus" concept includes the concept of "name owners" seems to imply that
an interface can only be implemented by a single client (and this is
where I think your multiple-process approach is breaking down).

That said, there is a flaw (IMO) in the way EDS uses D-Bus from the
beginning, and fixing that flaw may well be the answer which also fixes
this issue. The flaw I'm referring to is that EDS does not create any
GDBusServer instance for it's backends, meaning every client
communicates with EDS indirectly and over the session bus, while that is
only necessary for the client instantiation (i.e. opening a connection
to a backend should return a D-Bus address to a running D-Bus server,
instead of an object path).

So while currently we have this pattern:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

e_book_client_connect()
 -> D-Bus Server (send)
 -> EDS Factory (return object path of the D-Bus Client/EDS Backend)
 -> D-Bus Server (receive)
Return from e_book_client_connect()

e_book_client_get_contacts()
 -> D-Bus Server (send)
 -> EDS Factory (backend creates huge list of contacts)
 -> D-Bus Server (receive payload, choking up the session bus)
Return from e_book_client_get_contacts()


We should instead have this pattern:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

e_book_client_connect()
 -> D-Bus Server (send)
 -> EDS Factory (return D-Bus address of the D-Bus Server/EDS Backend)
 -> D-Bus Server (receive)
Return from e_book_client_connect()

e_book_client_get_contacts ()
 -> EDS Backend creates huge list of contacts
Return from e_book_client_get_contacts()


With the second pattern, each running EDS backend *is* a D-Bus server
instance with it's own D-Bus address, in which case:

 a.) EDS does not slow down the session bus by pushing payloads
     through it (the session bus is only used to obtain the D-Bus
     address of a running backend)

 b.) Pushing and receiving batches to and from EDS is faster, no
     middle man is needed

 c.) There can be fuss about each backend providing the same interface
     at one or more object path, because each backend is it's own
     encapsulated D-Bus environment and they don't compete for the
     interface name in the "session bus"

This is something I've recommended before and we've discussed on IRC
also, I don't think it would be very complicated to do especially if you
are already trying to split the backends into separate processes. I also
suspect that this approach will fix the problem you've stated in this
email, if I've interpreted it correctly ;-)

Hope this is of some help,

Cheers,
   -Tristan