Re: hi, I want to join the conduit development

[Alexandre Rosenfeld <alexandre rosenfeld gmail com>]

Really happy to meet you! Your help is critical to me!

Is it easier to get you by this new email: alexandre rosenfeld gmail com ?

Any email is fine by me.

 

I have read some code of conduit, but I'm sorry the real work will not start recently.

Would you please help me make clear some concept of conduit project?
I think it's important to go in the right direction.

Below is my understanding of conduit project. Please correct me if I'm wrong.

I think the core philosophy of conduit is "Sync anything between any device through any media".
All of the other sync program (unison, dropbox, etc) can only synchronize files.
Although in UNIX everything is a file, we can't take this view in the area of synchronizing.
It will be great to build an open ecosystem around conduit.
The current code has already implemented the "sync anything" part by a plug-in system.

Take a look at https://live.gnome.org/Conduit/Development, it has a lot of documentation about how the Conduit source code is architectured. Take a look at those links as well, the DataSource, etc, they will help understand the core concepts behind Conduit.

 

Conduit should play a role of "sync back-end", the current mainline code mixes back-end and front-end together, so "conduit daemon project" is a big step by splitting them up.
There should be many front-end, the current configuration ui (conduit-cfg-ui) should be one of them.
Some front-end may be embedded in application, like an Evolution plugin.

That was my project's final objective. I spent a lot of time making all of the dataproviders independent of the UI by making a abstraction of the configuration dialog. This means there is no longer any Gtk code in the dataproviders and mostly of the code that starts them also have no dependency on Gtk.

Conduit is mostly ready to run as a daemon, but one main issue is that it's written in Python and inherits it's threading issues.

For instance, Conduit and it's data providers all run in the same process, which means a bug in one dataprovider can affect any other synchronization that is being done by Conduit. 

And Python's threading model is very difficult to handle well, especially when you start working with bindings (something to do with locks when Python code calls C code, as far as I remember).

 

Administrator could create sync-group to sync system data.
Normal user could also create sync-group to sync his/her data. (how to deal priviledge? Require the user has same uid on both machines?)

Not sure if I understood, but please never tie a user data to his uid, it will create a bunch of other problems.

 

Conduit should be decentralized and it already is.
(Can you tell me how does conduit implements it?)

As I said, the dataproviders all run in the same process, so it's not quite as decentralized as it should.

 

Sync-group could be persistent or one-time.
one-time sync-group could be the back-end of "NFC sync" application.

Conduit could be used to do live-migration.
If we develop a virt-machine-live-migration plugin in which utilizes like libvirt, it can sync VM between desktop PCs.
We could also develop a plugin to do process-live-migration by Checkpoint/Restart.

Some type of sync could need application's co-operation.
Like If I live-migrate a game to another machine while the game is playing, the game should save it's state to a buffer, and conduit should start the game on the other end, sync the buffer, and restore game state by this buffer.


That all in my head now.
These are mainly about what conduit should / should not do. Conduit is a framework, so I think these are important.
Please correct me if I'm wrong, thanks!

It's mostly correct what you said, I'll just try to explain what I learned from Conduit.

First, it's more then just Dropbox because it understands what kind of data each party can handle. This is really awesome, since you can synchronize all kinds of metadata, such as ratings, play count, date taken, etc, for each kind of data. And it can also convert from one format to another, such converting videos to the right size or converting from wma to mp3.

Second, it connects applications and services without them ever knowing about each other. This is amazing, for instance you can hook your Shotwell library to any kind of online publishing, without having to wait for the Shotwell team to implement it. I began using Conduit to sync my iPod with my Rhythmbox library.

So here comes the difficult part, since it can sync virtually anything, it just depends on what dataproviders you write for it, it becomes quite hard to test it and especially to create a good user interface to it.

I believe Conduit should focus on a few use cases and do it right, even if it means not doing everything possible. 

For instance, each application already has the best interface for each data type, we shouldn't have to reimplement it in Conduit. What we need is to put Conduit inside each application.

One case which Conduit would be amazingly good at is to implement a system similar to the Android's share button. Click Share in an application and Conduit would present to you a list of dataproviders which would accept it.

But technically, Conduit is not up to the task. My vision for it would be to have a lightweight DBus daemon written in Vala that would spawn a process for each dataprovider, which would also communicate via DBus and could be implemented anyway it wishes (preferably inside the application that owns that dataprovider).

Anyway, I have nothing concrete as well, just a few ideas.