Re: [Tracker] Discussing the API of libtracker-miner

[Martyn Russell <martyn lanedo com>]

On 17/08/09 19:00, Adrien Bustany wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi all,

Hi :)

> currently, we have a branch of tracker named libtracker-namer, which
> includes a lib to be shared among all miners (if I got it right).

Yep.

> Currently, this lib includes a miner skeleton, and monitoring
> functions. The idea would be to include more features in this lib,
> merging the work I did for my gsoc project.

That sounds great.

I should add, I have taken a look at your work and it looks good, but I 
think we will be writing the miner API in C and can probably provide a 
vala binding for it. We want people to be able to write miners in 
whatever language they want and C is most portable for this. Also, we 
have a large portion of the work done in C already of course.

> The miners I coded all connect to remote web services to pull data
> like twitter status, flickr albums etc. They thus need to know about
> the network status before initiating a sync (not mandatory, but it
> prevents failing miserably). Also, they all need to store a token or
> user/pass tuple to authenticate against the remote web service.

I think the authentication part will be the hardest part to figure out 
for cross platform working and also for the Maemo platform which doesn't 
use GnomeKeyring either (AFAIK).

> Therefore, I propose the additions of the following services to the lib :
>
> 1. Network status
> Could use NetworkManager if available (we're not making a hard
> dependency here) to know if the computer is connected to internet or
> not. This would be abstracted with an interface to allow other network
> providers (I think OpenSolaris for example uses something else).
> NetworkStatus get_network_status ();
> with NetworkStatus being an enum with 2 values, CONNECTED and
> DISCONNECTED (better to put an enum here if ever we want to add more
> values, for example to handle cases like 3G connections where data
> transfers might be expensive for the user).

Yes, I have used this in Gossip before too, it is fairly straight 
forward, but I don't think we can use NetworkManager in Maemo and I 
don't know if KDE use it. We can probably use some other low level 
tricks to know if we have a connection. Any suggestions?

> 2. Tracker constants
> Currently the fs miner hardcodes Tracker's DBus path. Wouldn't it be
> simpler to put it in a shared header ?

Yes. We have that in libtracker-miner. There is a common DBus API header 
there which is unfinished at this point (I will come on to that later).

> 3. Credentials storage
> Note : this might not belong to libtracker-miner... Discussion is open!
> To connect to the webservice, you generally need a token or a
> user/pass tuple. Sometimes, you also need to store additionnal data,
> so a simple user/pass API is not enough. Again, things should be
> hidden by an interface.
> I currently use something like (vala syntax) :
> string get_password (string token_name, out HashTable<string, string>?
> user_data) throws GLib.Error;
> void forget_password (string token_name);
> void store_password (string token_name, string password,
> HashTable<string, string>? user_data) throws GLib.Error;

Again, this is problematic and needs some consideration. Ivan do you 
have any ideas about how this would work on the Maemo platform?

> 4. Configuration storage
> It'd be nice for all the miners to have an easy API to store key/value
> tuples. I've been told Martyn is working on it...

Well, we have tracker-config-file in libtracker-common and 
tracker-keyfile-object with some convenience APIs, but it is all in C, 
not vala. You can see how we have done tracker-config in 
tracker-miner-fs and how we use those APIs there.

> Thank you for your feedback !

No problem.

I should continue here with our current plans for the miner APIs:

Methods:

 * gchar * get_status()          - any string
 * gdouble get_progress()        - 0->1 or -1 if not supported
 * void    pause(gchar *reason)
 * void    resume()
 * void    start()
 * void    stop()

Signals:

 * started()
 * stopped(gboolean interrupted)
 * paused(gchar *reason)
 * resumed()
 * progress(gchar *status, gdouble progress)
 * error(gint code, gchar *reason)

These are the basic control and information APIs we intend for miners. 
If there is anything missing here, please let me know.

Also, I was thinking about writing the new applet in vala later on. But 
before I do that, we will probably add an API in libtracker-miner which 
will list all miners available, all miners running and basically service 
all the needs to query miners from tracker-status. The information shown 
in the applet should also be available from tracker-status about each 
miner.

Did I miss anything?

--
Regards,
Martyn