Re: GSoC 2011: Synchronizing F-Spot data

[Simon Lindgren <simon n lindgren gmail com>]

sön 2011-03-27 klockan 00:42 +0100 skrev Simon Lindgren:
> Hello list,
> 
> This is an evolution of my previous GSoC mails, with a much less
> ambiguous formulation. Obviously, if you disagree with my decisions or
> think I should clarify something, please tell me :)
> 
> Now I need to find a mentor.
> 

I have gotten some feedback on this list and on various IRC channels, so
I have updated my application. Also, I am still in need of a mentor and
since I still have not received any response from any of the F-Spot
maintainers (as noted in the MAINTAINERS file), I am CC:ing them
explicitly this time.

I am hoping to submit this via the GSoC site this weekend, but I still
need to code up a patch to prove my worthiness :)

Changes:
 * Some notes on why I consider Flickr, Amazon S3 and Conduit
problematic for implementing this particular task.
 * Add more things I need to look into before the project
 * Some more interesting functionality that may not be fully
implementable in one summer.


GSoC 2011: Synchronizing F-Spot data
====================================
By Simon Lindgren <simon n lindgren gmail com>

DRAFT 2

Abstract
--------
I'm a long time user of the F-Spot photo manager. I use it to tag images
for easier retrieval later. Currently, F-Spot stores its data in a SQLite
database in the users home directory. This has a few drawbacks, namely

 * The data is only available on one computer, and therefore:
 * It is difficult to work with the data at geographically separate locations

In my particular case, I want to browse and edit my images on my main
computer, which is a desktop. When traveling I only have my laptop with me
and thus I do not currently have access to my F-Spot database. A workaround
is to copy the database between the computers, but that is tedious and error
prone. This project aims to solve this problem by introducing a synchronization
solution capable of synchronizing the F-Spot data model in its entirety
across multiple F-Spot installations.

Requirements
------------
The synchronization must:
 * support synchronization of all current metadata
 * be flexible enough to allow future additions of data in
   a backwards compatible way
 * Support conflict detection and resolution

It would also be nice if it could
 * support additional data that might be added by addins
 * support lightweight transfer of on-file metadata changes
It might be too much to implement those last two things this summer though.

Technical approach
------------------
I have looked at some different approach to the problem, including using
a web based storage service like Amazon S3 or a Flickr Pro account, but
those kinds of approaches will probably make it hard to handle the
F-Spot-specific parts of the metadata, like versions and the tag tree.
I have also looked at Gnome Conduit, but my estimation there is that
it wouldn't solve that many problems. It would most likely add a few
of it's own too, like dealing with both Python and C# and it would make
integration into F-Spot harder to accomplish.
Therefore, my current thinking is that I will implement this as
a remote repository of the F-Spot data that support pushing and pulling
data to/from the repository and also some extra functionality to simplify
synchronization.
To communicate with the server an XML-based REST api will be used. XML makes it
easy to integrate XMP metadata, which I believe will cover much of the
per-image data. An addin to F-Spot keeps the local
F-Spot database up to date with the server so that F-Spot can continue to
use the already existing queries, and load images quickly from the local
harddrive. This addin will also track changes to the data and manage
synchronizing the changes back to the server.

Interesting extra features
--------------------------
These are extra features that might be nice to support, if there is time.
 * mhutch suggested that a concept like catalogs in Lightroom might be useful.
   Basically, this would be a way to only sync selected pictures.
   It could be implemented as a tag query, to tie in to existing F-Spot concepts.
 * Partial local cache. Perhaps this overlaps a bit with the above, but it
   transform the local pictures folder into a cache containing the most recently
   used files. Reduces initial bandwidth requirements and used storage space. 

Deliverables
------------
The deliverables of the project would be:
 * An addin for F-Spot to support synchronization
 * (if needed) patches for F-Spot to support the addin
 * A specification of the REST api and the data format
 * A synchronization server implementation

Preliminary Schedule
--------------------
Before May 23		Familiarize myself with the F-Spot codebase,
			Read Mono.Addins docs,
			Read up on file synchronization and
			conflict handling methods
			Read up on Unison
			Read through the Tomboy Snowy protocol for inspiration
May 23 	- May 26	Draft the design
			(serverside data model, REST api, plugin structure,
			decide on file synchronization approach, etc)
			This will be a little slow though, due to my exams
May 27	- June 1	School examinations
June 2	- June 5	Finish initial design,
			Make neccessary adjustments to F-Spot
June 6	- June 19	Implement a basic working system to the state
			that images and metadata can be uploaded.
June 20 - June 26	Implement downloading images and metadata.
June 27	- July 3	Implement authentication (I intend to code in a
			"get it to work" mode up to this point, but ofcourse
			with authentication and authorization in mind)
July 4	- July 17	Implement the rest of full synchronization,
			including conflict detection
July 15			Mid-term evaluation deadline.
July 18	- July 31	Implement automatic merging of changes
Aug 1	- Aug 14	Buffer time, in case of delays
Aug 15	- Aug 22	Bug fixing, testing, documentation and cleanups

About Me
--------
My name is Simon Lindgren and I am a third year Computer Science student
at Linköpings Unversitet (the Linköping University). I have used C# and mono
extensively for small homegrown experimentations, and I have gained a fair bit
of knowledge of how various .NET libraries works. I have previously looked
through the F-Spot code, although that was before the big reorganization.
I have also written a Finance::Quote module in perl. It is currently living
in it's own branch, at
https://github.com/pfenwick/finance-quote/blob/Morningstar/lib/Finance/Quote/Morningstar.pm.
Gnome patch is TBD.

-- 
Simon Lindgren