[Deja-dup-hackers] GSOC wrapup

[Urban Škudnik <urban skudnik gmail com>]

Hey guys,

as you know Google Summer of Code came to a finish this Monday and I  
thought that it would be a good idea to post a summary of work done -  
what was accomplished, things that I learned and areas which require  
additional work or improvements.

First, the work done.

As noted, I managed to complete the restoration of deleted files,  
which took the rough first month and a half or so. Personally I wished  
that this could be accomplished faster but as it turned out various  
small issues brought a project to virtual standstill for days (e.g.  
connecting signals emitted by UI designed in Glade, in retrospective  
completely trivial), but as it turned out after I figured them out and  
got a bit of a feeling with the project itself, Vala and development  
tools development went on rather steadily and so by the mid July the  
first part was mostly finished.

At this point I went to see to Europython to learn a few tips'n'tricks  
from the pros (well, as it turned out, there were more than just a  
few ;)) while Michael reviewed the branch.

After returning, I first started to work on a partial restore but had  
to quickly switch back to "restore missing" branch to fix various  
glitches with code and polish interface to make sure that the thing  
could be shipped to users with Gnome 3 that was, at the time, still  
planed for release in October. Although Gnome later decided to ship  
Gnome 3 in March, I finished the polishing up and can probably say  
that the thing can be shipped to users.

I still owe Michael testcases for my branch because in the last week  
of GSOC I couldn't quite get around testing functions/base that we  
have and decided to throw together the prototypes that I had for  
partial restore. With partial restore I also implemented a cache that  
will enable us much faster retrieval times of file history (which will  
bypass the need to scan through the entire backup history every time).

Current state of partial restore is that we have a working listing of  
files and some basic browsing (if anyone would like to actually  
restore files, it does not require much more work and I can probably  
add that rather quickly), but will require much more work - both  
partial restore itself and cache (for example, I still don't mark if a  
file was deleted). Currently (after-GSOC) I am developing a widget for  
breadcrumbs that Gnome still doesn't ship with (but plans to with  
Gnome 3 although as far as I know they still don't have anyone  
assigned to this yet).

If that was a short summary of what was accomplished, next thing to  
look at was what I learned.

Besides learning Vala basically from scratch and getting a hang of  
what can be done with it and how and getting a hang for the existing  
code base that Michael produced I also got a lot of knowledge about  
Gtk (especially in the last couple of weeks when I started to look at  
how to design my own widgets, which, as it appears, is sometimes  
characterized as "pushing it" :>) and, while developing cache, a bunch  
of new knowledge about data structures (B+ for the win! ;)) and key- 
value storages (since, at the moment, we are using Tokyo Cabinet [1])  
with which I haven't worked up to this point since most of my past  
projects never worked with that much data (we are talking about  
million(s) of rows of data on an average computer).

SQLite, which was also considered, was dumped because of a bit of an  
awkward working in Vala and because it appears that it would be much  
slower than KV database. If we decide at the later date that we  
require more functionality switching shouldn't be too hard to do.

As already pointed out partial restore requires much more work and  
will probably not be included in 10.10 but I can see it shaping up  
nicely for 11.04. I hope that in a near future we can also get some  
more developers on board since although it looks as though its a  
rather simple small program/project (personally I have to admit to  
underestimating it a bit at the very start), its development is not  
that simple. With more developers we will also be able to improve on  
more things and fix more bugs.

One thing maybe to consider is if we can get anyone from duplicity  
team interested in DD as well (following the reasoning that although  
duplicity is great for backups its not great for home users and if you  
really want it to go mainstream DD is just the project to do it). And  
with DD we have to target mainstream above all. It is also true that  
duplicity itself will require a lot more work so I can see convergence  
of both teams in the long run (but maintaining the separation of  
projects so that people will be able to use duplicity without DD).

My personal plan is to stick around the project and help out as much  
as I can but unfortunately I can't make any guarantees after October  
since school will start again and I will probably have to find a part  
time job. I talked a bit with Michael whether or not Canonical can  
support development efforts in the future but it seems that Canonical  
rarely supports such projects either with developers or by hiring  
contractors. A part of this can probably also be attributed to the  
fact that for the time being they are a lot more focused on enterprise  
sector.

Plans for short term future from me can therefore be shortly  
summarized in finishing partial restore with a proper cache (because  
duplicity will probably not switch to some other archiving format any  
time soon) and see where we need to go on from then.

At least for myself I can say that I had lots of fun and learned a  
bunch of new stuff and hope that I can keep working on these things in  
the future. I also hope I didn't introduce any new bugs to DD's  
existing code base but will of course fix 'em if warranty seal hasn't  
been broken ;)))

With some luck I hope to see you guys at UDS ;)

Cheers,
Urban

[1] http://fallabs.com/tokyocabinet/