Module Proposal for 3.0: Déjà Dup Backup Tool

[Michael Terry <mike mterry name>]

I'm not sure if this is too early, but it's been 6 months since
modules proposals opened for 2.30, so I'm guessing it's an appropriate
time for 3.0 modules?

I think GNOME could use a backup program.  There are some out there
already, but having a GNOME-blessed one would focus development
effort.

= What's my proposal? =

First, I'm specifically proposing my own pet project Déjà Dup (DD) for
inclusion in the Desktop release set.  DD is only one of many backup
programs out there. I think it's pretty sweet but would also like to
open the floor for comparisons to or inclusion of other programs.
Maybe mine isn't the best fit and overtures to other maintainers would
be appropriate.

Déjà Dup's story is that it only tries to be a data backup program for
disaster recovery. But aims to be the best little disaster recovery
program there is.  It's main website is
https://launchpad.net/deja-dup, and there is a mission statement for
more details of why it exists (http://live.gnome.org/DejaDup/Mission).

It is designed to do the "right thing" without getting in the user's
way.  It encourages backing up to a cloud service. It encourages
backing up on an automatic schedule. It is designed to target the
not-very-computer-literate. It prefers to backup too much rather than
too little. It integrates well into GNOME.

It's also basically a glorified frontend for the command-line backup
program duplicity.

Next, I will talk a little bit about where I see backups fitting into
GNOME and whether it needs such a program at all.

== User Trust ==

If the user doesn't trust their backup system, what's the point? A
backup system is only as strong as its restore, but by the time you
find out that your system fails the restore test, it's usually too
late.

Backups are tricky. We're responsible for user's data and they will be
very mad if we lose it. Maybe GNOME doesn't want to get into that
business, though one can always point to the boilerplate disclaimer of
fitness for any particular task. :)

Déjà Dup is a wrapper around the already-proven command line tool
duplicity.  (Although, Duplicity is a program under active
development.  So it's not necessarily a rock-solid
hasn't-needed-fixes-for-years tool.)  DD has a suite of automated GUI
tests. So I trust it (and use it). But these facts aren't user
visible.

I suspect user trust is a matter of time and usage. Unfortunately,
Déjà Dup hasn't been around very long (about a year and a half now).

== Why Not Just Live in the Cloud? ==

More and more of our lives are being stored remotely in Internet
services and clouds (flickr, facebook, Ubuntu One, rhapsody). This
seems to reduce the need for a backup of your data, if it all lives in
the cloud.

However, I don't think backup can ever be quite replaced. Unless a
cloud system is specifically designed to hold your data, encrypted,
with snapshots, with promises of availability, no-data-loss, and
integrated into your desktop, there is still a strong use case for
your own backup.

    * Most cloud services only keep track of your current files. If
you delete a file and later realize you need it a month ago, you're
out of luck.
    * Cloud services tend to be fragmented (different services for
different data). You don't have easy access to all your data from one
place.
    * There will likely be data that you don't trust the cloud to hold.
    * You often only put a 'weaker' version of the data in the cloud,
but keep the pristine version locally (e.g. photo upload to flickr).
    * You may have large amounts of data that aren't appropriate for upload.

Of course, there's no reason you can't store your backup files in the cloud.

I look forward to the sci-fi future where everyone uses thin clients
into server farms that offer transparent data safety and duplication.
I won't stand in that future's way for love of my own code. But in the
meantime, I think a backup program is a useful offering.

== What Does Backing Up Give Me? ==

There are already different solutions for the basic thrust of what
backups do (data protection).

    * Rollback to previous versions of files with btrfs or some
version of my-filesystem-is-a-vcs
    * Sync local files to the cloud (like Ubuntu One)

But none of those are able to meet all disaster recovery needs (of
three main sorts):

    * Revert to old version of file.
    * Recover a small number of lost (accidentally deleted) files.
    * Total system failure.

What many backup systems (and Déjà Dup specifically) provide are:

    * Rollback to previous versions
    * Automatic backups (i.e. no-intervention after initial setup --
if you have to do it manually, you're not doing it enough)
    * Incremental backups (space saving)
    * Encryption (so you can backup to non-trusted locations)
    * Ease of off-site backup (for really bad disasters like floods,
fires, or mass theft)

= Déjà Dup Details =

OK, hopefully I've convinced ya'll that a backup program is a useful
offering.  Here's more information about my specific proposal.

== Dependencies of DD ==

    * duplicity (>= 0.5.03 but newer versions enable more DD features)
    * python-boto (for Amazon S3 support)
    * valac (generally the latest version, but it's only needed for
building from tip)

These would need to be made external dependency modules?  (Except for
valac, which I believe is already one).

== Maintainership and Community ==

Despite my efforts to entice more developers, it's basically just me.
Only one other person has submitted patches that have been included.
I have several users that help test the unstable branch and report
useful bugs.

I'm biased, but I think I've been very responsive and active during
Déjà Dup's lifetime. There are frequent releases and regular
bug-fix-only and translation maintenance releases. I'm comfortable
with the amount of maintainer work that GNOME seems to require (point
releases leading up to a 6-month release).  In fact, for the past six
months, I've been releasing versions of the unstable branch along with
GNOME's schedule.

However, I suspect that inclusion into GNOME would (by virtue of a
massive amount of new users) create lots more bug activity and patch
submissions. This would create much more work for me (or at least a
sense of obligation to do more work).

Frankly, I'm content with my current DD workload and don't want to
increase it.  So I would probably scale back my feature work as more
maintainer work cropped up.  How OK is GNOME with modules that don't
add many/any features ever release?  I know there is a "Progress on a
regular basis" requirement, but I'm not sure how that's scaled for
developer team size.

Ideally I'd get a co-maintainer or some frequent developers? I'd even
be happier if someone else wanted to maintain it and I would just be a
developer on the project.  I'm curious about other maintainers'
experiences about how GNOME-inclusion leads to more developers or
maintainers.

== Resources ==

None of GNOME ftp, git, and bugzilla are used currently. Launchpad
serves all project hosting needs right now. I'm fine switching to
GNOME hosting, but am attached to Launchpad and wouldn't want to
switch until approved.

Uses GNOME wiki already.  If adopted, I would use bugzilla, git, ftp,
probably a mailing list?  I dunno, the normal new-project set of
resources.

== Adoption ==

There are some number of active users, but obviously actual statistics
are hard to come by.  There was an small article in Linux Format
magazine about it, and a little press on the Internet.

Currently packaged in Debian and Ubuntu. No RPM distros yet package
Déjà Dup (there is an open bug with a prospective spec file for
Fedora). Duplicity is basically packaged everywhere.

== License ==

DD is GPLv3+, no copyright assignment
Duplicity is GPLv2+, no copyright assignment

== GNOME Community Interaction ==

No particular overtures have yet been made. I'm excited to work with
the various GNOME teams (documentation, i18n, usability, etc). I'm a
GNOME contributor from way back and am familiar with the GNOME
community on a superficial level.

== 3.0 Readiness ==

No deprecated libraries are used.

I'm not sure how/if it would fit into the current 3.0 plan, but backup
is a missing hole in the market so to speak, so I suppose it fits in
by default?

== GNOMEness ==

    * GTK
    * Follows HIG
    * Integrates into nautilus via an extension
    * Uses session dbus 'locking'
    * Uses NetworkManager dbus API to watch network connection
    * Uses gnome-keyring to store passwords
    * Uses gio/gvfs
    * Keeps up with GNOME best practices (like X-GNOME-Fullname and the like)
    * Written in vala
    * Uses gnome-common and gnome-doc-utils

== Roadmap ==

I have several features in mind: Being able to browse your backup, a
'restore missing files' wizard, restoring a user's installed packages,
optical (DVD/CD) backup, better backup management, etc.

DD is *not* feature complete.  But what is there should be reasonably
polished.  I've historically focused on good experiences over
features.

= Testing =

If you use Ubuntu, you're in luck.  Various versions are in Jaunty,
Karmic, and Lucid.  I also make a Karmic PPA available of the latest
development version:
https://launchpad.net/~deja-dup-team/+archive/testing

Else, download the latest tarball and build like normal (you won't
need valac for this):  https://launchpad.net/deja-dup/+download

I would add it to jhbuild's proposed list, but the gnome 3.0 moduleset
doesn't exist yet.



Thoughts?
-mt