Re: [jokosher-devel] Crash recovery dialog

[Laszlo Pandy <laszlok2 gmail com>]

Today I finished the implementation of the new crash recovery feature.
I would put up a screenshot, but there's not much to see. If Jokosher
crashes, the next time you open the project you will be prompted to
restore the project.

If you aren't very technically inclined, all you should take away from
this is now, when using Jokosher, (assuming there are no bugs in the
recovery code) it is almost impossible to lose data[1].

Unlike almost every other desktop application[2] I have heard of,
Jokosher no longer uses timed backups. This means when you recover a
project, you won't get your project from 5 minutes before the crash, you
will get your project the way it was at the exact time of failure.
This is done by writing each action performed to a log on disk
immediately after it is completed. It causes no perceptible delay
because:
a) We only append to the log, never reading and rewriting it from the
beginning.
b) Each action is typically around 200 characters (bytes) of data.
c) Modern operating systems have very good disk buffering, and we are
not doing anything to mess with the buffer like seeking.

Most actions are simple, such as moving an event, you log the name
"move", the event that moved and the location it moved to. However there
are a few special cases. 
* If you create a new event, and specify to copy the file to the project
dir (the default setting), Jokosher will log the fact that there is a
file in the audio dir, and when recovering that project it will use that
file instead of trying to copy the old one again.
* When downloading an event from the web, if Jokosher crashes before
download is complete, it will restart the download on recovery. However
if the download finishes successfully, on recovery it will load the
event directly from the file that was already downloaded.

Code is available in this branch:
https://code.launchpad.net/~laszlok/jokosher/crash-protection
You can inspect the logs yourself by looking at the
<project-name>.incremental file in the project's directory.

Laszlo

[1] It is possible to lose data in the case that your file system has
not written the cache to disk in a while, and then the power goes out.
However barring any OS crash, when Jokosher dies the buffer will be
flushed to disk, and everything will be recoverable.

[2] With the exception of database oriented applications. For example
Evolution uses an SQLite3 database to store mail, therefore whenever you
change something it is committed to the database almost immediately
because databases are designed to be modifiable in place. The difference
here is that Evolution is not document based. Usually document based
apps don't use databases because it is too much overhead to have each
document in a separate database file. So I still know of no document
based app in which is it impossible to lose data.

On Mon, 2008-12-22 at 11:25 -0500, Jeff Ratliff wrote:
> 2008/12/21 Laszlo Pandy <laszlok2 gmail com>:
> > Hopefully everyone has tried out, or at least had a chance to see the
> > crash recovery feature we put into 0.10. The feature automatically saves
> > any open Jokosher projects every few minutes.
> >
> > My question is, is it best to prompt the user to restore any of the
> > crashed projects? The other option is to not have a dialog at all but
> > prompt for restore when the user opens that particular project. This is
> > the way Word and OpenOffice work. They don't have a single place to keep
> > track of all the recoverable files, but when you open a file it checks
> > for filename~ or filename.bak and checks if it can be recovered.
> >
> > The reason I am putting this on the mailing list is because I require
> > discussion on this issue. *Please respond* and at least tell me if have
> > or will ever run multiple Jokosher instances at once, even if you don't
> > have any other opinions. I would like to know if restoring multiple
> > projects is a common use case, and would like to hear any other insights
> > people have.
> >
> I think checking for crash recovery on a per project basis would be
> better. Jokosher works in terms of projects. This has always been an
> important concept (working with projects rather than working with
> Jokosher), and trying to recover all crashed projects at once breaks
> this model.
> 
> We already have a folder per project, so why not store recover info
> there and only check for recovery when that project is opened? Even if
> I ran multiple instances of Jokosher and had them all crash, I'd still
> recover each project one at a time.
> 
> Just my 2 cents.
> 
>