Re: Document database (was Re: Writing a GNOME mail client.)

[atmczen ibm net]

Hello  Braden,

On Mon, 19 Apr 1999 05:27:07 -0400
"Braden N. McDaniel" <braden@shadow.net> wrote:

> I'd strongly recommend getting Cooper's book [1] and reading it cover to
> cover. While the examples are generally given in terms of Windows, the
> ideas are applicable anywhere, and most of them are very good. Chapter 8
> is devoted to this particular item. Note, though, that this is a UI
> design book, not a database implementation manual. So it's sparse on
> implementation details.
> [1] ISBN: 1-56884-322-4
Thank you very much for the information.  Now I only need time to
read the book.

> Here are the basics of his description:
>  * Saving is automated; theoretically, it occurs as often as possible,
> but not so often that performance is compromised. There is no explicit
> "save" per se.
>  * Saving the document *does not* clear the Undo history!
>  * Undo is sophisticated, and should maintain a significant backlog. It
> should be persistent between sessions of the application.
>  * "Create copy" and "rename" are application functions.
>  * There is a notion of "milestone copies" that users can commit to
> permanently mark valuable points in the document's revision history.
Once again, I thank you very much for this briefing, which benefits
all list members.

Because you kindly took your time to brief the list members about the
book's teaching, I do my share by explaining the implementation idea.

The basic idea is that all the changes made by the client is
unconditionally written to a change log (the journal) and sorted out
by document ID's.

The sorting/logging process can take place in a client machine or on
a networked server.  In either case, it should be a different process
>from the UI.

If there is enough CPU time slice and disk idle, the changes are
merged to update the original document.  The update process
definitely runs as a separate process with a higher level of access
privilege to the system resources.

All of the change log entries would not be cleared unless someone
(client or admin) explicitly asks for it.  This allows multiple
undo's that can go all the way back to the creation of the documents,
even though there are many document or session close's in between.

By default, updated document is written to a location (seek pointer
offset) in the disk that is different from the original one.  This
provides some safeguarding to the data.  If the write to the new
location fails, the old data is still intact.

The milestone copy can be accomplished by grouping change log entries
with a revision ID.  For instance, if you want to create a milestone
copy now, and there are 10 entries in the change log (since the
creation of the document) for then document, those change log entries
are given the revision ID 1.  The next change you make to the
document will be assigned a revision ID 2.

To render a milestone copy number 1, after making 20 more changes,
the system only needs to reverse change log entries with revision ID
2.  There is no disk space waste.

> I agree with you that CORBA seems like an excellent foundation for
> dealing with the underpinnings of such a system. I also think that if
> GNOME were to incorporate this functionality, it would shortly leapfrog
> Windows and most other OSes in terms of usability by a significant
> margin.
Yes.  Once materialized, I would not want to use anything else.  I
would even bear with lack of some other convenient features or some
hardware support.

> I like the notion of maintaining the documents in file-system files
> while using a database for the index. This seems like the best
> compromise to guard against corruption of the actual documents. As far
> as the possibility of the database being corrupted, the fact that
> Applications Crash must be taken into account, and every safeguard needs
> to be taken to ensure that this results in the absolute minimum data
> loss.
Application crash is not an issue for salvaging valid data.
Application is not allowed to touch the database.  All it is allowed
is writing to the change log.

Now, I don't claim that I can implement all this myself with a
database.  However, I know it would be a lot more work if we are to
do this over file systems.

Zen

----------/* E-Mail : <atmczen@ibm.net > */----------