Re: What needs doing in the long run

[Seth Nickell <seth gnome org>]

On Sun, 2003-09-14 at 09:20, Curtis C. Hovey wrote:
> On Sun, 2003-09-14 at 01:36, Seth Nickell wrote:
> > The table layout and database aspects of Storage have been our focus
> > lately, presumably because they are most relevant to Sutra (which is a
> > more immediate goal). However, Marco has pointed out that both from a
> > self-motivation and PR standpoint it is a good idea to provide a
> > working, demo-able "desktop storage" at the same time. 
> 
> Always working code is the right way to go.  I'd rather make many rounds
> of safe changes, then introduce a risky change that really doesn't
> function right and frustrates users.  I don't think I can start on Sutra
> for a few months (when Medusa will be ready for a new DB/Schema), but I
> wanted to get my thoughts public now knoingw that I will be busy with
> Medusa for a few months.

Absolutely.

> > 1) Multi-user support
<snip>
> I've never had a problem with the user/role model in DBs, but this
> solution gives ownership to tables, not rows.  Each childnode and
> attribute would require ugo/acl column(s) with which user views are
> constructed.  This introduces some ugly version control consequences.

Yup. Oracle has the ability to have per-row ownership and permissions,
but obviously that is not a standard feature. Also, as mentioned below,
we really don't want to require postgresql authentication and would
rather use another mechanism for auth.

> > 3) Figure out how to provide accurate "file sizes" 
<snip>
> The only idea I have is that the parser that prepares the content for
> storage must count the bytes as it passes the data.  The file size is
> meta data stored as an attribute.  This isn't as easy as it sounds
> because I assume much of the content must be encoded as UTF-8 with
> variable byte sizes, or as text-friendly binary encoding (base64) which
> inflates the data size.  Filesystems deal with data size verses storage
> allocation now, where the small file takes up more space than it
> requires because of the filesystem type and block size.  It might be
> good to have a true file size and storage file size attribute.

To some extent all we really need is a ballpark number so users can
answer the basic "is this the thing that's filling my disk" question. I
think the relevance of file sizes has, to some extent, decreased over
the years. But I was thinking that for the sort of fascist installations
where sysadmins institute quotas this could get really tricky.

> > 4) Get notification feedback working 
> > Using NOTIFY and LISTEN for Postgresql, unfortunately tying us to
> > Postgresql for now, but c'est la vie. A storage daemon, as proposed in
> > (1), could potentially handle notification tasks for SQL servers that
> > don't support it. It could look at requests to modify nodes and send
> > notification events,  handle registering listeners, etc. There's no
> > reason this has to be done in the SQL server itself.
> 
> This problem sounds like we need a VFAM for our virtual filesystem. 
> Triggers might be a device that can be co-opted to do the function,
> allowing for alternatives to PG.  During an update/insert, the trigger
> calls an external function that sends a data changed message (much like
> a file changed message).  

This is using triggers to call a standard NOTIFY procedure which kicks
the client side postgresql. The problem with triggers is they need to be
able to inform something that something has changed...and they're not
standard across DBs.

> The next issue is locking.  Do some kinds of
> data require locking before changes?  Can some kinds of data use an
> optimist write flag?  

Yes, we need locking (at least, many clients will)...

-Seth