RE: [Summary] Meta-data/filesystem-encapsulation

[Christopher Curtis <ccurtis ee fit edu>]

On Mon, 17 Aug 1998, Dries, Joseph wrote:

> > Regarding changing the filesystem, this should not be a problem.  ext2
> > already supports many "attributes" (man chattr), one of which is
> > "compressed".  Even though this isn't yet supported, the idea can be used.
> > Add an attribute "metadata" or "extended attribute" (oops, is that too
> > much OS/2?) and this flag will get set in the file's inode.  Then when
> > 
> 	This shouldn't even be an option, unless you plan on porting ext2 to
> every supported *nix as a whole, and convince all those administrators to
> convert from jfs, ufs, etc. which will NEVER happen. GNOME != Linux, even
> though it is being developed there. And OS/2 is fine, it has lots of good
> ideas to inherit.

You miss the point (which may not be in this letter but in another) - ext2
has "spare" entries in its data structure.  I'm sure other filesystems do
as well.  Any of these can be used.  ext2 should simply be the easiest.

> > Probably best is a GNOME wrapper class around fread() and fwrite() where
> > the header can be automatically prepended for supported files and/or the
> > dbm/flatfile be updated simultaneously.  And while the dbm/flatfile could
> > get out of sync, it should be pretty easy to get something like that
> > cleaned up fairly painless and easily from cron or something.
> > 
> 	This is going to be useless on other OSes. Remember, that while

This is absolutely wrong.  I don't know why so many people don't see it. 
The assumption is that all GNOME programs will use the GNOME wrappers. 
Then, these wrappers can be compiled into their own library.  Then, if you
put the GNOME wrapper library in LD_PRELOAD, functions with the same name
will be called by the wrapper library instead of the C library, thus
making all non-GNOME programs immediately GNOME aware.  This solves the
problem of commands like 'mv', 'cp', etc, orphaning object data. 

> desktop. Make the assumption even that the user's home directory is NFS
> mounted off a SunOS server. Then you will be doing the right thing.

If each end is SunOS, then they'd have to use a modified Linux NFS server
that can send and acknowledge "metadata" bits.  This is unlikely to
happen.  But it would be ideal if it did.

I'm not suggesting a be-all end-all solution by any means - simply trying
to find the ideal.  Why not give Linux an advantage (seamless integration
of fs and metadata)?  This is a new OS - *our* OS, with *our* utilitities.
Modify 'find' too to deal with the metadata.

find / -name "*.jpg" \! -metadata Thumbnail: -exec ~/mkthumb {} \;

> > The nice thing about HPFS as well (and I guess NTFS, too) is that the EAs
> > could grow or shrink so that things like thumbnails of images could be
> > stored with the file itself as an attribute.  Of course this got lost
> > being copied to floppy, etc, but GNOME will have all those same problems.
> > 
> 	The thing that people are forgetting about HPFS and NTFS is that
> while they kept extended attribute data, their OSes were assuming a single
> user with local context. You cannot make that assumption under UNIX.
> Therefore you need to keep that data in seperate files. The next problem is

Of course - I said that as well.  That's what the dbm/flatfile was for.
BUT that's also why I give the example of the thumbnail.  *This* data
should be stored with the file and not in each users' account.  Same
applies for data like "Author".  These attributes should be assigned by
the owner of the file only and not overridable.

So in that sense, the file owner needs to give metadata attributes
"inheritable" flags at least for group and other.  Maybe the author
doesn't mind if someone changes their personal view of the thumbnail, but
they do not want anyone else ever changing the 'Author' attribute.

> that you cannot easily LD_PRELOAD against that for all OSes, especially when
> someone logs into an IRIX machine and manipulates the files from there, and
> then comes back to their Solaris GNOME desktop. The linking of the metadata
> file is obviously going to be broken. The best you can do is try to handle
> that gracefully.

I don't understand ... why would it be broken?  All GNOME applications
should be using the GNOME wrappers anyway (link against them either
staticly or to ensure they load before libc if that is possible) so the
only problem with the metadata is for non-GNOME, non-LD_PRELOAD-able
systems.  And these are bigger problems that won't be solved because these
programs (mv, cp) will always orphan the data.

--
Christopher Curtis               - http://www.ee.fit.edu/users/ccurtis
                                 - System Administrator, Programmer
Melbourne, Florida  USA          - http://www.lp.org/