Meta Data Monkey Wrenches

["Robert J. Slover" <robert slover Rose-Hulman Edu>]

I'd like to point out a potential problem and make a suggestion.  The
post is quite long, as I got to rambling a 
bit and wasted (?) my lunch hour.

I don't know what other people use/intend to use metadata for,
but for me type/kind/creator/icon is the least of it.  I like to attach
notes to things.  The Mac supports *some* of this.
There is a 'comment' block that I can add information to in the GetInfo
panel, and I can colorize my icons to flag certain things.  I don't
think that this is particularly a function of the Mac filesystem, since
the information is easily lost by rebuilding the desktop file (read:
database).  This is supposedly fixed in the newest versions of the OS.

There are several nifty things that get done with this on the Mac...one
of my favorites is a Netscape thing.  If I see a jpeg that I like and
rip it off of the web page and drop it into a folder (gotta love that) I
can later go back and do a
Command-I (GetInfo) and the URL is stuffed into the comment block.  I
sometimes use this instead of bookmarks, since people neglect to name
their pages and the subject of a URL is sometimes difficult to discern. 
A picture is worth a
thousand words, they say.  I've got a folder of jpeg's and gif's whose
icons are thumbnails...I have sort of a visual memory...makes it easy.

But I digress.  Point is, there are a lot of uses for MetaData, and
there will be more that have not yet been anticipated.  Even the
Macintosh provides poor, single user-oriented, support for MetaData. 
There's the first Monkey-wrench...Unix in almost any flavor is *not*
single user.  This guarantees the need for a database of sorts, even if
it is flat-file or directory tree
based.

Here's the second Monkey-wrench to be thrown into the works:  GNOME
purports to be Network-Aware.  This makes the document space (already
convoluted with symlinks) *no longer limited to the filesystem*.  The
most wonderful feature of GMC
is the VFS support...I'd like all Gnome apps to be able to take
advantage of that.  I'd also like to have my MetaData with objects in
those VFS spaces.  Then, let's not forget Web pages, etc.  As for
convoluted:

/afs/rose-hulman.edu/Users/faculty/slover/Public/HTML/index.html
/home/slover/samba/Public/HTML/index.html
ftp://slover@bogus.rose-hulman.edu/Public/HTML/index.html
http://www.rose-hulman.edu/~slover/index.html

These are just 4 possible paths to my web page.  There are several
more.  This might be a poor example, since it is pretty easy to write a
regexp that would identify ~/index.html and those other derivatives as
my home page.  Other documents in other paths would not necessarily be
too easy.

My points:

1) Anything that depends on the filesystem to support storing
   this information is broken from the start, as the document
   space is greater than the local file system.

2) Storing the meta-content local to the file within the
   filesystem is troublesome at best, and is a single-user
   approach.

3) GNOME is out in front on this one.  Everything else is still
   tied to the single-user, objects-occupy-a-single-namespace
   paradigm.  If the GNOME resolution is useful, portable,
   and simple, it stands a chance of being *the* resolution.

My thoughts:

I think that the local-content should be nothing more than an identity
attribute...sort of an 'object key'.  Ideally, it should live *inside*
the file (for new filetypes, or old ones that support comments or tags)
where it cannot get disassociated.  Less ideally, it should live locally
to the file (the .info approach).  Least ideally, perhaps regexp's and
hashes.  I think all should be supported, with the knowledge that only
the first is shell and filesystem 'safe'.

An object-key cache is probably appropriate to keep around, to
save opening a file each time the file browser wants to find
the object key to reference an icon.

My thinking is that this key should be something similar to the DCE
UUID.  Code is freely available on the OSF web site.

Related thoughts:

Once an object has been given a 'signature' or object-key, no new key
need ever be assigned.  There can be a per-user database and perhaps a
system wide database (and an internet-wide database?) which stores
meta-information.  I also get the gut feeling that the bulk of
interesting file types are documents and scripts which will in some form
support embedding a UUID string.  I read some comment about 'modifying
the file contents scares the bejesus outa me' and I'd agree. But
including a UUID tag or comment in a .tiff or .xpm when ee saves it is
something I would consider sane to do.  If we provided a 'create_uuid'
command-line utility and corresponding applet to manufacture ID's upon
request, I'd certainly use one or the other and paste the results into a
comment in my
scripts, making them meta-capable.

For some things, I *would* consider it safe to write a script to add
these object keys automagically...PDF's, Postscript Files, certain
scripts, for instance.

It'd sure be nice to browse through the 80 or so off-the-cuff Perl
scripts in my ~/tools directory with GMC and be able to write (and later
read) a short comment describing what they do instead of grepping
through or opening each one every six
months while thinking, "I know I have an ASCII to EBCDIC converter here
_somewhere_".

Regardless, on my system at least, the files I move around most
are the 'interesting' ones: documents, scripts, web pages, graphics. 
These, save for some graphics formats, can all support comments in some
way internally.  The interesting ones that don't let me alter their
structure too much tend to live in static places and regexp's are
probably good enough:
'/usr/local/etc/httpd/logs/access.log', '/var/log/messages'.

I'd also then consider it sane to submit a patch to the Apache guys that
allowed Apache to use the Gnome libraries to extract the UUID if it
exists in order to add a X-Meta-UUID: mime tag to everything it serves
up...maybe this would eventually make it into an HTTP spec.  Then it'd
be nice to see that the comment I attached to someone's URL is still
associated with it later when I encounter it at a mirror or somewhere in
AFS-land, or simply when the web page moves.

Anyway, just some thoughts.

--Robert

P.S.  I've been away from the Gnome list for over a week, and
      owe some e-mails to some others.  I've not fully digested
      everything that is going on....I saw some discussion
      between Elliot and Miguel regarding Orbit and network
      dynamism.  I'd suggest that perhaps (don't know) there
      are DCE related papers on the subject with possible
      solutions?  I seem to remember reading something a while
      back about 'end point mapping' and how to handle it when
      an object (machine, person) moves between cells...