Re: [Summary] Meta-data/filesystem-encapsulation

[Leareth <leareth geocities com>]

>> [NOTE: I don't understand this proposal. In fact, I don't understand
>> many of the sentences, I'm just pasting it from an email Leareth
>> posted. - jeske@chat.net]

Neither does anyone else, don't let it bother you...

>To summary: This is basically a global database proposal with the
>LD_PRELOAD twist.

Aeah, it's the db with a twist....

>I'm not to happy with this, because:
>-) a filename.metadata can be also copied manually quite easy by some
>   power user with standard POSIX tools :)

Metadata is either 100% transparent for all access points, with the exception
of the controling library, or it's not worth my time, speaking as a user &
developer.
>-) The metadata should be as near as possible with the data. Just consider
 
> an external SCSI harddisc that is taken from one site to another 
>one.
 
>  (With a global db, this looses the metadata in the process.)
>-) A global db is an invitation to desaster. One bad block can wreak havoc
>   the  complete database, while the same bad block with separate metafiles
>   breaks only one file.
>
>There are also disadvantages:
>* Many, potentially small, files.
>* It breaks the normal CLI UNIX abstraction of files. But so do all
>  proposals. (With the db approch the person needs to remember to extract
>  the info from the db, with directories he needs to remember that the
>  file is a directory in reality, and with .metadata he needs to remember
>  the companion file.)
>
>So my (andreas@ag.or.at)  proposal would be:
>a) Metadata is stored in a file with the same filename but an additional
>   .metadata extension.
>b) Logically speaking, the metadata is organized as a tree with data nodes
>   that have some types like (int, string, filename, etc.)
>   This would allow a lowlevel editing tool to be clever how to display
>   and edit the information.
>c) There is a standard layout for the tree. Example:
>   document/mimeType:MIMETYPE,
>   document/creator:STRING,
>   ...
>   appspecific/windowlayout/mainwindow:GEOMETRY
>d) If a file doesn't have it's own metadata, it inherits the standard
>   file type metadata.
>e) The file type is tried first by the file utility, and then as a last
>   resort by extension.
>
>Coming to the implementation, there are some points:
>*) There must be an ASCII representation. (Debugging aid, and vi is the
>   tool when desaster strikes.)
>*) Is ASCII proves to slow, some kind of libdb (or libgdbm, etc.) can
>   be used, and a utility that translates between ASCII and binary
>   representation must be provided.
>
>> Now here's we we get creative. We build both central & user databases,
>> the central with a list of files that contain the name(s) of the users
>> that need to be updated when the corisponding file is changed. Each
>> user, keeps a database of metadat on all files that have been
>> extended, includeing files from remote sites. This database will be
>> updated by a library that set & get's the data, and is updated by
>> libvfs when a file is changed through normal means such as mv. This
>This is difficult, as mv is basically a link/unlink or a copy.
>And with a copy, all you get is the fact that some program has read the
>file a, and wrote the file b. Without VERY EXPENSIVE caching it's
>difficult to be sure that a==b afterwards.
I'm of the opinion that, with some exceptions, a direct copy is one prograssive
bloce, hence if the file operations ocoure in ONE order it's a copy, other wise
it's a new file, hence new metadata.

	Order being,
	fopen file a
	fopen file b (and file b must be creating a new file.)
	if/when file b closes, it's checksumed against file a(or any other open
	files), and if equal, the meta data is copyed over.
	As moveing is handled via one system call, it's no problem.
There by caching is only initiated opening of a new file, and a moderate
incress in the close time of new files. and once small checksum stored
per file opened. Now this is not trivial, but it is reasonable for what it's
doing. I sugest one master database per file system mounted, makeing them
transportable. Hences metadata is stored per file system, not machine.

NORMAL, non metadata system.
__________                      _________
| LibC         | ----------| user apps |
^^^^^^^^^^^^                     ^^^^^^^^^^^^

Metadata enhanced system.

_____              _________                    _________
| LibC |------| metalib (1) |---------| user apps |
^^^^^^              ^^^^^^^^^^^                   ^^^^^^^^^^^
                            /\
                      ----------------
                      | Meta awear apps (2) |
                      ^^^^^^^^^^^^^^^^^^^^^

1> Where metalib is loaded useing LD_PRELOAD, and all apps are
auto "linked" against it at runtime, and any symbols that were linked
at compile time are over written by the symbols in any library listed in
the LD_PRELOAD statment. giveing you the chance to suply your own
version of std commands with hand written ones. Care must be taken
with this, as any symbols in your hand written library WILL over ride
thous in a app, or library with which it is linked. A very safe nameing
scheme must be adopted.

2> Apps the trully link(at compile time) with metalib, useing it's further
capabilitys to access & apply metadata.

Please read "Metadata, a complete, WELL DOCUMENTED, idea, and 
responce." for a more complete explaination.

--
Leareth <leareth@geocities.com>
-- You've got to lose, to know how to win.