Re: Making metadata storage SQL-driven

Alexander Larsson wrote:
On Thu, 2005-09-01 at 12:16 +0200, Christian Neumair wrote:

I really wonder why we rely on files for our metadata. It has issues [1]
where a synchronous fopen and read or write operation would really take
too long, requiring us to schedule reads/writes. Without doing any
performance measures, I think this could significantly speed up our
metadata code for big directories. I think we should do the same as
beagle, and rely on a tiny SQL server, doing all our metadata operations
synchronously. I'm not a big fan of EAs, since they seem to be some
different flavors and inconsistencies among various implementations.


I don't understand why a SQL server would be any slower than read() and
write(). I mean, it would still end up doing read/write calls. It seems
to me that all it does is highly complicating a simple key/value system.
And furthermore, doing i/o synchronously is definitely a no-no since not
doing that is at the core of the nautilus design.

I'm not saying the current system is great, but I don't see using an SQL
database as a better solution than just fixing stuff to be better.
Databases tend to have all sorts of complications like on-disk format
changes, complexity of installation, opaque file-formats, locking issue
with shared homedirs, fragility (if some part goes bad the whole db can
be corrupt), etc. Furthermore, we wouldn't have much use for most of the
features you get from an SQL database.

thats true for some DBs but not all.

For instance the embedded mysql needs no set up and it will work out of the box (its just a shared library) so installation and setup is a non-issue. It has no locking issues as the INNODB stuff in mysql is multi generational so readers and writers dont block each other and its totally safe as a write generates a new copy of a record and the commit on it is atomic so even of there's a power failure during a write corruption will not occur to anything committed.

With a dbus interface all access can be both sync or async so I dont see a problem there.

Also you need multiple tables to store or use RDF and contextual data which is highly relational in nature and cannot be efficiently stored in a single key value table.

My plan is to have a gnome-vfs driver to it so you can use nautilus to contextual browse your file metadata so you could for example enter a folder called "emails from bob" which would show corresponding emails etc. Contextual data is very hierarchical so this would be a good case for it.

Mr Jamie McCracken

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]