beagle r3536 - in trunk/beagle: BeagleClient Util beagled beagled/FileSystemQueryable

[joeshaw svn gnome org]

Author: joeshaw
Date: 2007-03-06 20:26:18 +0000 (Tue, 06 Mar 2007)
New Revision: 3536
ViewCVS link: http://svn.gnome.org/viewcvs/beagle?rev=3536&view=rev

Added:
   trunk/beagle/beagled/ExternalMetadataQueryable.cs
   trunk/beagle/beagled/NautilusMetadataQueryable.cs
Modified:
   trunk/beagle/BeagleClient/Property.cs
   trunk/beagle/Util/NautilusTools.cs
   trunk/beagle/beagled/AssemblyInfo.cs
   trunk/beagle/beagled/DumpIndex.cs
   trunk/beagle/beagled/FileSystemQueryable/FileSystemQueryable.cs
   trunk/beagle/beagled/FileSystemQueryable/LuceneNameResolver.cs
   trunk/beagle/beagled/LuceneCommon.cs
   trunk/beagle/beagled/LuceneIndexingDriver.cs
   trunk/beagle/beagled/LuceneQueryable.cs
   trunk/beagle/beagled/LuceneQueryingDriver.cs
   trunk/beagle/beagled/Makefile.am
   trunk/beagle/beagled/QueryDriver.cs
   trunk/beagle/beagled/Queryable.cs
   trunk/beagle/beagled/QueryableFlavor.cs
Log:
Mega Checkin!

Add external metadata support to beagle, and implement Nautilus metadata
support using this new system.  This commit adds support for persistent
properties, which live across document updates.  It also changes a large
block of the indexing process, replaces slow query methods with term
docs enumeration, and changes the field names for date formats since I
didn't like them before.

* NautilusTools.cs: Rewrite this to be iterator based, and extract
emblems and notes with timestamp information from the files.

* Property.cs: Add IsPersistent, which indicates whether this property
should be re-set on indexables after they are updated in the index.
Only makes sense with IsMutable also set.

* ExternalMetadataQueryable.cs: New file.  An abstract class which
implements IQueryable; it doesn't accept any queries and has no backing
Lucene index (or any other store), but can produce indexables which are
sent to a different backend.

* NautilusMetadataQueryable.cs: New file.  Extends
ExternalMetadataQueryable to actually provide Nautilus metadata to the
file system backend.  At startup, the files are crawled and metadata
attached to the files in the FSQ; as the metafiles are updated by
Nautilus, inotify events are caught and the metadata is updated in real
time.

* QueryDriver.cs: Implement simple backend dependencies, and handle them
when activating backends inside the Start() method.

* LuceneCommon.cs: Add support for persistent properties, and bump the
index major version to 18.  While I'm in here, change the field names
for the year-month and day breakdowns of dates, since I hated what they
were before.  Add a new MergeDocuments() method which moves persistent
properties from one document to another.

* LuceneQueryable.cs: Make the IndexFingerprint public.

* Queryable.cs: Add DependsOn, a field for specifying a backend to
depend on, and a IQueryable field, so that the backends can be accessed
directly from a Queryable instance.

* QueryableFlavor.cs: Add DependsOn, for setting in the attribute on
IQueryable classes.

* LuceneIndexingDriver.cs: Rewrite a large chunk of the indexing inner
loop.  We no longer fire off queries to get documents for property
change indexables, we use a term enumerator, and we also do them for
document additions (to support persistent properties).  This makes
indexing faster and use less memory, although some of the speed gains
are offset by the need to persist documents.  Merge in the persistent
properties from the old secondary doc into the new one.

* LuceneQueryingDriver.cs: Reimplement PropertyQuery() using term docs
enumeration, which should be faster in the rare cases in which this
method is called.

* FileSystemQueryable.cs: Handle non-file-move property change
indexables, for things like nautilus metadata.  Add an
ExternalToInternalUri() method, which maps from the file URIs everything
naturally sees (file:///home/joe/foo.txt) into the internal URIs the
file system backend uses (uid:HxfvSlpyq0SDFocBINgkuQ) to handle file
moves nicely.  External backends can use this method to correctly set
the URI on their indexable objects to ensure that the metadata is
correctly reflected.

* LuceneNameResolver.cs: Get rid of the SingletonCollector, it's not
needed anymore, because GetNameInfoById() and GetIdByNameAndParentId()
have been reimplemented in terms of term docs enumeration, which is
faster (always in the former function and almost always in the latter)
and uses less memory.

* AssemblyInfo.cs: Add NautilusMetadataQueryable.

* DumpIndex.cs: Add a property column for persistent properties.