beagle r4042 - in trunk/beagle/beagled: . FileSystemQueryable

[dbera svn gnome org]

Author: dbera
Date: 2007-10-08 22:24:37 +0100 (Mon, 08 Oct 2007)
New Revision: 4042
ViewCVS link: http://svn.gnome.org/viewcvs/beagle?rev=4042&view=rev

Modified:
   trunk/beagle/beagled/BuildIndex.cs
   trunk/beagle/beagled/FileAttributesStore_Sqlite.cs
   trunk/beagle/beagled/FileSystemQueryable/FileSystemQueryable.cs
   trunk/beagle/beagled/SnippetFu.cs
   trunk/beagle/beagled/SqliteUtils.cs
   trunk/beagle/beagled/TextCache.cs
Log:
A database is not suitable for storing files in general. A filesystem with fixed size block suffers from external fragmentation when lots of extremely small files are stored (e.g. the gzipped textcache files beagle stores for snippets).
Here we try to balance in between by storing all files more than 4K (1 block size) on disk and others in the database. The content is the same gzipped-data as before. Drastically cuts down the storage space. Hopefully, the speed would not suffer much even with a million emails. The limit of 4K is arbitrary and can be tuned. In my home directory, most of the small text cache files were generated from emails and were well below 4K mark.
The old textcache database should be automatically purged. But nevertheless it would be a good idea to manually delete .beagle and re-index everything.
Also, I removed some redundant function calls of TextCache and used bind-variables in sqlite wherever I could.