Re: [Evolution] Message Storage Limits

["Jean-Marc V. Liotier" <jim jipo com>]

On Fri, 2004-02-13 at 16:33, Victor Pechorin wrote:
> On Fri, 2004-02-13 at 11:39, Jean-Marc V. Liotier wrote:
> > Ext2/3 are more limited according to
> > http://www.linuxquestions.org/questions/archive/14/2003/09/1/42929 :
> > "the bookkeeping techniques of Ext2, such as its linked-list directory
> > implementation, do not scale well to large file systems (there is an
> > upper limit of 32,768 subdirectories in a single directory, and a "soft"
> > upper limit of 10,000-15,000 files in a single directory)"
>
> About ext3 file limit per directory:
> I have 149846 files in one directory and all works fine. So, there is no
> limit for files per directory on ext3.

Your first hand experience seems more credible than the link I fished
from Google, and it made me wonder why the author of that post mentioned
such a limit. So I dug a bit deeper and found the source of his comment.
It is the Linux kernel documentation, specifically
Documentation/filesystems/ext2.txt :

304 There is a "soft" upper limit of about 10-15k files in a single directory
305 with the current linear linked-list directory implementation.  This limit
306 stems from performance problems when creating and deleting (and also
307 finding) files in such large directories.  Using a hashed directory index
308 (under development) allows 100k-1M+ files in a single directory without
309 performance problems (although RAM size becomes an issue at this point).
310
311 The (meaningless) absolute upper limit of files in a single directory
312 (imposed by the file size, the realistic limit is obviously much less)
313 is over 130 trillion files.  It would be higher except there are not
314 enough 4-character names to make up unique directory entries, so they
315 have to be 8 character filenames, even then we are fairly close to
316 running out of unique filenames.

So there, the first paragraph clearly states that with the traditional
ext2 implementation there indeed is a performance hit from having a very
high number of files in a directory. But the user perceived impact of
that degradation depends on the CPU : the 10-15k files limit is a
relative one and thanks to ever growing cheap CPU power available, the
approximative number of files in a directory with no user unacceptable
degradation is now much higher than in the past. This is especially true
since these two paragraphs were added to
Documentation/filesystems/ext2.txt in April 2001 by Andreas Dilger
(adilger turbolinux com) (cf.
http://www.ussg.iu.edu/hypermail/linux/kernel/0104.2/1058.html) and the
CPU hardware has evolved considerably since then.

So the degradation still exist, but it is not felt anymore by most
people. That probably explains why you mention 149846 files in a
directory working fine from your point of view. But still you would
certainly benefit from a more modern file system such as Reiserfs, XFS
of JFS.

I have found that with even moderately large folders (a few thousands
messages), Evolution is noticeably sensitive to hard disk performance.
With similar CPUs and ReiserFS, the basic E-IDE disk I use in the office
is far from letting Eovlution display folders content as fast as my home
system with a pair of RAID 0 UW-SCSI mounted on /home. So do yourself a
favor and don't spoil whatever your disk's performance is with an
antiquated file system.