I've upgraded to 2.16 courtesy of Ubuntu feisty.
The good news is that memory consumption of beagled looks good.
The bad news is that IndexHelper is a huge CPU pig.
Here's an strace of what it does over and over again:
[pid 31822] read(29, "", 4687) = 0
[pid 31822] read(29, "", 4687) = 0
[pid 31822] read(29, "", 4687) = 0
[pid 31822] read(29, "", 4687) = 0
[pid 31822] read(29, "", 4687) = 0
[pid 31822] read(29, "", 4687) = 0
[pid 31822] read(29, "", 4687) = 0
[pid 31822] read(29, "", 4687) = 0
[pid 31822] read(29, "", 4687) = 0
[pid 31822] read(29, "", 4687) = 0
[pid 31822] read(29, "", 4687) = 0
[pid 31822] read(29, "", 4687) = 0
[pid 31822] read(29, "", 4687) = 0
[pid 31822] read(29, "", 4687) = 0
[pid 31822] read(29, "", 4687) = 0
[pid 31822] read(29, "", 4687) = 0
[pid 31822] read(29, "", 4687) = 0
[pid 14176] <... nanosleep resumed> NULL) = 0
[pid 14180] <... futex resumed> ) = -1 ETIMEDOUT (Connection timed out)
[pid 14176] time(NULL) = 1171031096
[pid 14176] semop(32768, 0xb79802ca, 1) = 0
[pid 14176] semop(32768, 0xb79802da, 1) = 0
[pid 14176] semop(32768, 0xb79802da, 1) = 0
[pid 14176] semop(32768, 0xb79802ca, 1) = 0
[pid 14176] waitpid(14175, 0xb79802a4, WNOHANG) = -1 ECHILD (No child processes)
[pid 14176] nanosleep({10, 0}, <unfinished ...>
[pid 31822] read(29, <unfinished ...>
[pid 14180] gettimeofday( <unfinished ...>
[pid 31822] <... read resumed> "", 4687) = 0
[pid 14180] <... gettimeofday resumed> {1171031097, 117680}, NULL) = 0
[pid 31822] read(29, <unfinished ...>
[pid 14180] futex(0xb791490c, FUTEX_WAKE, 1 <unfinished ...>
[pid 31822] <... read resumed> "", 4687) = 0
[pid 14180] <... futex resumed> ) = 0
[pid 31822] read(29, <unfinished ...>
[pid 14180] clock_gettime(CLOCK_REALTIME, <unfinished ...>
[pid 31822] <... read resumed> "", 4687) = 0
[pid 14180] <... clock_gettime resumed> {1171031097, 573677710}) = 0
[pid 31822] read(29, <unfinished ...>
[pid 14180] gettimeofday( <unfinished ...>
[pid 31822] <... read resumed> "", 4687) = 0
[pid 14180] <... gettimeofday resumed> {1171031097, 617792}, NULL) = 0
[pid 31822] read(29, <unfinished ...>
[pid 14180] futex(0xb791490c, FUTEX_WAKE, 1 <unfinished ...>
[pid 31822] <... read resumed> "", 4687) = 0
[pid 14180] <... futex resumed> ) = 0
[pid 31822] read(29, <unfinished ...>
[pid 14180] clock_gettime(CLOCK_REALTIME, <unfinished ...>
[pid 31822] <... read resumed> "", 4687) = 0
[pid 14180] <... clock_gettime resumed> {1171031097, 618414087}) = 0
[pid 31822] read(29, <unfinished ...>
[pid 14180] futex(0xb7914928, FUTEX_WAIT, 425445, {0, 98585913} <unfinished ...>
[pid 31822] <... read resumed> "", 4687) = 0
[pid 31822] read(29, "", 4687) = 0
[pid 31822] read(29, "", 4687) = 0
[pid 31822] read(29, "", 4687) = 0
[pid 31822] read(29, "", 4687) = 0
[pid 31822] read(29, "", 4687) = 0
[pid 31822] read(29, "", 4687) = 0
[pid 31822] read(29, "", 4687) = 0
[pid 31822] read(29, "", 4687) = 0
[pid 31822] read(29, "", 4687) = 0
[pid 31822] read(29, "", 4687) = 0
[pid 31822] read(29, "", 4687) = 0
It does this in a tight loop, over and over again, which is what's
eating the CPU.
File descriptor 29 has a file open on it:
beagled-h 14175 brian 29r REG 0,19 4687 18885 /data/jennifer_pc/c.bak/Program Files/icq/AteBrowser/96201529/cache/icq82F7.TMP (linux:/mnt/data)
Last thing in the .beagle/Log/current-IndexHelper:
20070209 00:21:34.7238 14175 IndexH DEBUG: +file:///data/jennifer_pc/c.bak/Program Files/icq/AteBrowser/96201529/cache/icqA393.TMP
20070209 00:21:34.7801 14175 IndexH DEBUG: +file:///data/jennifer_pc/c.bak/Program Files/icq/AteBrowser/96201529/cache/icqB035.TMP
20070209 00:21:34.7854 14175 IndexH DEBUG: +file:///data/jennifer_pc/c.bak/Program Files/icq/AteBrowser/96201529/cache/icq82F7.TMP
20070209 00:21:35.8920 14175 IndexH DEBUG: Helper Size: VmRSS=30.9 MB, size=2.53, 38.2%
20070209 00:51:37.4245 14175 IndexH DEBUG: No activity for 30.0 minutes, shutting down
20070209 00:51:37.4275 14175 IndexH INFO: Shutdown requested
20070209 00:51:37.4728 14175 IndexH DEBUG: CancelIfBlocking Beagle.Daemon.ConnectionHandler
20070209 00:51:37.5144 14175 IndexH DEBUG: Server '/home/brian/.beagle/socket-helper' shut down
20070209 00:51:37.5523 14175 IndexH DEBUG: (1) Waiting for 1 worker...
20070209 00:51:37.5525 14175 IndexH DEBUG: waiting for HandleConnection (1600)
Strange that the IndexHelper pid is 14175 yet the pid I have sitting in
a the loop is 31822. The above strace is indeed a strace of pid 14175.
The last few exceptions in
~/.beagle/Log/2007-02-08-22-09-18-IndexHelperExceptions are:
20070209 00:18:31.8029 14175 IndexH WARN EX: Unable to filter /data/jennifer_pc/c/WINDOWS/Temporary Internet Files/Content.IE5/MP6FSL4V/actsetup[1].tgz:
20070209 00:18:31.8029 14175 IndexH WARN EX: ICSharpCode.SharpZipLib.GZip.GZipException: Error baseInputStream GZIP header, first byte doesn't match
20070209 00:18:31.8029 14175 IndexH WARN EX: at ICSharpCode.SharpZipLib.GZip.GZipInputStream.ReadHeader () [0x00000]
20070209 00:18:31.8029 14175 IndexH WARN EX: at ICSharpCode.SharpZipLib.GZip.GZipInputStream.Read (System.Byte[] buf, Int32 offset, Int32 len) [0x00000]
20070209 00:18:31.8029 14175 IndexH WARN EX: at ICSharpCode.SharpZipLib.Tar.TarBuffer.ReadRecord () [0x00000]
20070209 00:18:31.8029 14175 IndexH WARN EX: at ICSharpCode.SharpZipLib.Tar.TarBuffer.ReadBlock () [0x00000]
20070209 00:18:31.8029 14175 IndexH WARN EX: at ICSharpCode.SharpZipLib.Tar.TarInputStream.GetNextEntry () [0x00000]
20070209 00:18:31.8029 14175 IndexH WARN EX: at (wrapper remoting-invoke-with-check) ICSharpCode.SharpZipLib.Tar.TarInputStream:GetNextEntry ()
20070209 00:18:31.8029 14175 IndexH WARN EX: at Beagle.Filters.FilterArchive.GetNextEntryTar () [0x00000]
20070209 00:18:31.8029 14175 IndexH WARN EX: at (wrapper delegate-invoke) System.MulticastDelegate:invoke_FilterArchive/ArchiveEntry ()
20070209 00:18:31.8029 14175 IndexH WARN EX: at Beagle.Filters.FilterArchive.DoPullSetup () [0x00000]
20070209 00:18:31.8029 14175 IndexH WARN EX: at Beagle.Daemon.Filter.Open (System.IO.FileSystemInfo info) [0x00000]
20070209 00:18:32.2599 14175 IndexH WARN EX: Unable to filter /data/jennifer_pc/c/WINDOWS/Temporary Internet Files/Content.IE5/MP6FSL4V/CUNPROT2[1].tgz:
20070209 00:18:32.2599 14175 IndexH WARN EX: ICSharpCode.SharpZipLib.GZip.GZipException: Error baseInputStream GZIP header, first byte doesn't match
20070209 00:18:32.2599 14175 IndexH WARN EX: at ICSharpCode.SharpZipLib.GZip.GZipInputStream.ReadHeader () [0x00000]
20070209 00:18:32.2599 14175 IndexH WARN EX: at ICSharpCode.SharpZipLib.GZip.GZipInputStream.Read (System.Byte[] buf, Int32 offset, Int32 len) [0x00000]
20070209 00:18:32.2599 14175 IndexH WARN EX: at ICSharpCode.SharpZipLib.Tar.TarBuffer.ReadRecord () [0x00000]
20070209 00:18:32.2599 14175 IndexH WARN EX: at ICSharpCode.SharpZipLib.Tar.TarBuffer.ReadBlock () [0x00000]
20070209 00:18:32.2599 14175 IndexH WARN EX: at ICSharpCode.SharpZipLib.Tar.TarInputStream.GetNextEntry () [0x00000]
20070209 00:18:32.2599 14175 IndexH WARN EX: at (wrapper remoting-invoke-with-check) ICSharpCode.SharpZipLib.Tar.TarInputStream:GetNextEntry ()
20070209 00:18:32.2599 14175 IndexH WARN EX: at Beagle.Filters.FilterArchive.GetNextEntryTar () [0x00000]
20070209 00:18:32.2599 14175 IndexH WARN EX: at (wrapper delegate-invoke) System.MulticastDelegate:invoke_FilterArchive/ArchiveEntry ()
20070209 00:18:32.2599 14175 IndexH WARN EX: at Beagle.Filters.FilterArchive.DoPullSetup () [0x00000]
20070209 00:18:32.2599 14175 IndexH WARN EX: at Beagle.Daemon.Filter.Open (System.IO.FileSystemInfo info) [0x00000]
20070209 00:18:38.9630 14175 IndexH WARN EX: Unable to filter /data/jennifer_pc/c/WINDOWS/Temporary Internet Files/Content.IE5/MP6FSL4V/media5631[1].jpe:
20070209 00:18:38.9630 14175 IndexH WARN EX: System.Exception: Invalid marker found 71
20070209 00:18:38.9630 14175 IndexH WARN EX: at Beagle.Util.JpegHeader+Marker.Load (System.IO.Stream stream) [0x00000]
20070209 00:18:38.9630 14175 IndexH WARN EX: at Beagle.Util.JpegHeader.Load (System.IO.Stream stream, Boolean metadata_only) [0x00000]
20070209 00:18:38.9630 14175 IndexH WARN EX: at Beagle.Util.JpegHeader..ctor (System.IO.Stream stream) [0x00000]
20070209 00:18:38.9630 14175 IndexH WARN EX: at Beagle.Filters.FilterJpeg.PullImageProperties () [0x00000]
20070209 00:18:38.9630 14175 IndexH WARN EX: at Beagle.Filters.FilterImage.DoPullProperties () [0x00000]
20070209 00:18:38.9630 14175 IndexH WARN EX: at Beagle.Daemon.Filter.Open (System.IO.FileSystemInfo info) [0x00000]
20070209 00:18:48.9093 14175 IndexH WARN EX: Unable to filter /data/jennifer_pc/c/WINDOWS/Temporary Internet Files/Content.IE5/MUSGC9PL/identcdm[1].tgz:
20070209 00:18:48.9093 14175 IndexH WARN EX: ICSharpCode.SharpZipLib.GZip.GZipException: Error baseInputStream GZIP header, first byte doesn't match
20070209 00:18:48.9093 14175 IndexH WARN EX: at ICSharpCode.SharpZipLib.GZip.GZipInputStream.ReadHeader () [0x00000]
20070209 00:18:48.9093 14175 IndexH WARN EX: at ICSharpCode.SharpZipLib.GZip.GZipInputStream.Read (System.Byte[] buf, Int32 offset, Int32 len) [0x00000]
20070209 00:18:48.9093 14175 IndexH WARN EX: at ICSharpCode.SharpZipLib.Tar.TarBuffer.ReadRecord () [0x00000]
20070209 00:18:48.9093 14175 IndexH WARN EX: at ICSharpCode.SharpZipLib.Tar.TarBuffer.ReadBlock () [0x00000]
20070209 00:18:48.9093 14175 IndexH WARN EX: at ICSharpCode.SharpZipLib.Tar.TarInputStream.GetNextEntry () [0x00000]
20070209 00:18:48.9093 14175 IndexH WARN EX: at (wrapper remoting-invoke-with-check) ICSharpCode.SharpZipLib.Tar.TarInputStream:GetNextEntry ()
20070209 00:18:48.9093 14175 IndexH WARN EX: at Beagle.Filters.FilterArchive.GetNextEntryTar () [0x00000]
20070209 00:18:48.9093 14175 IndexH WARN EX: at (wrapper delegate-invoke) System.MulticastDelegate:invoke_FilterArchive/ArchiveEntry ()
20070209 00:18:48.9093 14175 IndexH WARN EX: at Beagle.Filters.FilterArchive.DoPullSetup () [0x00000]
20070209 00:18:48.9093 14175 IndexH WARN EX: at Beagle.Daemon.Filter.Open (System.IO.FileSystemInfo info) [0x00000]
Interestingly enough that last file (to just choose one):
/data/jennifer_pc/c/WINDOWS/Temporary Internet Files/Content.IE5/MUSGC9PL/identcdm[1].tgz
is not in fact a compressed tarball but rather an MS Cabinet file:
$ file "/data/jennifer_pc/c/WINDOWS/Temporary Internet Files/Content.IE5/MUSGC9PL/identcdm[1].tgz"
/data/jennifer_pc/c/WINDOWS/Temporary Internet Files/Content.IE5/MUSGC9PL/identcdm[1].tgz: Microsoft Cabinet archive data, 232 bytes, 1 file
Not sure what else I can tell you. I will leave the process spinning
for a little while in case there is anything further I can report on.
b.
--
My other computer is your Microsoft Windows server.
Brian J. Murrell
Attachment:
signature.asc
Description: This is a digitally signed message part