Re: [Tracker] trackerd memory efficiency



ping

2009/1/23 Tshepang Lekhonkhobe <tshepang gmail com>:
On Fri, Jan 23, 2009 at 2:52 PM, Philip Van Hoof <spam pvanhoof be> wrote:
On Fri, 2009-01-23 at 14:41 +0200, Tshepang Lekhonkhobe wrote:
On Fri, Jan 23, 2009 at 2:36 PM, Carlos Garnacho <carlos imendio com> wrote:
On vie, 2009-01-23 at 14:08 +0200, Tshepang Lekhonkhobe wrote:
Hi,

README champions the daemon as memory efficient (3-30MB RAM) and I'd
like to send a patch that removes the whole paragraph since that's no
longer the case.


Could you back up such statement with valgrind logs? If that's truly not
the case, patches to make that paragraph true would be more appreciated,
really

No, but am just looking at top that tells me more of 35MB currently.

Note that top usually reports VmSize. VmSize is all of the memory being
used by the process. This is of course an irrelevant figure as that
includes the shared memory too (memory shared with other processes, like
libraries being mapped into memory already).

What you want to look at it usually VmRss. That's a far more interesting
number to know.

You can get the VmRss from the kernel by reading the /proc entries.

Valgrind will only report what malloc() & co have allocated. Also that
ain't a correct number as a kernel wont always immediately allocate the
entire requested size. Instead it creates pages as the memory that was
requested is being written to.

So if you do malloc(10M), then you won't be using 10M until you start
writing to all of the bytes of the 10M.

The extractor of tracker will for example request a rather large
allocation this way, but we assume the kernel to only really alloc as
pages are requested (to avoid having to excessively do realloc instead).

So just looking at top is definitely not a good way to measure.

What I meant was the RES column in top, just to be clear.

I anyways ran $ valgrind --leak-check=full --show-reachable=yes
--leak-resolution=high -v /usr/libexec/trackerd

and go the following output, and I don't know what it says:

==6792== LEAK SUMMARY:
==6792==    definitely lost: 36 bytes in 1 blocks.
==6792==    indirectly lost: 120 bytes in 10 blocks.
==6792==      possibly lost: 43,872 bytes in 84 blocks.
==6792==    still reachable: 6,253,647 bytes in 26,607 blocks.
==6792==         suppressed: 4,496 bytes in 110 blocks.
--6792--  memcheck: sanity checks: 29089 cheap, 220 expensive
--6792--  memcheck: auxmaps: 0 auxmap entries (0k, 0M) in use
--6792--  memcheck: auxmaps_L1: 0 searches, 0 cmps, ratio 0:10
--6792--  memcheck: auxmaps_L2: 0 searches, 0 nodes
--6792--  memcheck: SMs: n_issued      = 856 (13696k, 13M)
--6792--  memcheck: SMs: n_deissued    = 7 (112k, 0M)
--6792--  memcheck: SMs: max_noaccess  = 65535 (1048560k, 1023M)
--6792--  memcheck: SMs: max_undefined = 35 (560k, 0M)
--6792--  memcheck: SMs: max_defined   = 1348 (21568k, 21M)
--6792--  memcheck: SMs: max_non_DSM   = 849 (13584k, 13M)
--6792--  memcheck: max sec V bit nodes:    1581 (80k, 0M)
--6792--  memcheck: set_sec_vbits8 calls: 51138 (new: 1581, updates: 49557)
--6792--  memcheck: max shadow mem size:   13968k, 13M
--6792-- translate:            fast SP updates identified: 48,718 ( 88.1%)
--6792-- translate:   generic_known SP updates identified: 4,920 (  8.8%)
--6792-- translate: generic_unknown SP updates identified: 1,647 (  2.9%)
--6792--     tt/tc: 32,955,230 tt lookups requiring 50,735,061 probes
--6792--     tt/tc: 32,955,230 fast-cache updates, 10 flushes
--6792--  transtab: new        35,700 (767,467 -> 11,066,732; ratio
144:10) [0 scs]
--6792--  transtab: dumped     0 (0 -> ??)
--6792--  transtab: discarded  355 (7,022 -> ??)
--6792-- scheduler: 2,835,614,942 jumps (bb entries).
--6792-- scheduler: 29,089/43,763,705 major/minor sched events.
--6792--    sanity: 29090 cheap, 220 expensive checks.
--6792--    exectx: 12,289 lists, 11,820 contexts (avg 0 per list)
--6792--    exectx: 9,597,286 searches, 9,605,875 full compares (1,000 per 1000)
--6792--    exectx: 0 cmp2, 189 cmp4, 30,892,554 cmpAll
--6792--  errormgr: 1,500 supplist searches, 190,814 comparisons during search
--6792--  errormgr: 83 errlist searches, 199 comparisons during search


--
my place on the web:
floss-and-misc.blogspot.com




-- 
my place on the web:
floss-and-misc.blogspot.com



[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]