Re: [Tracker] trackerd memory efficiency

[Tshepang Lekhonkhobe <tshepang gmail com>]

ping

2009/1/23 Tshepang Lekhonkhobe <tshepang gmail com>:
> On Fri, Jan 23, 2009 at 2:52 PM, Philip Van Hoof <spam pvanhoof be> wrote:
> > On Fri, 2009-01-23 at 14:41 +0200, Tshepang Lekhonkhobe wrote:
> > > On Fri, Jan 23, 2009 at 2:36 PM, Carlos Garnacho <carlos imendio com> wrote:
> > > > On vie, 2009-01-23 at 14:08 +0200, Tshepang Lekhonkhobe wrote:
> > > > > Hi,
> > > > >
> > > > > README champions the daemon as memory efficient (3-30MB RAM) and I'd
> > > > > like to send a patch that removes the whole paragraph since that's no
> > > > > longer the case.
> > > >
> > > > Could you back up such statement with valgrind logs? If that's truly not
> > > > the case, patches to make that paragraph true would be more appreciated,
> > > > really
> > >
> > > No, but am just looking at top that tells me more of 35MB currently.
> >
> > Note that top usually reports VmSize. VmSize is all of the memory being
> > used by the process. This is of course an irrelevant figure as that
> > includes the shared memory too (memory shared with other processes, like
> > libraries being mapped into memory already).
> >
> > What you want to look at it usually VmRss. That's a far more interesting
> > number to know.
> >
> > You can get the VmRss from the kernel by reading the /proc entries.
> >
> > Valgrind will only report what malloc() & co have allocated. Also that
> > ain't a correct number as a kernel wont always immediately allocate the
> > entire requested size. Instead it creates pages as the memory that was
> > requested is being written to.
> >
> > So if you do malloc(10M), then you won't be using 10M until you start
> > writing to all of the bytes of the 10M.
> >
> > The extractor of tracker will for example request a rather large
> > allocation this way, but we assume the kernel to only really alloc as
> > pages are requested (to avoid having to excessively do realloc instead).
> >
> > So just looking at top is definitely not a good way to measure.
>
> What I meant was the RES column in top, just to be clear.
>
> I anyways ran $ valgrind --leak-check=full --show-reachable=yes
> --leak-resolution=high -v /usr/libexec/trackerd
>
> and go the following output, and I don't know what it says:
>
> ==6792== LEAK SUMMARY:
> ==6792==    definitely lost: 36 bytes in 1 blocks.
> ==6792==    indirectly lost: 120 bytes in 10 blocks.
> ==6792==      possibly lost: 43,872 bytes in 84 blocks.
> ==6792==    still reachable: 6,253,647 bytes in 26,607 blocks.
> ==6792==         suppressed: 4,496 bytes in 110 blocks.
> --6792--  memcheck: sanity checks: 29089 cheap, 220 expensive
> --6792--  memcheck: auxmaps: 0 auxmap entries (0k, 0M) in use
> --6792--  memcheck: auxmaps_L1: 0 searches, 0 cmps, ratio 0:10
> --6792--  memcheck: auxmaps_L2: 0 searches, 0 nodes
> --6792--  memcheck: SMs: n_issued      = 856 (13696k, 13M)
> --6792--  memcheck: SMs: n_deissued    = 7 (112k, 0M)
> --6792--  memcheck: SMs: max_noaccess  = 65535 (1048560k, 1023M)
> --6792--  memcheck: SMs: max_undefined = 35 (560k, 0M)
> --6792--  memcheck: SMs: max_defined   = 1348 (21568k, 21M)
> --6792--  memcheck: SMs: max_non_DSM   = 849 (13584k, 13M)
> --6792--  memcheck: max sec V bit nodes:    1581 (80k, 0M)
> --6792--  memcheck: set_sec_vbits8 calls: 51138 (new: 1581, updates: 49557)
> --6792--  memcheck: max shadow mem size:   13968k, 13M
> --6792-- translate:            fast SP updates identified: 48,718 ( 88.1%)
> --6792-- translate:   generic_known SP updates identified: 4,920 (  8.8%)
> --6792-- translate: generic_unknown SP updates identified: 1,647 (  2.9%)
> --6792--     tt/tc: 32,955,230 tt lookups requiring 50,735,061 probes
> --6792--     tt/tc: 32,955,230 fast-cache updates, 10 flushes
> --6792--  transtab: new        35,700 (767,467 -> 11,066,732; ratio
> 144:10) [0 scs]
> --6792--  transtab: dumped     0 (0 -> ??)
> --6792--  transtab: discarded  355 (7,022 -> ??)
> --6792-- scheduler: 2,835,614,942 jumps (bb entries).
> --6792-- scheduler: 29,089/43,763,705 major/minor sched events.
> --6792--    sanity: 29090 cheap, 220 expensive checks.
> --6792--    exectx: 12,289 lists, 11,820 contexts (avg 0 per list)
> --6792--    exectx: 9,597,286 searches, 9,605,875 full compares (1,000 per 1000)
> --6792--    exectx: 0 cmp2, 189 cmp4, 30,892,554 cmpAll
> --6792--  errormgr: 1,500 supplist searches, 190,814 comparisons during search
> --6792--  errormgr: 83 errlist searches, 199 comparisons during search
>
>
> --
> my place on the web:
> floss-and-misc.blogspot.com



-- 
my place on the web:
floss-and-misc.blogspot.com