[Tracker] A lot more VmRSS memory usage while indexing since 0.14



Hi guys,

In a use-case on a mobile phone that used to use 0.14 we see an
increased VmRSS memory usage of tracker-miner-fs since the 0.17 based
version that we started using.

I have attached a massif dump, you can vizualize this with Massif
Visualizer.


Example snapshot:

#-----------
snapshot=40
#-----------
time=15010364654
mem_heap_B=390075965
mem_heap_extra_B=107877155
mem_stacks_B=0
heap_tree=peak
n5: 390075965 (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
 n2: 149302848 0x5E3FB26: g_array_maybe_expand (garray.c:771)
  n2: 149298400 0x5E400FE: g_array_insert_vals (garray.c:516)
   n2: 149298240 0x5CB5346: g_file_info_create_value (gfileinfo.c:1084)
    n2: 149298176 0x5CB6746: g_file_info_set_file_type (gfileinfo.c:1798)
     n1: 149298176 0x5D6AD8A: g_local_file_enumerator_next_file (glocalfileenumerator.c:413)
      n1: 149298176 0x5CAAA46: next_files_thread (gfileenumerator.c:639)
       n1: 149298176 0x5CDA166: run_in_thread (gsimpleasyncresult.c:861)
        n1: 149298176 0x5CC569E: io_job_thread (gioscheduler.c:177)
         n1: 149298176 0x5EA4EFE: g_thread_pool_thread_proxy (gthreadpool.c:309)
          n1: 149298176 0x5EA41A6: g_thread_proxy (gthread.c:801)
           n1: 149298176 0x609FD12: start_thread (pthread_create.c:310)
            n0: 149298176 0x618EBB6: ??? (in /lib/libc-2.15.so)
     n0: 0 in 1 place, below massif's threshold (01.00%)
    n0: 64 in 3 places, all below massif's threshold (01.00%)
   n0: 160 in 2 places, all below massif's threshold (01.00%)
  n0: 4448 in 2 places, all below massif's threshold (01.00%)
 n3: 123641386 0x5E9A1FA: g_strdup (gstrfuncs.c:356)
  n1: 102578495 0x5D66E9A: canonicalize_filename (glocalfile.c:203)
   n2: 102578495 0x5D6A88E: _g_local_file_new (glocalfile.c:289)
    n1: 102578246 0x5D6A912: g_local_file_resolve_relative_path (glocalfile.c:539)
     n1: 102578246 0x5C9F3E2: g_file_resolve_relative_path (gfile.c:819)
      n1: 102578246 0x485340A: file_enumerate_next_cb (tracker-crawler.c:736)
       n1: 102578246 0x5CAA69E: next_async_callback_wrapper (gfileenumerator.c:298)
        n1: 102578246 0x5CDA2C6: g_simple_async_result_complete (gsimpleasyncresult.c:767)
         n1: 102578246 0x5CDA3AE: complete_in_idle_cb_for_thread (gsimpleasyncresult.c:835)
          n1: 102578246 0x5E728C6: g_idle_dispatch (gmain.c:4657)
           n1: 102578246 0x5E75DC6: g_main_context_dispatch (gmain.c:2539)
            n1: 102578246 0x5E760C2: g_main_context_iterate.part.17 (gmain.c:3146)
             n1: 102578246 0x5E76646: g_main_loop_run (gmain.c:3339)
              n0: 102578246 0xF5AE: main (tracker-main.c:975)
    n0: 249 in 2 places, all below massif's threshold (01.00%)
  n1: 20977775 0x5CA9F0E: _g_file_attribute_value_set_byte_string (gfileattribute.c:731)
   n1: 20977775 0x5D6D1A2: _g_local_file_info_get_nostat (glocalfileinfo.c:1415)
    n1: 20977775 0x5D6AD7E: g_local_file_enumerator_next_file (glocalfileenumerator.c:412)
     n1: 20977775 0x5CAAA46: next_files_thread (gfileenumerator.c:639)
      n1: 20977775 0x5CDA166: run_in_thread (gsimpleasyncresult.c:861)
       n1: 20977775 0x5CC569E: io_job_thread (gioscheduler.c:177)
        n1: 20977775 0x5EA4EFE: g_thread_pool_thread_proxy (gthreadpool.c:309)
         n1: 20977775 0x5EA41A6: g_thread_proxy (gthread.c:801)
          n1: 20977775 0x609FD12: start_thread (pthread_create.c:310)
           n0: 20977775 0x618EBB6: ??? (in /lib/libc-2.15.so)
  n0: 85116 in 164 places, all below massif's threshold (01.00%)
 n5: 93438192 0x5E9897A: g_slice_alloc (gslice.c:1003)
  n2: 42128852 0x5E98ED6: g_slice_alloc0 (gslice.c:1029)
   n2: 42016000 0x5DEEEA2: g_type_create_instance (gtype.c:1872)
    n2: 42009816 0x5DC99EA: g_object_constructor (gobject.c:1849)
     n1: 41989784 0x5DCC7E6: g_object_newv (gobject.c:1632)
      n3: 41989784 0x5DCCE9A: g_object_new (gobject.c:1542)
       n2: 23327860 0x5D6D30A: _g_local_file_info_get (glocalfileinfo.c:1476)
        n1: 23327840 0x5D6AD66: g_local_file_enumerator_next_file (glocalfileenumerator.c:405)
         n1: 23327840 0x5CAAA46: next_files_thread (gfileenumerator.c:639)
          n1: 23327840 0x5CDA166: run_in_thread (gsimpleasyncresult.c:861)
           n1: 23327840 0x5CC569E: io_job_thread (gioscheduler.c:177)
            n1: 23327840 0x5EA4EFE: g_thread_pool_thread_proxy (gthreadpool.c:309)
             n1: 23327840 0x5EA41A6: g_thread_proxy (gthread.c:801)
              n1: 23327840 0x609FD12: start_thread (pthread_create.c:310)
               n0: 23327840 0x618EBB6: ??? (in /lib/libc-2.15.so)
        n0: 20 in 1 place, below massif's threshold (01.00%)
       n2: 18661056 0x5D6A882: _g_local_file_new (glocalfile.c:288)
        n1: 18660864 0x5D6A912: g_local_file_resolve_relative_path (glocalfile.c:539)
         n1: 18660864 0x5C9F3E2: g_file_resolve_relative_path (gfile.c:819)
          n1: 18660864 0x485340A: file_enumerate_next_cb (tracker-crawler.c:736)
           n1: 18660864 0x5CAA69E: next_async_callback_wrapper (gfileenumerator.c:298)
            n1: 18660864 0x5CDA2C6: g_simple_async_result_complete (gsimpleasyncresult.c:767)
             n1: 18660864 0x5CDA3AE: complete_in_idle_cb_for_thread (gsimpleasyncresult.c:835)
              n1: 18660864 0x5E728C6: g_idle_dispatch (gmain.c:4657)
               n1: 18660864 0x5E75DC6: g_main_context_dispatch (gmain.c:2539)
                n1: 18660864 0x5E760C2: g_main_context_iterate.part.17 (gmain.c:3146)
                 n1: 18660864 0x5E76646: g_main_loop_run (gmain.c:3339)
                  n0: 18660864 0xF5AE: main (tracker-main.c:975)
        n0: 192 in 2 places, all below massif's threshold (01.00%)
       n0: 868 in 32 places, all below massif's threshold (01.00%)
     n0: 20032 in 3 places, all below massif's threshold (01.00%)
    n0: 6184 in 2 places, all below massif's threshold (01.00%)
   n0: 112852 in 47 places, all below massif's threshold (01.00%)
  n2: 32674180 0x5E3FB92: g_array_sized_new (garray.c:195)
   n1: 32659004 0x5CAC41E: g_file_info_init (gfileinfo.c:345)
    n1: 32659004 0x5DEEF56: g_type_create_instance (gtype.c:1892)
     n1: 32659004 0x5DC99EA: g_object_constructor (gobject.c:1849)
      n1: 32659004 0x5DCC7E6: g_object_newv (gobject.c:1632)
       n1: 32659004 0x5DCCE9A: g_object_new (gobject.c:1542)
        n2: 32659004 0x5D6D30A: _g_local_file_info_get (glocalfileinfo.c:1476)
         n1: 32658976 0x5D6AD66: g_local_file_enumerator_next_file (glocalfileenumerator.c:405)
          n1: 32658976 0x5CAAA46: next_files_thread (gfileenumerator.c:639)
           n1: 32658976 0x5CDA166: run_in_thread (gsimpleasyncresult.c:861)
            n1: 32658976 0x5CC569E: io_job_thread (gioscheduler.c:177)
             n1: 32658976 0x5EA4EFE: g_thread_pool_thread_proxy (gthreadpool.c:309)
              n1: 32658976 0x5EA41A6: g_thread_proxy (gthread.c:801)
               n1: 32658976 0x609FD12: start_thread (pthread_create.c:310)
                n0: 32658976 0x618EBB6: ??? (in /lib/libc-2.15.so)
         n0: 28 in 1 place, below massif's threshold (01.00%)
   n0: 15176 in 9 places, all below massif's threshold (01.00%)
  n2: 9293528 0x5E99836: g_slist_prepend (gslist.c:265)
   n1: 9292000 0x4853496: file_enumerate_next_cb (tracker-crawler.c:370)
    n1: 9292000 0x5CAA69E: next_async_callback_wrapper (gfileenumerator.c:298)
     n1: 9292000 0x5CDA2C6: g_simple_async_result_complete (gsimpleasyncresult.c:767)
      n1: 9292000 0x5CDA3AE: complete_in_idle_cb_for_thread (gsimpleasyncresult.c:835)
       n1: 9292000 0x5E728C6: g_idle_dispatch (gmain.c:4657)
        n1: 9292000 0x5E75DC6: g_main_context_dispatch (gmain.c:2539)
         n1: 9292000 0x5E760C2: g_main_context_iterate.part.17 (gmain.c:3146)
          n1: 9292000 0x5E76646: g_main_loop_run (gmain.c:3339)
           n0: 9292000 0xF5AE: main (tracker-main.c:975)
   n0: 1528 in 40 places, all below massif's threshold (01.00%)
  n1: 9292000 0x485347A: file_enumerate_next_cb (tracker-crawler.c:328)
   n1: 9292000 0x5CAA69E: next_async_callback_wrapper (gfileenumerator.c:298)
    n1: 9292000 0x5CDA2C6: g_simple_async_result_complete (gsimpleasyncresult.c:767)
     n1: 9292000 0x5CDA3AE: complete_in_idle_cb_for_thread (gsimpleasyncresult.c:835)
      n1: 9292000 0x5E728C6: g_idle_dispatch (gmain.c:4657)
       n1: 9292000 0x5E75DC6: g_main_context_dispatch (gmain.c:2539)
        n1: 9292000 0x5E760C2: g_main_context_iterate.part.17 (gmain.c:3146)
         n1: 9292000 0x5E76646: g_main_loop_run (gmain.c:3339)
          n0: 9292000 0xF5AE: main (tracker-main.c:975)
  n0: 49632 in 42 places, all below massif's threshold (01.00%)
 n2: 23321100 0x5E53ADE: g_datalist_id_set_data_full (gdataset.c:474)
  n1: 23321000 0x485346E: file_enumerate_next_cb (tracker-crawler.c:741)
   n1: 23321000 0x5CAA69E: next_async_callback_wrapper (gfileenumerator.c:298)
    n1: 23321000 0x5CDA2C6: g_simple_async_result_complete (gsimpleasyncresult.c:767)
     n1: 23321000 0x5CDA3AE: complete_in_idle_cb_for_thread (gsimpleasyncresult.c:835)
      n1: 23321000 0x5E728C6: g_idle_dispatch (gmain.c:4657)
       n1: 23321000 0x5E75DC6: g_main_context_dispatch (gmain.c:2539)
        n1: 23321000 0x5E760C2: g_main_context_iterate.part.17 (gmain.c:3146)
         n1: 23321000 0x5E76646: g_main_loop_run (gmain.c:3339)
          n0: 23321000 0xF5AE: main (tracker-main.c:975)
  n0: 100 in 7 places, all below massif's threshold (01.00%)
 n0: 372439 in 310 places, all below massif's threshold (01.00%)

Attachment: massif.out.19468
Description: application/valgrind-massif



[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]