[Shotwell] shotwell consumes all available memory on startup

Tue Jan 25 20:12:54 UTC 2011

Adam-

Thanks for the ongoing help :)

I have 2G RAM and about 4G swap.

johncc at liberator:~/Pictures$ free -m
              total       used       free     shared    buffers     cached
Mem:          2008       1044        963          0        116        306
-/+ buffers/cache:        621       1387
Swap:         4095        509       3586

Seems like:

johncc at liberator:~$ mkdir foo
johncc at liberator:~$ shotwell -d foo/

... still starts to eat RAM in the same way.

johncc at liberator:~/$ pg shotwell
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
johncc    3356 86.2 84.0 4451664 1727856 pts/3 Rl+  20:00   7:15

It does seem to have stabilized at that point, in that it's not grabbing 
more RAM as far as I can tell, and there is still potentially a bit left!

johncc at liberator:~/$ free -m
              total       used       free     shared    buffers     cached
Mem:          2008       1987         20          0          1         49
-/+ buffers/cache:       1936         71
Swap:         4095       2918       1177

I guessed there was no point trying the subsequent tests, but let me 
know if you want me to.

John

On 25/01/11 18:36, Adam Dingle wrote:
> John,
>
> First of all, the error "Directory Sony2 with ##### entries considered
> invalid" is unrelated and is spurious: this is a manifestation of
> http://trac.yorba.org/ticket/1872 .
>
> Your gconftool commands look correct, so we've ruled out that
> auto-import is causing the memory flood; that's good.
>
> - How much RAM do you have on your system?
>
> - Can you do this? Start Shotwell with an empty library (e.g. "shotwell
> -d ~/foo"), then import just 1000 of your photos. (Here's one way to do
> that: use gnome-search-tool, otherwise known as Places->Search for
> Files, to find all files in ~/Pictures, then select just 1000 of them
> and drag them into Shotwell.) After you've done this, how much physical
> and virtual memory is the 'shotwell' process using, e.g. as reported by
> the GNOME System Monitor?
>
> - Now repeat the previous experiment with 5000 photos. Once again, how
> much physical and virtual memory is the 'shotwell' process using?
>
> adam
>
> On 01/24/2011 04:33 PM, John Carlyle-Clarke wrote:
>> Hi Adam-
>>
>> The progress dialog disappears before the memory usage starts to
>> increase.
>>
>> johncc at liberator:~/Pictures$ du -shx
>> 34G .
>> johncc at liberator:~/Pictures$ find | wc -l
>> 17696
>>
>> None of the pictures RAW format.
>>
>> I don't have full GNOME, so hopefully I did this right:-
>>
>> johncc at liberator:~/Pictures$ gconftool-2 --set --type=boolean
>> /apps/shotwell/preferences/files/auto_import false
>> johncc at liberator:~/Pictures$ gconftool-2 -R
>> /apps/shotwell/preferences/files
>> import_dir = /home/johncc/Pictures
>> commit_metadata = true
>> auto_import = false
>>
>> I still get the memory flood after this.
>>
>> By the way, I just noticed a spurious error I get sometimes. I'm not
>> sure if it's connected, but I can't find any directory called Sony2!
>>
>> johncc at liberator:~/Pictures$ shotwell
>> Error: Directory Sony2 with 20307 entries considered invalid; not read.
>>
>> Best regards,
>>
>> John
>>
>> On 25/01/11 00:09, Adam Dingle wrote:
>>> John,
>>>
>>> - To be clear, does the progress dialog disappear, or does it remain
>>> on-screen while the memory usage increases?
>>>
>>> - How many photos are in your Shotwell library? Approximately how many
>>> of them are RAW?
>>>
>>> - Do you have the option to watch the library directory for new files
>>> turned on? If so, could you try the following? While Shotwell is not
>>> running, start gconf-editor, find the value
>>> /apps/shotwell/preferences/files/auto_import and uncheck it to disable
>>> library watching. Now start Shotwell again. Does the memory flood still
>>> occur?
>>>
>>> adam
>>>
>>> On 01/24/2011 03:45 PM, John Carlyle-Clarke wrote:
>>>> I'm having a problem with shotwell 0.8.1 on Archlinux 64-bit. When I
>>>> start it, the progress dialog appears, followed by the UI. After a few
>>>> seconds, the UI becomes unresponsive and it starts to increase its
>>>> memory usage until it consumes all RAM and swap.
>>>>
>>>> The end of the strace, which coincides with the beginning of the
>>>> increasing memory use, is:-
>>>>
>>>> futex(0x7f6893be1ea0, FUTEX_WAKE_PRIVATE, 1) = 0
>>>> futex(0x7f6894830528, FUTEX_WAKE_PRIVATE, 1) = 0
>>>> futex(0x7f6894830528, FUTEX_WAKE_PRIVATE, 1) = 0
>>>> futex(0x7f6894830528, FUTEX_WAIT_PRIVATE, 2, NULL) = -1 EAGAIN
>>>> (Resource temporarily unavailable)
>>>> futex(0x7f6894830528, FUTEX_WAKE_PRIVATE, 1) = 0
>>>> futex(0x7f689508d8e8, FUTEX_WAKE_PRIVATE, 1) = 0
>>>> brk(0x5a2d000) = 0x5a2d000
>>>> brk(0x5a88000) = 0x5a88000
>>>> mmap(NULL, 499712, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS,
>>>> -1, 0) = 0x7f6888c62000
>>>> mmap(NULL, 999424, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS,
>>>> -1, 0) = 0x7f6888b6e000
>>>> mmap(NULL, 1994752, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS,
>>>> -1, 0) = 0x7f6888987000
>>>> futex(0x7f6893be1ea0, FUTEX_WAKE_PRIVATE, 1) = 1
>>>> mmap(NULL, 3985408, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS,
>>>> -1, 0) = 0x7f68885ba000
>>>> mmap(NULL, 7966720, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS,
>>>> -1, 0) = 0x7f687ad4f000
>>>> futex(0x7f6893be1ea0, FUTEX_WAKE_PRIVATE, 1) = 1
>>>> mmap(NULL, 15929344, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS,
>>>> -1, 0) = 0x7f6879e1e000
>>>> mmap(NULL, 31854592, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS,
>>>> -1, 0) = 0x7f6877fbd000
>>>> futex(0x7f6893be1ea0, FUTEX_WAKE_PRIVATE, 1) = 1
>>>> mmap(NULL, 63705088, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS,
>>>> -1, 0) = 0x7f68742fc000
>>>> mmap(NULL, 127406080, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS,
>>>> -1, 0) = 0x7f686c97b000
>>>> mmap(NULL, 254808064, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS,
>>>> -1, 0) = 0x7f685d67a000
>>>> mmap(NULL, 509612032, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS,
>>>> -1, 0) = 0x7f683f079000
>>>> mmap(NULL, 1019219968, PROT_READ|PROT_WRITE,
>>>> MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f6802478000
>>>> mmap(NULL, 2038435840, PROT_READ|PROT_WRITE,
>>>> MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f6788c77000
>>>>
>>>> Not sure what else I can usefully provide! Let me know...
>>>>