Re: [Gimp-developer] Python FU Scripting -- Memory Leak? or How to release python gimp objects properly to manage memory usage?

[Tom Vrankar <twv cox net>]

And now, the rest of the story. I've completed this project, and 
everything worked out. Turns out that my perceived "memory leak" was 
nothing more than normal growth of the undo stack; disable undos, and 
the problem goes completely away. As for gimp_image_delete(), it occurs 
as part of deleting the last display of an image, just as documented, so 
it wasn't necessary (or possible) to call it explicitly.

In fact, using the Gimp for this project worked out quite well (based in 
part on the ovation the resulting video received). Just over 7100 frames 
representing 24 frames-per-second HD-720, rendered in a touch over 8 
hours on an older quad-core Intel. Beyond what I mentioned above, I 
called layer copies and saturation and brightness and blurs for drop 
shadows, rotation transforms, bump mapping, feathered dissolves, and 
loaded around 50 .png images with transparency into layers over the 
course of the animation. The only problems that remained were with the 
API. I guess there's a lot of history there, but inconsistencies in it 
make it a chore to learn. And when you call a function potentially 
thousands of times, you can't tolerate the pointlessly repeating 
"deprecated" dialog attached to many functions -- I got it the first 
time: it's deprecated, but it's still there _now_.

Some of my API issues are just gripes. A very few of my layers were the 
size of my image; almost all were either larger or smaller. It took me 
an embarrassing amount of time to understand the differences between 
coordinate systems and why they had to be so -- x, y usually reference 
the image's upper left (in my mind, the image isn't a thing the way 
layers are things; the image merely acts like a window view on your 
layers), except when transforming a layer, in which case x, y reference 
the layer's upper left. At least x and y always grew right and down. But 
of the functions I used that involved angles, each one seemed to zero or 
rotate in a different direction, and both degrees and radians appear for 
different functions in the API. I spent some effort trying to be careful 
about using int (round (x)) for integer vs float parameters, but had 
hoped the API would secretly do this for me, in either direction, and it 
might, but the docs don't hint at this. I tripped a number of times over 
opacity going from 0. to 100., though alpha goes only from 0. to 1. (the 
latter makes more sense to me for both).

But a few things might be broken. I haven't found a way to specify a 
dummy "parent" group, so had to code image.insert_layer (layer, position 
=0); but gimp_image_insert_vectors() is uncallable as far as I can tell, 
so I had to pre-create a named vector object and keep re-using it. Most 
functions pass around objects instead of IDs, except 
gimp_vectors_remove_stroke(), which actually takes an ID (I noticed some 
other function return an ID; I can't recall which one it was, but I know 
it's there).

Besides a little grumbling, that I could (eventually) get the Gimp to do 
as many things as it did was amazing, and allowed me to create a very 
unique presentation which could not be confused with something out of 
Powerpoint, and made a hack like me look like a pro -- if only to people 
who don't use the Gimp.

-------- Original Message --------
Subject: Python FU Scripting -- Memory Leak? or How to release python 
gimp objects properly to manage memory usage?
Date: Wed, 15 Feb 2012 23:42:31 -0500
From: Tom Vrankar <twv cox net>
To: gimp-developer-list gnome org

I'm using python-fu to script a sequence of animation frames in Partha's
Gimp 2.7.5 for win 32 (Vista) (Thanks!) It's expected to be a relatively
long sequence when done, probably several thousand frames, but the
animation is pretty simple, very algorithmic, just a series of layer
pans, zooms, fades, a little text, and drawing and stroking a growing
path, over and over -- a glorified slideshow. I don't mind if it takes
overnight to render. Anyway, I got the first version to load, and start
writing frames out as .pngs (targeting a later assembly process using
ffmeg). But it crashes gimp entirely around frame 175 or so (it starts
out quick, and slows noticeably near the end). Turns out it leaks memory
pretty badly, and it dies right about when task manager shows the memory
use in the gimp process just passing 2GB. Signed 32-bit integer
pointers, I guess.

Image is 1280x720 RGB, largest layers are 4800x3400 or so. Maybe 10
layers at most at any one time.

The original intent was to manipulate, copy, and create layers as needed
from within a single image, make a new layer from visible, write that
out, then remove old layers and start over for the next frame. I can get
individual commands and short sequences to appear to have the desired
effect when typed into the console (after a lot of reviewing source on
github -- this is a pretty cranky API; don't get me started on the
*_insert_* methods "parent" parameters!). Thinking that I was missing
something about properly managing gimp or python, I tried isolating a
much simpler set of commands, basically copying a reference layer from
one image into a new displayed image, then destroying that new image and
starting over. Just entering them into the console repeatedly I can see
the same gimp process memory usage jump up each iteration, at first
accumulating just under 1MB on each step, then apparently growing
steadily larger. I can see from the layer toolbox that my layer count
doesn't seem to grow larger than intended, though I suppose there could
be layers not properly inserted or deleted that might not show in the
toolbox.

I'm stumped on how to use python gimp to create this animation (and I
came to gimp because synfig couldn't handle all my over-one hundred
source images). I haven't found any example scripts that deal with this
scale yet. Do the experts on this board have any corrections to what I'm
doing wrong, or any tips or tricks that I might apply to manage this
issue? For example, if it's not my script, are certain methodologies
more leaky than others (such as opacity changes vs visibility changes,
copy-pasting visible to a re-used layer vs creating a new layer from
visible, layer copies within an image vs across images, layer
creation/destruction vs image creation/destruction)?


Here's the simple command sequence that demonstrates the "leak". I start
with an image containing an RGB layer 4800x3400 or so at layer[1]
(there's a transparent top layer). Open task manager and watch memory
use in the gimp process. Open the python-fu console, initialize gp
=gimp.pdb and src =gimp.image_list()[0], then iterate the following:

img =gp.gimp_image_new (1280, 720, 0)
dsp =gp.gimp_display_new (img)
lyr =gp.gimp_layer_new_from_drawable (src.layers[1], img)
img.insert_layer (lyr, position =0)
gp.gimp_display_delete (dsp)

I see memory use jump after each iteration. Any help is appreciated.

twv@