Cheese and OpenGL - 3rd week report

["Filippo Argiolas" <filippo argiolas gmail com>]

Hey all, here is my third summer of code report, I'm writing it quickly from an
occasional internet connection, so expect me to probably forget something or to
make a couple of mistakes ;)
Update: I abused a bit of the occasional connection so the report is a bit
longer than I thought :P

===================
  3rd week report
===================

At the end of last week I thought I was almost done with infrastructure work
(GLSL gobject interface, test application, basic gstelement to make tests).  So
I began this week concentrating on the most awesome yet challenging part of my
summer job, the effects!

I think I've made some good progresses with the distortion/warp effects, I
completed about an half of them (mirror, squeeze, dent and that one that makes
your face look like a square), and the other ones (bulge, fisheye, twirl, .., ?)
shouldn't be difficult to write (just been too lazy to do it :P).
There is surely some room for improvements yet, since some of them depends on
several arbitrary parameters/constants (e.g. the "squeezing" amount, the size of
the square, etc), I still have to "tune" them a little bit to reach wanted
results.

I also tried something with per-pixel color transforms (sepia, negative, thermal
camera, ...) but I'm not convinced yet with what I've got.
I've found a great, smart way to realize this kind of effects working with a
webcam shot on gimp, manipulating color curves and creating a 256x1 texture that
can be directly used within the code simply saving it with a .c extension.

So if you find some cool transform that can be realized using the gimp curve
tool only, just save it an send it to me! (hint: some fake crossprocessing
effect?)

While working on gimp to make some "effect prototype" reproducible with glsl
shaders, I realized that cooler effects are the ones with multiple filters
applied subsequently or to different levels and blended together.  This is not
easy to achieve with a single fragment shader because it only can do "in-place"
per pixel/fragment transforms; it can, of course, sample neighbor pixels but
only to compute current pixel's color/position.

Furthermore a "desirable" feature would be to easily apply multiple effects
together and that's quite difficult to realize without a simple way to use the
output of an effect as input of another.
So, how to do this with opengl? I spent some time reading OpenGL docs and
extensions specs, and I think the best way to do this is with FBOs (Frame Buffer
Objects) and multiple rendering passes. This way each effect, or each filter
within a single effect, can render to a texture which can be used as the input
texture for the next effect/filter.

To be able to support this "multiple rendering passes", I had to leave the
amazing part of creating effects and return to the backend work :P

So I rethinked a little bit the way my little library manages shaders, I dropped
the idea to have multiple shaders for each program object, thus having two
classes GstGLShader and GstGLProgram interfacing with each other. I decided to
have a single GstGLShader class that manages three GLhandle-s (a vertex shader,
a fragment shader and a program object). So each shader has its own program
object and cares to initialize and clean up it.

The previous approach was better with a single rendering pass: since only one
program can be applied at the same time, each shader had to be linked to that
single program with several limitations (most of all having only one "main"
function). With multiple rendering stages each one can have its own program
object with no more need to link different shaders to the same program.

I spent the rest of the week implementing this new idea and rearranging my
plugin to fit the new api. I didn't have the time to test it well, so I didn't
commit my changes yet but I promise to put everything online soon!

That's almost everything I did in the past week.. Remaining stuff can
be tagged as
"experimental" so I'll talk about it in next reports if it'll lead to something
useful.
Please note that I don't know if I'll be able to work the next week because
I have to study for an exam, I'll try to find some time to test the
new code and commit it.

Cheers,

Filippo