GNOME.org
 

Re: [Gimp-developer] AI algorithms in GIMP

Hi guys,

I think Casey's suggestion of super resolution is an excellent idea and
would be useful to a lot of people using GIMP. The two algorithms which
have been doing quite well are:

1. SRGAN: The relevant paper is at https://arxiv.org/abs/1609.04802 . This
has yielded some very nice results so far and is now considered to be
state-of-the-art in the computer vision community. Implementing this will
be my first priority.

2. SRResNet: The paper is at https://arxiv.org/abs/1609.04802 . This
algorithm also gives quite good results.

Now to train these neural nets, I will have to prepare a dataset consisting
of low resolution and high resolution images and divide them into training,
testing and validation sets. I'll try to write a script which pulls
high-resolution CC-A/CC0/CC-A-SA images from the internet and downscale
them in python to prepare their low resolution counterparts. I only hope
that my GPU has enough memory to train the model, otherwise I am done for.

What I would like to do is create a python script (most of these libraries
are available in Python) and then pass the image along with the scaling
factor into this from inside GIMP. I've never worked with GEGL before but I
will try finding out how to use this.

Since I am working on this in my free time and have not implemented neural
nets much, this might take some time for me to learn. I'll try keeping you
guys informed.

Any suggestions are welcome.

Thanks,
Maitraya.

On Sun, Jan 20, 2019 at 12:41 AM Jehan Pagès <jehan marmottard gmail com>
wrote:


Hi!

On Thu, Jan 17, 2019 at 12:14 PM Maitraya Bhattacharyya via
gimp-developer-list <gimp-developer-list gnome org> wrote:


Dear devs,

I have recently joined the mailing list because I wanted to contribute my
two pennies to GIMP development (since I use it for my work). I had a look
at the proposed plan for GIMP and wondered if people would be interested
in
including some popular AI algorithms for several image processing tasks.



We are definitely interested by any AI algorithms. At least I am.

I would be interested in writing implementations of some of these

algorithms into gimp if someone can commit to writing a frontend/GUI for
it.



The hard point here is "if someone can commit to […]". It's a bit hard to
commit without knowing much (unless you are paid, then you have no choice
;p). Usually it's the other way around: you propose something. It doesn't
have to be with a great GUI or whatever. Then if we like what we see, we
will definitely add our own 2 cents to the pool. This is usually how most
features are done around here, when someone contributes a patch with a very
cool idea, then we review and often fix/change what we think is needed
(sometimes just a bit, sometimes deeply).

I often wrote crap GUI myself and others came to the rescue with ideas and
code. :-)



It would be great if we can make a list of these algorithms to implement
and rank them according to priority.



I would suggest to *not do that*. :-)
Basically we are not a company, we don't sell GIMP and don't have huge
plans for the next decade. Well "officially", we do have a roadmap and
such, but if you follow GIMP development, you'd see it is more flexible and
experimental than some rigid plan.
Making a huge list with big plans for the future "might" be just the way
to spend a lot of time and kill your project in the end.

Instead I would propose **you** just select **one** such algorithm which
you find is great and even irrefusable since it would be so fucking awesome
and useful! Then you implement and propose it and we will be so amazed that
we just have to include it and do a proper GUI for it. That sounds like the
best plan.

From there, you can go on with more awesome ideas. :-)
You may even be able to start doing more organized work with a list of
algorithms after, etc. But for a first patch, I would suggest you just take
the lead.



As for my background, I am a theoretical physicist making simulations for
HPCs (in C/C++) and interpreting their data (in Python). I have a


reasonable workstation to train neural nets, if necessary. Be warned that I




There is only a single very important part about AI algorithms which need
training: we will want the code to train, the data, etc. everything as free
software/Open Data and properly documented.
I have worked with trained algorithms in Free Software where the trained
data is just dropped as-is, and once the original author disappears, this
is unmaintainable. In particular it cannot be improved or fixed or nothing,
because we don't have the code to re-generate the neural networks (or
alike). This can only be a recipe for failure long-term.

So AI in GIMP? Yeah definitely! But it has to be reproducible generated
data, with the whole training infrastructure available and properly
documented and the input data under Libre license as well.



have never written a GUI software in my life and I don't know the GIMP
codebase at all. I envision these to be standalone scripts which can be
called in from the GIMP interface.



Cool. Depending on the idea, it may be interesting to rather implement it
directly as a GEGL operation (GEGL is our graphics engine). Maybe you don't
even need to make a GUI then. Just make a GEGL op, and when it is done, run
it with examples to show us how good it is, and we might just get into the
game to make it a proper GUI.



Please let me know what you think.



And here you are! I hope we will see soon a lot of baby robots in our
code. ;-)

Jehan




Cheerio,
Maitraya.

Senior Research Fellow
Center for Excellence in Space Sciences India
Indian Institute of Science Education and Research Kolkata
_______________________________________________
gimp-developer-list mailing list
List address:    gimp-developer-list gnome org
List membership:
https://mail.gnome.org/mailman/listinfo/gimp-developer-list
List archives:   https://mail.gnome.org/archives/gimp-developer-list




--
ZeMarmot open animation film
http://film.zemarmot.net
Liberapay: https://liberapay.com/ZeMarmot/
Patreon: https://patreon.com/zemarmot
Tipeee: https://www.tipeee.com/zemarmot
[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]