Re: [BuildStream] Proposal: A small number of subprocesses handling jobs

[Tristan Van Berkom <tristan vanberkom codethink co uk>]

Hi Again,

On Wed, 2019-02-27 at 20:54 +0900, Tristan Van Berkom via buildstream-list wrote:
> Hi Jonathan,
[...]
> I think that if we are going to change the model for tasks, it should
> be first to change over to a threading model rather than a fork model
> first.

Directly after writing this I was punched in the teeth by the GIL.

I ended up giving these papers a read:

    https://emptysqua.re/blog/grok-the-gil-fast-thread-safe-python/
    http://python-notes.curiousefficiency.org/en/latest/python3/multicore_python.html

Not sure what we can do really... I'm not finding much in the way of
materials about safely disabling the GIL, the second paper mentions an
interesting option of compiling Python down to machine code (iiuc) and
thus removing the requirement of a GIL.

Cheers,
    -Tristan