Re: [BuildStream] Issues tagged `optimization`

[Tiago Gomes <tiago gomes codethink co uk>]

On 23/10/2018 15:32, Daniel Silverstone via BuildStream-list wrote:

> [Track file dependencies][56] filed by Sander Striker on July 31st 2017 talks
> about attempting to optimise away execution of build steps if we can determine
> that all the inputs which *matter* to the step can be shown to be identical.

Has anyone considered using ccache here? We could minimize the overhead 
of cache misses by disabling ccache if there isn't any old artifact in 
the CAS for the element being built.

> [Cache calculation trashes too much the IO][573] filed by Tiago Gomes on Aug
> 13th 2018 raises the point that whenever we re-scan the cache to calculate its
> size we end up doing a lot of IO and we can end up evicting useful things from
> the page cache.  Tiago proposes a fairly "simple" approach to resolving this,
> though it will take someone experienced with the CasCache code to decide how
> complex it'll turn out to be to implement.  Tiago had a go at implementing a
> fix in MR 671 but this MR has been paused for quite some time which points at
> the possibility that it needs extra thought.  It'd be helpful if Tiago might
> weigh in and explain what the situation is here. >

As you've pointed out, a MR was submitted to address this issue, which 
fell through the cracks despite my repeated rebases and attempts of 
getting it merged. This PR is now out-of-date, and no longer correct due 
the introduction of the SandboxRemote class, which also writes to the 
cache. I won't have time to work on this MR again, so if anyone wants to 
pick it up feel free. I will unassign myself from the issue.

Regards,

Tiago