Re: [BuildStream] Issues tagged `optimization`

[Jim MacArthur <jim macarthur codethink co uk>]

On 26/10/18 12:45, Tiago Gomes via BuildStream-list wrote:
> On 23/10/2018 15:32, Daniel Silverstone via BuildStream-list wrote:
>
> > [Cache calculation trashes too much the IO][573] filed by Tiago Gomes 
> > on Aug
> > 13th 2018 raises the point that whenever we re-scan the cache to 
> > calculate its
> > size we end up doing a lot of IO and we can end up evicting useful 
> > things from
> > the page cache.  Tiago proposes a fairly "simple" approach to 
> > resolving this,
> > though it will take someone experienced with the CasCache code to 
> > decide how
> > complex it'll turn out to be to implement.  Tiago had a go at 
> > implementing a
> > fix in MR 671 but this MR has been paused for quite some time which 
> > points at
> > the possibility that it needs extra thought.  It'd be helpful if 
> > Tiago might
> > weigh in and explain what the situation is here. >
>
> As you've pointed out, a MR was submitted to address this issue, which 
> fell through the cracks despite my repeated rebases and attempts of 
> getting it merged. This PR is now out-of-date, and no longer correct 
> due the introduction of the SandboxRemote class, which also writes to 
> the cache. I won't have time to work on this MR again, so if anyone 
> wants to pick it up feel free. I will unassign myself from the issue.
We intend to separate the artifact and remote execution caches [1] so it 
may not be ultimately necessary to account for SandboxRemote. We may 
allow the two stores to be in one location, but as long as the 
optimisation in MR 671 is optional, we should be OK in any case.

[1] 
https://mail.gnome.org/archives/buildstream-list/2018-October/msg00046.html