Re: slow creation of ostrees using pungi/koji

[Dusty Mabe <dusty dustymabe com>]

On 08/18/2017 09:59 AM, Colin Walters wrote:
> On Fri, Aug 18, 2017, at 09:31 AM, Dusty Mabe wrote:
> > I've noticed that it can be really slow to create ostrees in Fedora using 
> > pungi/koji. I think most of the time is spent in the 'committing'
> > phase and believe this probably is related to the fact that we are
> > writing a ton of small files.
> >
> > Is there anything we can do to improve this? We are essentially accessing
> > the ostree repo via an nfs mount (so it's just a mountpoint on the local system).
> > I wonder if it would be better to work in a temporary directory and then have some
> > sort of 'ostree push' that would be more efficient.
>
> Previous threads:
>
> https://mail.gnome.org/archives/ostree-list/2015-October/msg00007.html
> https://bugzilla.gnome.org/show_bug.cgi?id=756540
>
> One thing that might be an issue here (though I'm just guessing without
> profiling) is that today the `ostree commit` logic is synchronous.  On local
> filesystems it's OK, but on NFS all the latency probably adds up.  However,
> `ostree pull-local` (which is a way to do "push") is multithreaded/async.
>
> One best practice is to commit to a `bare-user` repository first.  This
> is mentioned in the docs:
> https://ostree.readthedocs.io/en/latest/manual/buildsystem-and-repos/
>
> So something like this:
>
> ostree --repo=build-repo init --mode=bare-user
> # Clone the latest commit from upstream
> ostree --repo=build-repo pull-local /path/to/nfs/repo ${ref}
> # Generate new commit
> rpm-ostree --repo=build-repo compose tree --touch-if-changed=changed.stamp ..
> if test -f changed.stamp; then
>   ostree --repo=/path/to/nfs/repo pull-local build-repo ${ref}
> else
>   echo no changes
> fi

Thanks for this info. I'll try to investigate to see the difference in the two
ways of doing this.

Not entirely related because this is about `rpm-ostree`, but it would be interesting
if we could add a mode to `rpm-ostree compose tree` that would do something like that
for us. (i.e. copy everything to tmp ostree repo and then compose and then push back
to original source). This is kind of a special corner case, though. So maybe not. This
workflow would be interesting if we supported 'composing' for a remote, so copy down
changes (if you use a cachdir for rpms then you can cache an ostree repo too), compose, 
and push up to the remote over ssh (like the original ostree push proposal suggested).