Re: ostree create-usb performance

[Phaedrus Leeds <mwleeds endlessos org>]

Thanks for the reply Dan. I put some results from trying perf and strace on
github here: https://github.com/ostreedev/ostree/issues/2291

Seems like time is mostly spent on compression and --disable-fsync works as
advertised, if I'm reading them correctly.

On Wed, Mar 3, 2021 at 10:12 PM Dan Nicholson <dbn endlessos org> wrote:
> On Wed, Mar 3, 2021 at 7:18 PM Phaedrus Leeds via ostree-list
> <ostree-list gnome org> wrote:
> > Unfortunately the performance of the create-usb command is well below what one would expect from a modern 
> > computer: it takes about 63 minutes to copy 8092 MiB which works out to 2.14 MiB/s, on a computer that 
> > can easily do 85 MiB/s when copying much of the same data packed into a static delta with "ostree 
> > static-delta generate". At least two things might be the culprit of the slow performance when using 
> > create-usb instead of a static delta: (1) there is an inherent performance limitation when copying 
> > several thousand individual files instead of a static delta composed of a few files because of the extra 
> > work of filesystem operations at the stop and start of each file, or (2) the reduced performance is due 
> > to all the `fsync()` calls ostree makes during the copy. That latter theory seems experimentally ruled 
> > out because the performance is comparable if one uses `ostree create-usb --disable-fsync` followed by a 
> > `sync` command. The former theory probably is a large part of the problem but perhaps there is some way 
> > we can do better? Any ideas?
>
> Have you tried profiling it? perf could get a pretty good
> understanding of where the time is going. You might also try strace to
> confirm that ostree is really not calling fsync.