OSTree has a command, create-usb, for copying refs
to a USB drive (or really any filesystem) so they can be used for
offline updates. This command is used (along with the similar "flatpak
create-usb") by the eos-updater-prepare-volume
script
used in Endless to copy the OS along with any necessary flatpaks to a
USB drive for use in offline updates (where in this case offline means
not connected to the Internet).
Unfortunately
the performance of the create-usb command is well below what one would
expect from a modern computer: it takes about 63 minutes to copy 8092
MiB which works out to 2.14 MiB/s, on a computer that can easily do 85
MiB/s when copying much of the same data packed into a static delta with
"ostree static-delta generate". At least two things might be the
culprit of the slow performance when using create-usb instead of a
static delta: (1) there is an inherent performance limitation when
copying several thousand individual files instead of a static delta
composed of a few files because of the extra work of filesystem
operations at the stop and start of each file, or (2) the reduced
performance is due to all the `fsync()` calls ostree makes during the
copy. That latter theory seems experimentally ruled out because the
performance is comparable if one uses `ostree create-usb
--disable-fsync` followed by a `sync` command. The former theory
probably is a large part of the problem but perhaps there is some way we
can do better? Any ideas?
Generating static
deltas and copying them instead may be an option, but it comes with
computational and disk space costs, and it's not clear to me if it's
backwards compatible in the sense that existing computers would be able
to find and use updates formatted that way.
Making
the copy multithreaded might also be an option, but I'm not sure how
much work it would be, and at least in some quick testing invoking rsync
via xargs to copy a commit checkout, it didn't seem to help.
For
context on why this matters to Endless, the OS Copy to USB feature is
so slow that users often think it's broken, and the situation is of
course much worse if USB 2.0 is used.
Somewhat
relatedly, I would like to work on making libostree API for create-usb
soon, as part of the effort to get offline updates support in upstream
gnome-software (included in the ongoing
UI refresh).
Among other benefits it would mean gnome-software can get proper
progress reporting for app/OS copies. If anyone has a strong opinion on
what that API should look like I'm all ears. I think we probably want a
GVariant parameter for options so it's extensible.
Looking forward to hearing any thoughts on this,
Phaedrus (Matthew) Leeds
Endless OS Foundation
