ostree create-usb performance

OSTree has a command, create-usb, for copying refs to a USB drive (or really any filesystem) so they can be used for offline updates. This command is used (along with the similar "flatpak create-usb") by the eos-updater-prepare-volume script used in Endless to copy the OS along with any necessary flatpaks to a USB drive for use in offline updates (where in this case offline means not connected to the Internet).

Unfortunately the performance of the create-usb command is well below what one would expect from a modern computer: it takes about 63 minutes to copy 8092 MiB which works out to 2.14 MiB/s, on a computer that can easily do 85 MiB/s when copying much of the same data packed into a static delta with "ostree static-delta generate". At least two things might be the culprit of the slow performance when using create-usb instead of a static delta: (1) there is an inherent performance limitation when copying several thousand individual files instead of a static delta composed of a few files because of the extra work of filesystem operations at the stop and start of each file, or (2) the reduced performance is due to all the `fsync()` calls ostree makes during the copy. That latter theory seems experimentally ruled out because the performance is comparable if one uses `ostree create-usb --disable-fsync` followed by a `sync` command. The former theory probably is a large part of the problem but perhaps there is some way we can do better? Any ideas?

Generating static deltas and copying them instead may be an option, but it comes with computational and disk space costs, and it's not clear to me if it's backwards compatible in the sense that existing computers would be able to find and use updates formatted that way.

Making the copy multithreaded might also be an option, but I'm not sure how much work it would be, and at least in some quick testing invoking rsync via xargs to copy a commit checkout, it didn't seem to help.

For context on why this matters to Endless, the OS Copy to USB feature is so slow that users often think it's broken, and the situation is of course much worse if USB 2.0 is used.

Somewhat relatedly, I would like to work on making libostree API for create-usb soon, as part of the effort to get offline updates support in upstream gnome-software (included in the ongoing UI refresh). Among other benefits it would mean gnome-software can get proper progress reporting for app/OS copies. If anyone has a strong opinion on what that API should look like I'm all ears. I think we probably want a GVariant parameter for options so it's extensible.

Looking forward to hearing any thoughts on this,

Phaedrus (Matthew) Leeds
Endless OS Foundation

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]