Re: OSTree updates/pulls from LAN and pendrive

[Colin Walters <walters verbum org>]

On Thu, Apr 14, 2016, at 08:12 AM, Krzesimir Nowak wrote:

Hi,

I'm tasked with writing/updating a software that updates the OSTree based system from a bit different sources than the usual URL from the "remote" configuration. The possible sources are a computer on the LAN or just a pendrive.

 

Updating from LAN involves writing a server that publishes itself and then just serves the files from the OSTree repository. It would have a lot in common with ostree's trivial httpd, but with additional complication that the server's OSTree repo is usually of a "bare" type, not "archive-z2".

 

Right, this is https://mail.gnome.org/archives/ostree-list/2016-March/msg00000.html

 

 

1. Would an API (likely a single function) for making it easier to write a server serving what ostree pull would expect be welcome in libostree? It would likely take an OstreeRepo and a path relative to OstreeRepo's repodir and return a GInputStream via an output parameter (likely a GFileInputStream for content objects in archive-z2 repos or for non-content objects or deltas or summary files and a gzipping GConverterInputStream for content objects in bare repos). It would also return an error if path was outside of the repo directory (like ../../etc/passwd) or it was a directory or it was not not found. Something like:

 

I think the webserver should have an API like get/refs/foo/bar  get/objects/0c/9231 and use the API to look things up rather than directly referencing the content.  Before calling e.g. `ostree_repo_load_object_stream()` it would call `ostree_validate_checksum_string()`.

 

2. Could we add another option to ostree_pull_with_options that overrides the URL used for fetching objects? Currently the remote_name_or_baseurl can be either a remote name or a file:// URL. When updating from LAN, we would like to pretend that the objects came from some specific remote despite using a different URL. That would basically mean using the override instead of ostree_repo_remote_get_url when remote_name_or_baseurl is not a file.

 

It doesn't matter to the core much right now *where* the objects came from.  If you pull from a local mirror, then perform a later update, we don't redownload objects we already have.

 

That said, I would like to support something like git's "alternates" file.

 

 

 

3. We want to provide updates from a local filesystem. Sounds at first as a solved problem - ostree pull-local (or the call to the ostree_repo_pull_with_options function with a "file://" URL). But there are some things to consider:

- whether the update is easy to produce - means it would be best if we could just use some ostree library functions without messing in the repo directory ourselves or inventing too much of our own metadata.

- whether the update is easy to consume - look above, but on the client side. Ideally, the call to ostree_repo_pull_with_options should suffice.

- whether the update works for any previous version - it would be nice to have a single update that works on several computers with different older versions of the system.

- whether it really fits the limitations of a typical pendrive - it would be best for the update to be small as possible and contained in only a few directories/files.

We had some ideas for it, like:

 

I'm not fully following this discussion - things like "works for any version" can involve different tradeoffs, but the simplest seems to me to just have an archive-z2 repo with the full contents of the latest version, rather than trying to do deltas on the USB key...but that depends on OS size and such.

 

It's hard to answer this without being a bit more concrete about sizes of different components and whether "any version" is a nice-to-have or requirement, etc.