Re: OSTree updates/pulls from LAN and pendrive



On Fri, Apr 15, 2016 at 5:52 PM, Colin Walters <walters verbum org> wrote:
On Thu, Apr 14, 2016, at 08:12 AM, Krzesimir Nowak wrote:
Hi,
I'm tasked with writing/updating a software that updates the OSTree based system from a bit different sources than the usual URL from the "remote" configuration. The possible sources are a computer on the LAN or just a pendrive.
 
Updating from LAN involves writing a server that publishes itself and then just serves the files from the OSTree repository. It would have a lot in common with ostree's trivial httpd, but with additional complication that the server's OSTree repo is usually of a "bare" type, not "archive-z2".
 

Yes.
 
 
 
1. Would an API (likely a single function) for making it easier to write a server serving what ostree pull would expect be welcome in libostree? It would likely take an OstreeRepo and a path relative to OstreeRepo's repodir and return a GInputStream via an output parameter (likely a GFileInputStream for content objects in archive-z2 repos or for non-content objects or deltas or summary files and a gzipping GConverterInputStream for content objects in bare repos). It would also return an error if path was outside of the repo directory (like ../../etc/passwd) or it was a directory or it was not not found. Something like:
 
I think the webserver should have an API like get/refs/foo/bar  get/objects/0c/9231 and use the API to look things up rather than directly referencing the content.  Before calling e.g. `ostree_repo_load_object_stream()` it would call `ostree_validate_checksum_string()`.

Yeah, as long as the API is similar to OSTree repo layout, so we can do ostree pull on it, I'm fine. Anyway, thanks for pointing me to the ostree_repo_load_object_stream function - I missed it.
 
 
2. Could we add another option to ostree_pull_with_options that overrides the URL used for fetching objects? Currently the remote_name_or_baseurl can be either a remote name or a file:// URL. When updating from LAN, we would like to pretend that the objects came from some specific remote despite using a different URL. That would basically mean using the override instead of ostree_repo_remote_get_url when remote_name_or_baseurl is not a file.
 
It doesn't matter to the core much right now *where* the objects came from.  If you pull from a local mirror, then perform a later update, we don't redownload objects we already have.

Sure, but ostree_repo_pull_with_options does use the remote_name_or_baseurl also to get some options (like URL, metalink, GPG verification, branches) and we want to temporarily override only the URL.
 
 
That said, I would like to support something like git's "alternates" file.

I'm not sure how git "alternates" fits into this use case. I never used it, but from what I understand (after quick documentation and code reading), you can use it to point git to other object stores on the same computer. So if it is on the server, client will know about additional locations of objects stores for fetching the objects. If it is on the client, it basically tells git not to copy objects but reuse some other local object store. There's also the http-alternates file with similar function, but that looks like a strictly server-side thing.

In our case the objects stores are on different computers.

 
 
 
3. We want to provide updates from a local filesystem. Sounds at first as a solved problem - ostree pull-local (or the call to the ostree_repo_pull_with_options function with a "file://" URL). But there are some things to consider:
- whether the update is easy to produce - means it would be best if we could just use some ostree library functions without messing in the repo directory ourselves or inventing too much of our own metadata.
- whether the update is easy to consume - look above, but on the client side. Ideally, the call to ostree_repo_pull_with_options should suffice.
- whether the update works for any previous version - it would be nice to have a single update that works on several computers with different older versions of the system.
- whether it really fits the limitations of a typical pendrive - it would be best for the update to be small as possible and contained in only a few directories/files.
We had some ideas for it, like:
 
I'm not fully following this discussion - things like "works for any version" can involve different tradeoffs, but the simplest seems to me to just have an archive-z2 repo with the full contents of the latest version, rather than trying to do deltas on the USB key...but that depends on OS size and such.

What would be the tradeoffs of using the archive-z2 repository with only the latest commit? From my small experiments we seem to lose the commit history, so I'm not sure if it is possible to roll back/checkout the earlier commits (they are unreachable at that point). Anything else?

I tried experimenting with doing "ostree --repo=<path/to/pendrive>/repo; ostree --repo=<path/to/pendrive>/repo pull-local --depth=0 --verbose /ostree/repo" as root, but got some strange problems that needs investigating (something like syscall 314 [sys_ioprio_set?] which returned EPERM, didn't have that problem when pulling to repo on harddrive instead).
 
 
It's hard to answer this without being a bit more concrete about sizes of different components and whether "any version" is a nice-to-have or requirement, etc.
 
 

_______________________________________________
ostree-list mailing list
ostree-list gnome org
https://mail.gnome.org/mailman/listinfo/ostree-list




--
Kinvolk GmbH | Graunstr. 43, 13355 Berlin
Geschäftsführer/Directors: Alban Crequy, Chris Kühl, Iago López Galeiras
Registergericht/Court of registration: Amtsgericht Charlottenburg
Registernummer/Registration number: HRB 171414 B
Ust-ID-Nummer/VAT ID number: DE302207000


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]