On Mon, 2017-03-20 at 15:14 +0000, Philip Withnall wrote:
On Fri, 2017-03-17 at 09:21 -0400, Colin Walters wrote:On Wed, Mar 15, 2017, at 12:55 PM, Philip Withnall wrote:The format would need to scale to multiple repositories (for example, if a machine used OSTree for the OS, and flatpak for apps). Any machines on the local network which advertise repositories matching the local repository’s canonical remote URI,Maybe instead we should have a UUID field in repositories in the repo/config? We could easily change `ostree init` to generate one. (Although it'd be prone to being copied if people used `rsync`)Talking to Alex today, this is something which could potentially cause problems with flatpak, even if we found some way of determining equivalence classes between remotes: flatpak only allows app updates from the same named remote (not even a differently named remote with the same URI). I assume this could be changed in flatpak, but it’s something we should think about first. Again, it comes down to the question of how we determine whether two remotes should be considered equivalent, such that pulling from either one of them to a local repository would be acceptable.
Changing it in flatpak is a fairly major task: a lot of flatpak makes the implicit assumption that each app has exactly one remote, which does not change.
We could do it on the basis of GPG keys, given that each organisation should have a different GPG key. But then we’d want to use another property as well, because one organisation might have multiple classes of repository. For example, Endless might sign both its flatpak and OS repositories with the same GPG key — but we wouldn’t want to try and pull from the OS repository on a network peer into our local flatpak repository.
I think I’m going to put a branch together which adds a new URI scheme to libostree, of the form ‘ostree-gpg:ABC123’, where ABC123 is the GPG key ID for the repository to pull from. Resolving this URI would look for remotes which use that key and are advertised on the local network or locally mounted disks (or, as a fallback, in a config file which maps GPG key IDs to HTTP remotes on the internet). This would require that vendors use different keys for different classes of repository, but that’s good hygiene anyway. This approach basically uses the GPG key ID as a pre-existing unique identifier for a remote, and abstracts URI resolution through it. This will require adding some API to libostree to get the result of resolving the ‘ostree-gpg:’ URI to HTTP, which will typically happen when a user of libostree pulls the summary for a repo. The user can then either pass the resolved HTTP URI back into ostree_repo_pull() as a URI override to pull objects; or can pass the ‘ostree-gpg:’ URI back in and potentially have it resolve to a different network peer to pull the objects from. This approach avoids the problems of conflating HTTP mirrors with separate remotes which are potentially not in sync with each other, which was a problem with my earlier plan to use some form of mirrorlist= for this. However, this approach doesn’t currently allow for parallelisation of downloads — for example, if an ‘ostree-gpg:’ URI resolves to multiple local network peers, it would be faster to download from all of them in parallel, and to handle failover from one to another within ostree_repo_pull(). I think the problem of parallelisation should be solved separately — since it applies to all kinds of libostree remotes. Since OSTree is content addressed, it should be opportunistically possible to pull data from any remote. The other problem related to this solution is that of key rotation. If the GPG key ID is used as an identifier for finding remotes, it can never be changed (or all the remotes for a local repository will effectively disappear until the repository configuration is changed to use the new GPG key ID). I don’t think this should be a problem if key rotation is handled using subkeys, rather than the master key with the given ID. The subkeys can be rotated instead. This does depend on OSTree supporting subkeys everywhere, which is something I haven’t tested. Philip
Attachment:
signature.asc
Description: This is a digitally signed message part