Re: Moving existing ostree users to a new branch - take 2

[Daniel Drake <drake endlessm com>]

On Thu, May 4, 2017 at 8:01 AM, Alexander Larsson <alexl redhat com> wrote:
> In fact, i'm currently looking into doing this, and the design will
> probably be quite similar. i.e. some metadata in the summary file, that
> if set will rewrite the configuration of the remote.

We should definitely sync on this then. And hopefully anything I
contribute at the ostree level can also be used by flatpak, at least
partially.

In my current implementation, I place data in a currently-unused a{sv}
field attached to the new ref. But if we're now going to allow
redirects to other remotes, we'll need to move to to the overall a{sv}
field at the end of the summary file, where we could introduce
one-to-one redirects like:

 redirects: {
   old_ref: (new_remote, new_ref)
 }

(new_remote could be NULL/empty if the new_ref is in the same remote)

We'd need a canonical place to store redirect info (which is later
included in the summary file). Perhaps a new file "redirects" inside
the ostree repo (same dir as 'config'), something like:

   [old_ref]
   new_remote=https://...
   new_ref=foo

> However, I don't think modifying the repo configuration in the middle
> of a pull operation like this patch does it is right. For instance,
> flatpak has a much more complex use of repos to do "pull as user then
> install systemwide using polkit". Changing the config where we pull is
> a no-op as we essentially throw that away, and changing the system one
> needs more complex operation.

We do need to store the redirects data on the client somewhere, in
order to support separate "pull" (network access) and "deploy" (no
network access).
I saw that the pull operation may cache the summary file but I imagine
it is not correct to rely on a cache (which could be purged before
deploy).
It would be nice if we could mirror the data format on both server and
client. In the above proposal, the client would manipulate the same
kind of "redirects" ini file; or would that run into the same kinds of
problems for flatpak?

> In the case of forwarding the entire remote there is further complexity
> in that you need to re-download the summary if the URL changed. Also,
> the branch name changing affects a lot of extra things in flatpak,
> since this changes the application id, and these are things that ostree
> doesn't know about.
>
> So, instead of hiding this in the pull operation we will do this in a
> separate step that flatpak runs before it tries to install/update
> something. I think something similar is probably better for ostree
> admin update too.

Makes sense. Perhaps something like:

A new ostree_sysroot_upgrader_check_redirect() function which
downloads the summary file and stores any relevant redirects info
locally. This would be run in the the "ostree admin upgrade" pull step
but not if --deploy-only is used (which presumes that pull was run
previously).

A new flag OSTREE_SYSROOT_UPGRADER_PULL_FLAGS_FOLLOW_REDIRECT which
will cause ostree_sysroot_upgrader_pull() to read the locally stored
redirects info, rewrite the remote config (if the remote changed), And
it would modify the OstreeSysrootUpgrader in memory to point at the
correct new remote/ref. Then (when not in deploy-only mode) the pull
would happen from the new remote, and the new ref.

Upon deploy it would use the data stored in OstreeSysrootUpgrader in
order to deploy the correct new ref.

And presumably at the implementation level we'd find a way to have the
core part of that in the lower layers so that it can be shared with
flatpak (which I presume is not using OstreeSysrootUpgrader) - or we
do that as a second implementation step.

Thoughts? Please correct/improve any of the above ideas, I do not have
much experience in the ostree codebase, but if I can continue getting
help on the design I'll keep working on the implementation.

Thanks,
Daniel