Re: Push to remote repo



Part of what we want to do is to allow pushing from multiple builders.
We have separate branches for dev, staging and production, and it
would be nice if we could run these builders independently and in
parallel. We could do it with a hokey push -- rsync over the objects
directory, then rsync up the individual ref file, but working around
ostree is really awkward.

On Wed, Oct 14, 2015 at 9:24 AM, Dan Nicholson <nicholson endlessm com> wrote:
I opened https://bugzilla.gnome.org/show_bug.cgi?id=756540 about this,
but Colin requested to discuss on list.

Currently the way you "push" a local repository to a remote is via rsync
or some other out of band mechanism. Unfortunately, non-ostree
mechanisms have no idea about the structure of the repository and can
only blindly copy the repository recursively.

This means the best you can do is commit builds directly into the public
repository or have a build repo that has 1:1 correspondence to the
public repository. I.e., you can "push" with rsync without --delete, but
you're still likely to update refs files you didn't want to. This is
especially an issue if the public repo is to host binaries from multiple
architectures since in that case it becomes much more difficult to keep
the build repos of each repository appropriately synchronized.

It would be awesome if ostree supported a push API like git does.
Ideally it would just use ssh as a transport to remain agnostic of the
auth mechanism, but I have no idea if this would require a full
protocol. Here's info on git's protocols, FWIW:

https://github.com/git/git/blob/master/Documentation/technical/pack-prot
ocol.txt

And here's Colin's response:

The lack of push was intentional; People out there push binaries into
production environments that came from developer laptops, which I
think is crazy.  There *are* valid use cases for pushing binaries
directly to stage without going through a clean CI/CD builder, but IMO
not production.

This to me is entirely orthogonal. If someone wants to push out binaries
they built on their laptops, they can find a way to do it. We _are_
trying to push from a clean builder, it's just that I want the builder
to be on a different server than the public repo. Since we're also
building ARM binaries, this would mean that the public repo would need
to be hosted on the same ARM machine, which seems entirely unnecessary.
In fact, we have multiple stages of public repos, and moving between
them is currently a pain.

For a delivery server doing binary-level promotion, you can use `pull
-local` today to migrate commits between two local repositories, or
have the target repository do a pull.

That ends up with a broken history on the production repo, so the next
step is to pull the commit, make a new commit with that same tree
content, then discard the pulled commit.  This needs documenting and
should probably have some built in workflow.

This again assumes that the builder and the repo host are on the same
machine, which seems unnecessarily restrictive. Currently, we pull the
latest commit from the public repo to maintain history, make a new
commit on top of it, then rsync the entire repo back to the public
server. All of this is really fragile, and the setup to make the public
server use pull-local would be pretty complex.

I'd love to hear others' thoughts on this since it seems like a hole in
the feature set to me. Obviously, I'd love to see the full
expressiveness of the git transports and refspecs, but anything to get a
single ref + objects to a remote would be great.

--
Dan
_______________________________________________
ostree-list mailing list
ostree-list gnome org
https://mail.gnome.org/mailman/listinfo/ostree-list



-- 
  Jasper


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]