Re: Git hosting

[Olav Vitters <olav bkor dhs org>]

On Wed, Mar 05, 2008 at 06:25:50AM -0700, Elijah Newren wrote:
> On Tue, Mar 4, 2008 at 9:44 PM, Behdad Esfahbod <behdad behdad org> wrote:
> >  On Mon, 2008-02-18 at 09:28 +0100, Olav Vitters wrote:
> >  > On Sun, Feb 17, 2008 at 03:46:29AM -0500, Behdad Esfahbod wrote:
> >  > > We were discussing this on #gtk+ the other day.  Seems like transition
> >  > > to a nextgen VCS, preferably git, would be needed in near future.  Many
> >  >
> >  > Please confirm the near future means early 2009. This so that the
> >  > conversion can be fully prepared and tested.
> >
> >  That would happen whenever it's ready to happen.  But yeah, early 2009
> >  sounds good.
> 
> It may be worth considering waiting a little longer; we might get a
> nicer solution.  I agree with Olav that all VCSes suck...in one way or
> another.
> 
> I'll try to help out with the migration whenever we decide on what and when.
> 
> >  > > I assume the merits of nextgen VCS tools are not disputed.  I personally
> >  > > prefer git because that's widespread on fd.o and many of us around GTK+
> >  > > stack already use it.
> >  >
> >  > Heh. I like that approach (known + used by fd.o). I'd still like other
> >  > requirements. Meaning: what is really needed? Or do you think all DVCS
> >  > are currently basically the same -- or some that aren't will be in ~6
> >  > months?
> 
> No, DVCSes are not the same and won't be within 6 months.  Keith's
> post a long time ago about "Repository Formats Matter" had all kinds
> of problems contentwise (as mercurial developers pointed out), but the
> title of the post was absolutely dead on.  Each VCS uses a different

Yeah, I read that, but it wasn't clear *at all*. I now understand you
want multiple branches within one directory.. or something. However, I
understood it as using sha1 everywhere.. which e.g. bzr has now too (so
I considered it as the same). Same for hg.

> format, and the format will often hinder or enable different
> capabilities.  The fact that SVN has taken YEARS to try to get useful
> merge functionality whereas many VCSes were able to implement it in
> weeks is a direct reflection of the fact that the SVN repository
> format is simply orthogonal to the needs of merging.  Sure, merging
> can be shoe-horned into the svn implementation -- which turns out to
> be exactly what they are doing, but they've hit all kinds of
> unexpected snags along the way that they keep on having to try to fix.
>  (Hmmm...maybe I should finish those notes of mine and make this a
> blog post)

SVN not being nice is well understood. :-)

> I crave the wonderful usability of bzr.  In my mind, it's a notch
> above the rest.  svn and hg aren't too far off, but bzr has some extra
> polish.  However, it has parallels to svn here in that the format is
> going to limit its utility to many people.  I was talking to Ian
> Clatworthy (bzr dev) about all kinds of bzr and DVCS issues; cool
> things in bzr, stuff to put in his writeups, etc.  He did his best to
> answer questions and sell bzr, but when I pointed out the most obvious
> and painful missing feature in bzr that I'd miss the most from using
> git -- being a branch container -- there was simply no response.  The

You mean multiple branches with one directory right? Why is that so
useful?

> ramifications of this single feature are huge, though -- it means much
> more than what you'd expect from a single listed feature.  In fact,
> when you look closely enough, you find that *several* of the features
> being developed by bzr in many ways amount to partial workarounds for
> this missing functionality.  Also, if you read through the bzr layout
> model and read over their documentation, you find that their
> assumption of branch==directory goes all the way to their very core,

IMO that is what makes bzr so good in usability. Easy to understand what
it does. I sort of get Git a bit more now, and I understand you can link
one repos/checkout/whatever to multiple upstream thingies.

> so this will be something that would be extremely difficult for them
> to change.  And I'd be worried about pushing bzr on the developers of
> large and low-level modules, because missing this feature would be
> painful to them.
> 
> And the odds of being able to fix git's UI within 6 months seems like
> a long shot.  You'd have to rip and replace their manpages (or at
> least hide them), since they are really worse than useless for new
> developers.  In fact, those manpages are really only any good for
> someone who knows enough to write their own porcelain for git (at
> which point they become very useful and even awesome).  And that's
> only the biggest problem, there's quite a number of UI warts to fix as
> well...
> 
> hg is kind of a middle ground here as far as these features are
> concerned, but a number of people have found themselves hitting a
> barrier with the hg implementation as well.
> 
> >  > > My proposal is to offer git hosting as an opt-in option, not switching
> >  > > all modules to it.  That seems to have worked for fdo, though more and
> >  > > more projects are moving.
> >  >
> >  > I am not in favour of Git, nor switching only some of the modules. I'd
> >  > like to move everything. This means the preparation is longer due to
> >  > more testing. However, perhaps we can switch gradually (again: not
> >  > rushed). Meaning: one module at a time.. but all modules will switch.
> 
> What's the rationale for having everybody use the same thing?  I will
> likely agree with it, but it is worth noting that some of the reasons
> for such a requirement may not be as important anymore.  I'll note the
> changes in progress already mentioned in this thread to allow

It more than doubles the cost of development if not all projects use the
same thing. Now I have to learn Git as well as Bzr (or whatever). Oh,
but I'm a sysadmin. So that means I can't assume most of anything
anymore. A project could be hosted in Git; maybe Bzr. To get useful info
I have to write scripts that assume Git first then Bzr, maybe something
else.
Of course, script being scripts is that something will always fail
unexpectedly (happens with SVN). So I fix bzr. Next day I'll learn about
some sudden failure of some Git command.


> translators to not have to use a VCS at all, and also the existence of

I like looking ahead, but transiflex != using a VCS. Further, ATM it
doesn't exist within GNOME. Meaning: someone has to do stuff to make
that happen.

> a simple tool called 'mr' which can be used to interact with projects
> under half a dozen different VCSes all using the same commands (sure,
> you're limited to the lowest common denominator, but that essentially
> amounts to the common operations of cvs/svn).

Hmm.. that sounds nice.

> >  > I know Git has something like:
> >  >   git://git.gnome.org/~username/myrepos
> >  >
> >  > Is such a thing wanted? IMO I'd rather have real repositories (like any
> >  > SVN user can create repositories ATM). For pet projects that aren't
> >  > suitable as a general GNOME repository, use something other than GNOME
> >  > to host it.
> 
> I think you'd even find people starting to want it with bzr or hg,
> really.  Sure, not everyone would want it, but the modules with many
> developers will at some point realize these capabilities and start
> clamoring for this kind of support...or else roll their own solution.
> Even among modules with only a few maintainers you'll see some
> adoptions of this functionality.  I've already seen these kinds of
> things spring up all over the place, and not just for git.

I don't think such a thing is usable. What is the benefit if you have to
be an insider to know about such things? E.g. if someone
branches/clones/whatever from gtk+, then I want a page about Gtk+ to
list the branches/clones.
Yes, you could have that in some personal space.. that is just an
implementation detail. If I'm some interested person in Gtk+, I wouldn't
want to find out 6months later  that e.g. behdad did a lot of nice work
in his personal space located *somewhere*.

> >  > Questions:
> >  > - What is the DVCS version of svn:external(s?)
> >
> >  Umm, I'm sure git has this feature.  Carl tells me about it all the
> >  time.  Don't remember the name right now.
> 
> git-submodule?  It's a joke IMO.  Nearly no useful functionality yet
> and already it has some nasty UI issues and a couple gotchas to boot.
> 
> (Great idea which would be superior to svn:externals, IMO, but the
> implementation is extremely lacking at best.)

Can you confirm the rest doesn't have this (Hg/Bzr)?

> >  >   - I've edited http://live.gnome.org/DistributedSCM.. could you please
> >  >     go over that? Please keep it 'post' integration. e.g. the SVN link
> >  >     is not important. Only that you can reliably convert the repository
> >
> >  Your changes look accurate, at least to me.  Regarding git having to be
> >  installed on the remote server, I don't understand, isn't that the case
> >  for all systems?  You sure need cvs server installed to serve CVS.
> 
> I haven't read the whole page, but I think you are referring to:
> "Publishing a git branch to your public webspace is not obvious (last
> time I tried) and requires git to be installed on the remote server.
> Not good if you have a shitty ISP."
> 
> No, I can publish git repos on a public website without git installed
> on the server.  I completely agree with it being unobvious in git
> (it's a series of 6 seemingly random incantations, some of which are
> not git commands).  However, updating such an uploaded repository
> without git installed on the server is essentially a big nasty script
> you write yourself or a mirror-only operation.  But then again, not
> sure that mirror-only would be such a big deal.
> 
> I'm not sure how this is relevant to GNOME, though...

Nah, it wouldn't be. When you have git on the server, it is easy though
right? (git 'easy')

> >  >     Note: tailor is *not* suited for converting GNOME modules! The DVCS
> >  >     should have tools for this (SVN->DVCS).
> 
> Absolutely, tailor is harder to figure out than git, IMO.

Plus it doesn't convert everything.

> Luckily, bzr, hg, and git all have tools for svn migrations.

You know the quality?

> >  > - translator things:
> >  >   - only checkout a file (highly preferred) / dir (like SVN)?
> 
> I disagree with this being a requirement; even the svn developers are
> considering dropping this feature (and appear to agree that it's worth
> it to gain performance and other abilities of DVCS systems).

There has to be a *known working proven* solution for translators or
we'll have a big problem. Transiflex is one idea. However, it has to be
accepted and *working*.

> >  >   - same for documentation things
> 
> As I've stated elsewhere, svn, hg, and bzr have good documentation.
> hg even has a book online to match the cvs and svn ones at red-bean,
> which is really cool.  Git's online tutorial is actually decent, but

I'll try that tutorial again. However, Git is severely lacking into
explaining the multiple branches thing, the 'index' and linking other
repositories. Further, there is a lot of usage of things you might
understand if you're used to Git, but I don't understand at all.
'master' ?!? Lots of those btw. Never sure if something is local or not.
How it all interacts. Plus you can't ignore it.

> their built in help is abysmal and worse than telling users "just try
> something -- good luck!" (even despite the nasty gotchas lurking with
> git).

I meant s/things/people/. I talked about l10n, but I didn't want to skip
shaunm etc.

> >  >   - *very* easy to commit to e.g. gnome-2-22 branch, 'trunk', etc
> 
> svn does this in one step, you can turn it into a single step in bzr
> (a combined local commit + push; see 'bound branches'), and in hg and
> git it will be just two steps (commit + push).
> 
> I don't think this rules out any system.

I couldn't see the branches within Git. You had to use some weird
command to see the upstream branches. For me that was another 'wtf'.

> >  One can write shell scripts that make it very easy to do so...  We can
> 
> Bah!  "The tool is broken but I know how to work around it".
> 
> The tool is stupid and lacking if you have to do this, IMO.
> 
> >  For example, with SVN,
> >  tagging (and branching too) is REALLY HARD.  I use the following script:
> 
> Yes, svn is also stupid and lacking.
> 
> >  > Note that:
> >  > - needs lots of changes in Damned Lies
> >  >   - must be done before the switch!
> >  > - buildbot should be fine
> >  > - we need to upgrade the SVN (version control) server.. waiting for new
> >  >   LTS, etc (will be done after 2.22.. half 2008 somewhere)
> >  > - switch might be delayed due to GNOME schedule (e.g. during freezes
> >  >   seems bad)
> >  >
> >  > - needs changes in various sysadmin scripts
> >  >   - forgot specifics
> >  >   - 'gnomeweb' thingy that rebuilds the various websites (there are
> >  >     *loads* of these)
> >  >   - rather not maintain multiple VCS in those systems.. too painful
> 
> Ah, here's part of the answer to my previous question.  Would be nice
> to have a list of these somewhere; it'll be needed regardless of the
> chosen migration path.

There is a *lot* of this. Going to be painful.. especially with Git.

> >  > - Need to have packages for RHEL5, Ubuntu this years LTS, Fedora8,
> >  >   possibly Debian (3.0 IIRC)
> >  >   - should have backports when new versions come out.. or easily
> >  >     buildable (meaning: I can do rpm, not deb)
> 
> I'm pretty sure all systems satisfy this.  I've even rebuilt my own
> rpms of bzr, hg, and git since Fedora wasn't as aggressively following
> their release schedules as I was (I didn't want to wait more than a
> couple days in some cases), and it has been very simple.  I've also
> rebuilt the svn rpm (to get bzr-svn working); that one was by far the
> most painful, though mostly just a long waiting game.

It isn't even about packaging. It is about maintenance. I wake up,
notice that Red Hat released some security updates for X packages...
this is in an email where it specifies which servers have been updated
and which couldn't be.
I don't watch security announcements. Job of the distribution, same for
important bugfixes.

> >  > - I *really* *really* like something with good and easily usable Python
> >  >   bindings (better that SVN one.. feels like C in Python)
> 
> I haven't looked at the bindings side myself, but from what I've read
> bzr would be the winner here, then hg, and git doesn't have much of
> anything at all.  (git is really nice from a shell scripting side...in
> fact, it seems like it was designed for someone else to write a
> wrapper around it...oh, wait, it was.  Too bad no one has made a good
> one yet and published it)

I hope the existing user base of Git still allows them to reimplement
all commands.

> >  > - All DVCS can do something like svn export
> >  >   $vcs://$vcs.gnome.org/repos/path/$FILE right?
> 
> Yes, they all have it, though they don't idiotically require a URL
> like svn (another example of stupid UI that needs a script): svn
> export, bzr export, hg archive, git archive.

So you can only export a file right? No need for the whole history? Plus
it is *fast* even when done externally?

-- 
Regards,
Olav