Re: [ Revised Proposal ] Continuous Builds in GNOME

[Tristan Van Berkom <tristan vanberkom codethink co uk>]

First,
  I've been on vacation this week and have been able to put time into
this, with the hope that next week I will be able to focus on $dayjob
without regretting too much not being able to take part in this
important debate. I hope we are going somewhere. Also, in closing in
this mail, I will just highlight my motivations for bringing up this
proposal in the first place.

Sri,
  Please just gloss over the technical stuff and read the end, I think
you are in a position to effect change and this is written with you in
mind (from the "Why do I think..." heading).

On Fri, 2016-06-17 at 15:17 +0200, Milan Crha wrote:
> On Fri, 2016-06-17 at 17:11 +0900, Tristan Van Berkom wrote:
> > I dont believe you have any such functionality in jhbuild.
> >
> > At best, you can specify the sha1 of the commit you want to build
> > of
> > a given module, this would not let you single out one commit,
> > omitting it from history in integration and at least attempt to
> > apply
> > subsequent commits without that one included, re-including that
> > commit in the correct place in history only when CI infrastructure
> > would find integration unbroken.
>       Hi,
> I never thought of a single commit removal from a set of commits. To
> be
> honest, that sounds even more crazy (not 'crazy' like 'insane'). :)
> We are talking about master, the development version of the software.
> If you'd like to tell me that you are at commit X and feature which I
> added at commit X-3 does not work for you, then trying to figure out
> why exactly you do not see that feature is a real nightmare and waste
> of time for both the reporter and the developer, especially when you
> want to remove only single commit from a history. Not talking that
> following commit can "build on top of the previous commit", which is
> quite usual.
>
> > The thing is, either way; your work flow will not be interrupted at
> > all with this approach, ...
> See above.

Milan,
    I think that if your commit broke integration with the wider GNOME
ecosystem, that will matter to you and you will be quite aware of which
commit in master broke it and why until it is fixed. I think we have to
ensure that people take integration breaking commits seriously and
ultimately, an integration breaking commit would block any stable
release.

That said, in your above scenario; a disparity between master and
integration is a serious issue, but consider the alternative:

If a new comer wants to get involved in GNOME, they probably have a
specific itch to scratch, say with Evolution or EDS; they will either
be able to build everything up to EDS & Evolution easily; have an easy
experience, and be able to submit a patch to bugzilla, or, they will
fail to build and get side tracked because the bleeding edge of master
for all GNOME modules doesn't build.

In other words (and this has absolutely nothing to do with using
jhbuild or not) when you build everything from master and it breaks
before you are even able to work on the issue you wanted to work on,
this is discouraging; this means that we are offloading our technical
debts to newcomers, forcing them to deal with our bugs before being
able to even submit a patch they wanted to contribute.

Would you prefer that a potential EDS contributor be able to submit a
patch to bugzilla, even if it does not match up *exactly* against
master ? Or would you prefer that your potential contributor get
discouraged while trying to build EDS against the latest gcr,
libsecret, libgdata or such, or just get bogged down trying to fix an
EDS bug to adjust to some API churn in a dependent GNOME library ?

I think receiving a patch for what the contributor wanted to work on is
preferable, even if you happen to fall on a patch that doesn't apply
exactly against master, in the 1% of the time that EDS integration is
broken and lagging behind.

[...]
> I said earlier in the thread that the current situation works for me
> the best. There is clearly stated what the continuous builds from, at
> which exact commit it "stopped updating" certain module, because of
> some breakage, and everything references the real master branch. That
> the continuous uses jhbuild might be just a coincidence, even it
> sounds
> like the moduleset is targeted for the Continuous builds, when it can
> influence an environment for any developer using the jhbuild (I do
> not
> use it, this is how I understood how it works in the jhbuild world).

I'll just note again this is not about jhbuild particularly, jhbuild in
master tries to build everything from master, sometimes it breaks and I
think that's natural in the present state of affairs.

gnome-continuous does *not* build jhbuild modulesets, it uses it's own
json format for, also building everything from master:

    https://git.gnome.org/browse/gnome-continuous/tree/manifest.json

This is about getting the bleeding edge, or something as close as
possible to the bleeding edge to be always buildable; for two reasons:

  a.) To quickly be able to identify exactly which commits are blocking
      the integration of every bleeding edge in GNOME

  b.) To allow new contributors to be able to submit a patch against
      our projects without getting bogged down in solving our own
      technical debt.

I can see that you think the approach I propose is wildly complex, I
don't really agree on that; but; I think we do agree that we should
choose the path of least resistance, the least complex path to
satisfying all the requirements of this endeavor. But we really have to
solve all the requirements in order to provide a reasonable alternative
to policing commits directly in master and introducing revert noise in
transitional periods.

At this point I wont go over the technical merits of which solution
will work, as pointed out this is not a jhbuild only issue so managing
this in a special jhbuild module set is not really a viable alternative
either, I'm happy to debate other simpler alternatives if provided.

Instead, as I don't know what time will be afforded me to continue this
conversation, I just want to highlight my own motivations for spending
time on this proposal.


Why do I think policing master is a hostile move ?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The original proposal works, the "lets force everyone to work closer
together" and make master always buildable, well, it almost works
except for the few cases where we make things difficult for
refactorings and API churn where things will inevitably break, but
overall it works - but it _only_ works if we all share the same vision.

Why do I consider this to be a hostile approach ? This is a deep
question, it's not _only_ because it means someone can step in and
revert things forcibly, after an extremely short period of breakage (a
day or two ?).

No, I consider this to be hostile because we would be dictating that
every project's priority and primary objective should be to create a
unified desktop (or tablet ?) experience. We are basically assuming a
sort of "play with the desktop team or don't play at all" mentality,
and this makes me sick, really.

Now, I'm sorry to pick on clocks again but really, since the early 90's
we have had thousands of incarnations of clock applets, binary clocks,
fuzzy clocks, we've seen it all really. Designing a cute new clock app
for the beauty of the desktop is a nice thing to do, and it's an easy
thing for young blood to work on - I hope to turn these new gadget
developers into strong engineers over time, but I want the best of the
best to *want* to play on our team.

Why do people devote their time to working on some free software ? I
mean asides from those who are lucky enough to get paid to do so, why
do people devote their time to FOSS ?

My answer; people look up to us, to the individual projects and their
maintainership, people want to write gedit, people want to write GTK+,
people want to participate in a project that tries to be the absolute
best at what it does.

Sometimes being the best text editor is not alway inline with adding an
awkward headerbar which only makes sense if integrated into the GNOME
desktop environment, for example. In these cases we need to be
understanding that the reason we have gedit in the first place is
because of the good graces of Paulo Borelli et al, and that they
probably know what is best for a good text editor; we have to be
understanding that they probably want to not *only* be the best text
editor in GNOME, but just the best free software text/code editor that
exists for the featureset they provide.

By taking a stance that: "If your project is a part of GNOME, then
automatically you care about the GNOME Desktop Environment more than
anything else" is discouraging to those developers who just want to
make the best software that exists, this is after all the spirit of
FOSS, this is what attracts us in the first place - I can't think of a
worse move for GNOME than to try to bring all of these contributors
inline, and sacrifice their idealism for the sake of a fancy new
headerbar. Now I'm picking on headerbars but really; they are really
fine and nice; but we have to introduce things like this which make the
GNOME experience better without for example, sacrificing the user
experience of using our best softwares under KDE or other environments
(headerbars in Ubuntu are getting better but were a bit of a disaster
at first).

In closing, I think we have become rigid over time and that by pushing
the "GNOME Desktop Environment" agenda so vigorously in recent years we
have become unwelcoming to those great minds and idealists which just
want to create the best software that exists. Trying to reconcile
everything under the GNOME umbrella to be always buildable in master is
like a final nail in the coffin, it's where we draw the line and what
was once a great ecosystem of amazing software becomes; merely a
desktop environment with modules that are completely only self serving.

Best Regards,
    -Tristan