Re: platform compatibility policy

[Havoc Pennington <hp redhat com>]

Colm Smyth <Colm Smyth ireland sun com> writes: 
> QA and review are two ways to ensure that the enhancement in no
> way breaks compatibility. I like the idea of short release cycles,
> so I really like having automated test scripts and an ABI tester
> to assure confidence in each release.

I like the idea too, but we don't have _any_ tests right now and
haven't established that we're going to be able to get people to do
them.

If the policy read, "large interface additions should have been
thoroughly tested and should have automated tests" that would have the
effect of prohibiting them until we get better testing, which would be
fine. ;-)
 
I'm still somewhat concerned about the issue of developer
confusion. Say you want to target TurboLinux, Red Hat, and Mandrake,
and Solaris: at the moment you just need to know if they all ship
GNOME 1.4. But say they have 1.4.2, 1.4.3, 1.4.1, and 1.4.4. Which
interfaces is it OK to use? Are we going to document things in that
much detail?

I don't want to see people use an interface only to find it wasn't
"portable."

> You're making a distinction between a code change that adds an
> interface and a code change that doesn't. To me, any significant
> set of code changes needs to have some kind of review. If
> you "release" without some kind of testing and/or review, I would
> see it that you've just checked in some new code.

I agree with the principle.

With some exceptions, such as the C library and gcc (and GConf! ;-)),
the way free software mostly works now is that we rely almost entirely
on beta testing. One disadvantage of beta testing as a test method is
that it takes forever, and anytime you make any large code change you
have to reset the clock.

In stable branches, only very small bugfixes go into the code, because
they can be easily verified. In general the maintainer is responsible
for such verification but has to do it by basic manual testing and
just thinking things through. So they have to be small. This works in
practice, we haven't introduced much in the way of stable-branch bugs.
But it depends on the smallness of changes.

One concern at the moment is that we don't even have a reasonable test
harness for automated testing of GUI stuff.

I know with the help of some QA people at Sun we're starting to
address these issues, but we're far from having them solved...

> This is really an aside, but I don't know how many people have
> come across the concept of 'extreme programming'; 

I've read the book, and was impressed with its correspondence to lots
of the methods we use in free software, with the very notable
exception of the test suite idea. 

I was convinced of the value of the test suite by using one for GConf
(even though my suite for that kind of sucks and is incomplete, it
caught dozens of bugs and makes me much more confident about changing
things).

Havoc

_______________________________________________
gnome-hackers mailing list
gnome-hackers gnome org
http://mail.gnome.org/mailman/listinfo/gnome-hackers