Communications in Projects...

[jg pa dec com (Jim Gettys)]

I promised to draft this last summer: I did, but did not see a good
opportunity to send it when people were listening (the battles of
last summer had died down by the time I drafted this), and then was out for
surgery.  So here are some thoughts, I hope we can have a useful discussion.


			Communications in Projects
As the size of projects increases, the need for discussion and communication
increases.

This is a direct consequence of scaling and interdependencies of projects.

Gnome itself has become a very large "umbrella" project, and itself depends
on other projects: e.g. GTK+, Linux, XFree86, various libraries, etc.

Similarly, different parts of the Gnome project itself are more or less 
dependent on each other.  So we have certain core libraries, that effectively 
everyone depends on, Bonobo, which we'd like more applications to use 
(and therefore depend on), and so on.

For example, we all depend on stability/progress in GTK+ (and the underlying 
X Window System).

There are some inferences we can draw:

	1) some interfaces become involate with time.  For example, the X Window
	System has provided absolute upward compatibility since 1988 
	(approaching 14 years now). 
 
	I think people all agree that we'd be shooting ourselves in the
	feet to make incompatible changes here (to use an example that does not
	pick on any Gnome technology).

	Technologies in this catagory can only move upward in compatible ways: 
	recent X examples of compatible change are the Render and RandR extensions.
	
	Network protocols are hardest to change, as you have no control
	over the other endpoints, and the value is precisely the ability
	to interoperate.  And this can be frustrating, as you really get
	to live with and appreciate your mistakes :-(.  And down-revision
	network services provide a challenge to interoperability.

	2) some interfaces can be versioned (and things successfully coexist
	simultaneously.  For example, there are multiple versions of Xt, and GTK
	in the world, and things can/do work.  But among a single major version,
	version, similar compatibility constraints to 1) exist.

	3) some technologies allow for multiple co-existing environments,
	for example, language environments.  Programmers vote with their feet, 
	and it is healthy for there to be competing projects.  Inside such
	a technological island, 1, and 2, still apply.

Ok, what does this say about components of the Gnome project?

	o The consequence of successful components (libraries, embeddable 
	applications) is that more and more people use them. 

	o The consequences of this success is that incompatibilities/changes 
	affect many more people, so change is much harder.

	o As things succeed, there is a natural progression to 2), and sometimes
	to the 1) endstate (if competing technologies are not possible: as ESR
	has pointed out, some technological winners are "winners take all".

This has implications on our own behaviors:

	o if you work on a part of the gnome project (e.g. many applications)
	that no one else depends on, you can be your own little "god".  
	If people don't like your behavior, they'll build a replacement.  Little
	communications of your plans are needed or required (though I'd still
	strongly recommend such behavior if you want anyone else to help you!).
	The project as a whole does not suffer if you are as secretive as you
	like (though end users may not like you!).

	Note, however, that many projects need to grow beyond what a very
	small project with such autonomy can handle.  Natural selection
	effects are very strong in open source projects, and your prima
	donna project is likely to be supplanted by others if you behave
	this way.  Selection is as often on personalities and behavior as
	on technical merit!

	o if you work on a library or component others depend on, you have
	a responsibility that increases with your success to gather input,
	get feedback, communicate plans, etc.  This is particularly critical whenever
	incompatible changes are needed/planned, as this generates work for others,
	and their buy in is essentially required for your (and the project's)
	success.  Projects can live or die on their ability to handle change.

	o if development of key technology is mostly done in a small
	teams, particularly when commercially supported, much paranoia can
	be avoided by good communications of plans, enhancements
	and changes (particularly incompatible ones that affect other 
	projects).  Recommendation: do all your development on open
	mailing lists, reserving a infrequently used list for proprietary,
	commercial discussions.  If you already have heavily used internal
	only mailing lists, habits die hard, so turn them off and reestablish
	public lists to force this separation, or your developers won't
	bother to switch.

	This is not only true for commercial companies: for what 
	seemed like good reasons years ago, XFree86 behaved much as a closed
	group for years, and I think most people in Gnome appreciate
	the fact that they've opened up.  Arrange your lives to minimize
	the "secret" communications necessary.  The needs are real, but we
	should avoid as much as we can such secrecy: when unnecessary,
	it is poisonous to open source projects.

	o make incompatible changes as early as possible in major versions,
	to give time for down stream changes to percolate through: human
	communication takes time!  And taking responsibility to identify
	who is likely to be impacted, and how much, is in order.  This
	homework should reduce the anguish of releases.

	o if you are working on a "winner take all" part of gnome, your
	responsibilities for stability and careful compatible evolution
	is paramount, and good communications becomes absolutely vital.  
	If you don't like that responsibility (and the constraints and work 
	it imposes), working elsewhere in the project is probably wise.

	o and being civil to each other really helps the communications,
	along with an attitude of "do not attribute to malice what can be
	attributed to incompetance".  Lets all treat ourselves well,
	and avoid "ad hominem" attacks.  Don't presume bad motives of people:
	more commonly they have good reasons, or just blew it entirely!
	
I hope this sparks serious discussion.
				- Jim


--
Jim Gettys
Technology and Corporate Development
Compaq Computer Corporation
jg pa dec com