Here is my interview with Laszlo about working upstream. I'm open to any suggestions on editing. (I'm getting a bio.)
I want the article to be useful to other companies that might be having trouble working upstream.
Stormy
-------------
Laszlo, many companies have trouble working upstream. I understand that
Sun made a big push a few years ago to work more effectively upstream
and it's been very successful.
Q: Can you describe the problem that Sun was having before you started your upstream push?
A: When we started to work on porting GNOME to Solaris, it always took
us a good while to get to the point where we could build it and use it.
We
needed lots of code changes. By the time we finished, a newer version
of GNOME was out. The new version would be equally broken on Solaris
and most of our code changes didn't apply to the new version, so the
whole process started again and we never seemed to catch up.
We
realized that we needed to work with the GNOME developers to integrate
our patches in the upstream code so we wouldn't need to do this for
each and every release.
Working upstream was good in theory, but in practice, our
patches were all 6 months behind. Sending patches that were against
6-month-old code didn't help much because the code had moved on and the
maintainers were not interested in forward-porting our patches. So we
realised we have to work with the latest and greatest community code
and send patches as soon as we notice that the code breaks on Solaris.
Q: What does it mean to push something upstream?
A: First of all, writing patches that are acceptable in the
community code base (i.e. a fix for Solaris doesn't break the code on
other OSs) and then sending it to the maintainer/list/bugzilla as soon
as possible, so they can easily incorporate it in the community code
base.
Q: What did you have to change internally? What did you have to do differently in order to work upstream?
A: We had to adopt our internal processes to make sure we didn't start lagging behind the community code base again.
We need to keep working with the latest code, reporting issues upstream as soon as we find them.
Catching up is much harder than keeping the pace.
Just like upstream, we have two active branches of our desktop: the
latest stable version goes into Solaris and goes to our customers, and
we only fix the really nasty bugs in this branch. The development
branch is what we concentrate our efforts on. We keep updating the
development branch with the latest GNOME development tarballs as soon
as they are released and each tarball has an internal owner who is
responsible for updating internal patches when they break. If they do
their jobs well and get their patches upstream, they have less work to
do. That's quite an incentive. :)
Another incentive for
Sun developers to write good patches is that we require internal review
of all patches that are not pushed upstream, whereas, if a patch is
pushed upstream, we trust the maintainer's opinion so it need not be
reviewed internally.
Q:
Does that mean you have more people working on GNOME now? Do you have
to have an internal developer for every external GNOME maintainer?
No.
Like in the community, one person can maintain more than one package.
Since our internal maintainers are only responsible for maintaining
(or rather: reducing) the difference between the upstream code and
Sun's code, they are able to maintain many packages.
Q: Was it a hard sell internally?
A: Not really. It makes a lot of sense to work this way.
Q: Did pushing things upstream slow you down?
A: On the contrary. It made us more efficient than ever before. We
manage more packages, keep them better updated and they work much
better on our platform, out of the box.
Q: Do all of your changes make it upstream? What happens when they don't?
A: No, not all changes are accepted. There are different situations.
The worst one is if a patch we submit is ignored. In that case we
don't know if it's acceptable or needs work. In some cases the
upstream maintainer has different ideas about a fix. We want our
engineers to negotiate and rewrite their patches if needed, until they
are accepted. Finally, in some cases the maintainer disagrees with
the bug report itself and is not willing to accept the changes in any
form. In this case we need to re-evaluate the bug internally.
If we decide that we still want it, because our customers want it,
we will keep the patch. These are patches that we really hate because
they need to be merged with the upstream code changes all the time.
Q: What types of things do you push upstream? Are these Sun specific
features? Things just your customers have run into? Things you think
would be cool?
A: Preferably everything. In the case of features, it's a good
idea to work with the upstream developers from day one and maybe
develop in a branch of the upstream SCM.
Q: Are you still able to get customers the products they need, when they need them and still work effectively upstream?
A: Yes. We can make the changes that the customers require in
the stable desktop and not necessarily push that specific change
upstream. At the same time we come up with a long term solution for
the problem and implement it in the latest development code.
Q: Do you push out changes to customers and then push them upstream? Or
do you push them upstream and make sure they are accepted before you
use them in customer products?
A: We concentrate on the development version of the desktop and
push most fixes upstream before the stable release. Patches that do not
get accepted before the stable release will make it in the customer
product first.
Q: When someone in the company is new to open source, how do you educate them on how best to work with the upstream community?
A: Working with the community is part of the process, if you
like, part of our job description. In my experience most people get it
quickly. Others may need a little push.
Q: What advice would you give a company trying to work upstream?
A: Encourage your developers to communicate and work closely with
the community. Send them to developer conferences and summits to meet
them face to face. It's much easier to work with someone when you can
put a face to a name and code changes are more easily accepted after a
beer or three. :)
Q: What could the community do better to help companies like Sun work effectively with upstream?
A: Review our patches in a timely manner. The worst feedback is no feedback.
Q:
For companies looking to work more closely upstream, can you give them
an idea of how hard this was to implement? Did it take a lot of
convincing? A lot of time upfront? Are you saving a lot of time now?
When a company decides to use code from an open source project,
they give up control over the changes and directions of the code base. They become part of the community and can influence
decisions just like any other community member, but ultimately they lose control. Their best shot is to participate in the
community and understand, discuss, influence design decisions.
However, the company still has full control of their internal processes. They can and have to define them such that they
can
provide what their customers want while they still go with the
community and not against it. The initial implementation is likely to
take significant effort, but they will save that later.
On Wed, Jun 3, 2009 at 9:05 AM, Paul Cutler
<pcutler foresightlinux org> wrote:
Hello GNOME Journal team!
Just a friendly reminder that articles are due one week from today for the next edition of GNOME Journal. Please email the list with your articles in plain text, or one zipped file if it includes images.
I didn't hear anything from my reminder last week - so I'm going to assume you're all on track! If not, please drop me or the list an email. I have some flexibility if you need extra time. If you need help or want collaboration as well, please let me know.
Thanks!
Paul
