Re: PROPOSAL: Upgrading every RHEL server to RHEL5 on Sun April 1st

[Matthew Galgoci <mgalgoci redhat com>]

> Date: Wed, 14 Mar 2007 22:13:32 +0100
> From: Olav Vitters <olav bkor dhs org>
> To: gnome-sysadmin gnome org, owen bkor dhs org,
>     Matthew Galgoci <mgalgoci redhat com>
> Cc: release-team gnome org
> Subject: PROPOSAL: Upgrading every RHEL server to RHEL5 on Sun April 1st
>
> I propose to upgrade every RHEL server from RHEL3/4 to RHEL5 on Sunday
> April 1 2007. Very likely things will break, that is why I NEED
> additional sysadmins to standby.
>
> The servers involved (http://live.gnome.org/Sysadmin/Servers):
>  * container, our NFS server, serving files for the other RHEL servers
>  * menubar, our mail and DNS server
>  * window, containing loads of websites (www.gnome.org,
>    developer.gnome.org, etc, etc)
>  * button, containing MySQL and mango
>  * label, containing moinmoin (wiki sites like live.gnome.org)
>  * box, containing bugzilla.gnome.org and LDAP (hosts our accounts)
>
> I can handle Bugzilla, DNS, moinmoin, window, mail, MySQL. I want
> sysadmins for the rest (NFS, LDAP).
>
> What might not be down:
>  * socket, containing svn.gnome.org (mail will be though)
>  * progress, containing i18n overview
>  * socket, monitors the websites
>
> Why April 1st:
>  * Sunday, so hopefully less people who are affected by the downtime
>  * 1 week before 2.18.1 release (http://live.gnome.org/TwoPointSeventeen)
>  * long enough after 2.18.0 release
>  * nice date
>
> When announcing, we have to announce it as 2, perhaps 3 day outage. This
> is how much downtime I expect before the main services are back up.
>
> Questions:
> Matthew: Will you be available at that date? Preferably somewhere early
> in the morning.
> release-team: objections?
> sysadmins: available? suggestions/objections?

I will not be onsite, but I should be able to work remotely. I need for you
to dictate to me how the machines should be installed and leave the
installations to me.

As of now, sunday april 1 is clear. I would like to start no earlier than
noon EDT because the NOC staff will not be there before then should we need
NOC intervention.

I would like to see some kind of service migration plan for the outage. I think
we could do rolling updates, where we migrate services from one machine to
another, then reinstall the OS, then migrate services back, and so on.

-- 
Matthew Galgoci
GIS Production Operations
Red Hat, Inc
919.754.3700 x44155