bxc -> bgo: Rough first pass at a brain dump

[David Hoover <karma deadmoose com>]

It's nowhere near comprehensive yet, but I managed to get some time
today to mull around some of the details of what all needs to happen for
the bxc -> bgo migration. Attached is a copy of the file so far, so if
people (who all's on this list, anyways? Me, Andrew, Gerardo, &
bugmaster?)

A couple big points (which aren't necessarily in that document) that
I'll bring up explicitly, though:
     1. This document (and the eventual scripts that will be written
        based on it) should go someplace. If I'm the only person who'll
        be doing much active hacking on it, I can just keep it in my own
        local cvs repository, but if others will be doing much more than
        advising over email, it should probably go into gnome cvs, no?
        Also, it'd potentially be handy to keep around someplace real
        for the future when something like this is needed again (see
        also: lost knowledge of the eazel migration). Anyone's thoughts
        on that? If it goes into gnome cvs, someone should come up with
        a decent name, and I'll need a gnome cvs account; I've done
        patches here & there, but I don't actually have cvs access.
     2. On a similar vein, is it worth making a bugzilla product for
        this? Again, if I'm more or less the only person hacking on it,
        I'm okay managing issues/todo items/etc just in a file. Actual
        bugs is probably overkill, but I just want to bring it up now
        while it's convenient to do whatever.
     3. How about the direct mysql access to the two bugzillas? bgo's
        probably the easier of the two to manage that for, but it's
        unfortunately the less interesting one to me. Andrew; are you
        looking into this at all, or should I? If the latter, could you
        give me some names to start harassing?
     4. My very first item in the file is fairly important; bgo is
        running bugzilla 2.16.5, but can anyone tell me what bxc is?
        *crosses fingers for something without drastic table changes*

Those are sort of the big 4 I wanted to bring up first. I'll be mucking
around more with my little TODO file here in the meantime, since there's
still LOTS to write down, but if people could skim over this iteration
when they get a chance & send me feedback, it'd be much appreciated.

-- 
David Hoover <karma deadmoose com>

Information Gathering:
 * Find versions of bugzilla (bgo = 2.16.5, bxc = ?) - If bxc is 2.17, there's
   a lot more black magic involved. Here's hoping for very similar versions...
 * Find list of products to migrate (Bonobo, Connector, Evolution, EDS, GAL, 
   GdkPixbuf, Soup, and XST ?)
 * See below (in Random Thoughts) re: freezing the bugzillas when we do this
   for real
 * When this gets done for real, will we have direct mysql access to both
   bugzillas? If not, we'll have to structure things drastically differently
   in order to do all the reading from bxc on one host, then shovel the data
   over to somewhere else where we can munge it and talk to bgo. If that's the
   case, what we probably should do is just get a dump of bxc and fire it up
   elsewhere for the migration
 * Where will this be running for real? What kind of constraints will we have
   on CPU usage, memory usage, disk usage, and wall time? (Ideally, I'd like
   to make this as logically simple as possible, quite possibly involving 
   lots of large intermediate files, since spending time optimizing something
   that will in essence only get run once seems a waste, and likely to 
   introduce bugs)
 * What is the purpose of bugs.bug_file_loc? Is it actually in use? Do we need
   to migrate things pointed to with those?
 * Query all the duplicates and see if we have any duplicates referencing 
   bugs in products that we're not planning to migrate. Hopefully we won't,
   but if we do, we need to come up with a plan (possibly just involving manual
   appends to the relevent bugs post-migration)

===============================================================================
Random Thoughts:

 * How are we going to generate new numbers for things like bugs.bug_id,
   components.id, etc? They're autoincrementers so I suppose the right thing
   would be to just blindly insert our new things and let it automagically 
   assign us a new number. Last I checked, that was easier said than done
   (you essentially had to insert, then query explicitly looking for what you
   just inserted), so my gut feeling is that I'd rather just freeze bgo, find 
   the highest number from each autoincrement field, and manually create my own
   numbers. Depending how long this whole shebang takes to run, though,
   freezing bgo might not be feasible...
 * Speaking of freezes, how's that going to work? Presumably we'll need to 
   freeze bxc, since having people adding new bugs while we're doing this would
   be ugly (though if we did it right, we wouldn't break, and we'd just have to
   look to see if there are any bugs/attachments/etc in the appropriate 
   products that don't exist in our translation tables once we're done... That
   doesn't address people updating info on existing bugs, though, so we probably
   WILL need to just freeze it.
 * There's a clean way to lock products so new bugs can't get filed, right?
   (I'm thinking about how bxc should look once we're done, since we want the
   old stuff to still be there, with status MOVED, but we don't want to have to
   keep telling people "hey, thanks for filing that bug, um, let's refile it
   over here now")
 * Is it worth trying to do magic when migrating user profiles over? This is
   in regards to user groups, etc. Probably, in general, it's easiest to just
   migrate people and then later by hand deal with any special prividges
 * When migrating users, are we going to run into a single person using diff
   email addresses on bxc & bgo? (i.e. foo ximian com on one and foo gnome org
   on the other, or some such) Probably need to just run comparisons and see
   how many bxc email addresses exist that don't exist on bgo. If that number 
   is large enough, we may need to look into human intervention

===============================================================================
Flow:

 * Get all relevent entries from profiles table and either create accounts at
   bgo, or record pre-existing profiles.userid's
 = We now have a mapping of profiles.userid's to the ones on bgo
 NOTE: This could be done well in advance of the actual migration

 * Get products from bxc, make matching ones in bgo (watch out for duplicates
   like bonobo) - Should anything get renamed? (i.e. what's 'connector' 
   really called, and what SHOULD it be called?)
 = We now have a mapping of products.product to new ones. This may be
   excessive, since we potentially aren't changing any names, but it's 
   probably easy enough to implement since we'll be keeping a zillion lookup
   tables anyway. Also, it might need to be more magic if bxc is recent 
   enough to have products.name not products.product

 * Get components from bxc, make matching ones in bgo (once again, we might have
   weirdness with duplicates like bonobo)
 = We now have a mapping of components.id to new ones

 * Whole bunch more......