Re: call for collaborators: bug isolation via remote program sampling

[Ben Liblit <liblit cs berkeley edu>]

Havoc Pennington wrote:
> The main challenge I see is that GNOME distributes betas as source
> only.

Good point, though one does commonly see beta binaries for various 
GNOME-based applications.  Perhaps I should be targeting these end user 
applications rather than the core desktop per se.

> So I believe we'd need some way to let people build their own
> instrumented code.

That could certainly be done, with a little work on my end to get our 
instrumentation infrastructure packaged up nicely.  There's still the 
question of how to get people to use it, though.  If each person builds 
their own, I've got quite a marketing challenge to convince each of 
those people to download and use our instrumentor.  With precompiled 
binaries, I only need to convince one person to use our system (plus an 
explicit opt-in dialog for end users).

> We do have the automated bug report tool bug-buddy that already
> uploads backtraces, so perhaps that could be extended to also upload
> the instrumentation data?

Our analysis works by identifying behavioral differences between 
successful and failed runs.  Right now, that means it needs both crash 
reports as well as non-crash reports.  Eventually we may get to the 
point where we can use crash reports only, but for now, we need both. 
Bug-buddy would only cover the crashing half of the picture.

(This is part of the reason I intend the opt-in to be *very* explicit. 
People are used to reporting crashes.  Reporting non-crashes is unusual, 
so I want to make sure that each user clearly knows what's going on.)

Thanks for the feedback, Havoc!  This is all good food for thought.