Re: call for collaborators: bug isolation via remote program sampling

[textshell neutronstar dyndns org]

On Fri, Apr 25, 2003 at 02:58:51PM -0700, Ben Liblit wrote:
> Havoc Pennington wrote:
> >The main challenge I see is that GNOME distributes betas as source
> >only.
> 
> Good point, though one does commonly see beta binaries for various 
> GNOME-based applications.  Perhaps I should be targeting these end user 
> applications rather than the core desktop per se.
> 
> >So I believe we'd need some way to let people build their own
> >instrumented code.
> 
> That could certainly be done, with a little work on my end to get our 
> instrumentation infrastructure packaged up nicely.  There's still the 
> question of how to get people to use it, though.  If each person builds 
> their own, I've got quite a marketing challenge to convince each of 
> those people to download and use our instrumentor.  With precompiled 
> binaries, I only need to convince one person to use our system (plus an 
> explicit opt-in dialog for end users).

Many users the betas from source, but not manually. Most use one of the
autobuilders (Garnome, jhbuild, vicious-build-scripts) so they would be a
reasonable 'easy' target. This way user have the choice to enable it in the
compile or not. They could even recompile a single package with instrumentation
to help debugging a problem.

> 
> >We do have the automated bug report tool bug-buddy that already
> >uploads backtraces, so perhaps that could be extended to also upload
> >the instrumentation data?
> 
> Our analysis works by identifying behavioral differences between 
> successful and failed runs.  Right now, that means it needs both crash 
> reports as well as non-crash reports.  Eventually we may get to the 
> point where we can use crash reports only, but for now, we need both. 
> Bug-buddy would only cover the crashing half of the picture.
> 
> (This is part of the reason I intend the opt-in to be *very* explicit. 
> People are used to reporting crashes.  Reporting non-crashes is unusual, 
> so I want to make sure that each user clearly knows what's going on.)
> 

The problem is that nobody knows what this reports reveal (Do you even know the
*practiacal* implications?). So most likely we have to assume to reveals any
action done by the user and any file read by the programm. Is that correct?

If this is true, it might be nice if the user could run part of the data
processing local. But that might not be to helpful to  you.

Martin H.