Re: Behavior Trends in Nautilus and other Desktop Apps

[Jason Fletchall <fletchal cs wisc edu>]

Karsten Bräckelmann wrote:
> I am not aware of your goals, or your methods. I do, however, know 
for a fact that it is not sufficient to look at the UI exposing part
> of the stack to tell the culprit.
>
> Sometimes it isn't even sufficient to compare the presumably crashing
> code at the top of the stacktrace to identify duplicate issues. 
Unlike most apps, Nautilus is of such type...

Actually, talking about stack traces brings up a good point - we do 
collect stack traces, but knowing that they are not always the best 
indicator of program behavior, we also dig much deeper.  For example, we 
instrument branches, function return values, GObject reference-count 
updates, and more.  With the thousands of reports we get from each 
Nautilus release, we are able to associate crash probabilities with 
various program actions.

I and two others are currently working on a tool to visualize this at a 
source-code level.  But since we haven't finished that tool, our 
findings so far show just overall program crashes, i.e. "crashiness" 
rather than "bugginess", which is useful for different reasons.

> Please, don't get me wrong. I definitely do not mean to talk your 
approach down. Neither your findings, nor your engagement. Personally, I 
very much appreciate any effort to get madness like this under control 
and to study the subject. :)

I'm glad we're appreciated!  We like to think that we're on the good side :)

> "In the field". That means the user visible part, counting 
duplicates, instead of treating them as one single "bug"?

Our deeper analyses, like I described above, can tell the difference 
between bugs.  You should check out these experimental results from Ben 
Liblit's work: 
<http://www.cs.wisc.edu/cbi/pldi-2005/exif/sampled/index.xml>.  The 
important part there is that the system examined 2,211 failures and 
attributed them to just 3 bugs.  All three were reported to the libexif 
developers and confirmed as real problems.

> That should result in some better empirical data about the 
"buggy-ness" of an app.

Indeed!

> I believe the GNOME Bugsquad to be the best possible contact for 
investigating any such details. There is quite some development going
> on recently. Also, there are APIs [1] to use bugzilla to gather some
> meta information.

Thanks for the suggestion!

-Jason Fletchall