Re: Behavior Trends in Nautilus and other Desktop Apps

[Karsten Bräckelmann <guenther rudersport de>]

> > Any Bugsquad member and occasional bugzilla triager knows that
> > [Nautilus 2.14 was buggier than 2.16 or 2.18]. ;)
> 
> Ah, OK.  I guess a 10% crash rate is hard to miss when you spend lots of 
> your time triaging bug reports.  :-)

Oh, I didn't expect you guys to actually have subscribed to the list,
judging from the first post. I did, however, hope it. :)


Anyway, I did NOT say what you quote out of context above.

I did agree that Nautilus crashed rather often a couple of versions ago.
However, that does NOT translate into being (more) buggy. Actually it
turned out, that a lot of the crashes (read: distinct stacktraces)
gathering an immense log of duplicates indeed have been triggered by the
very same cause. That makes *one* bug, and a shitload of crashes.
Granted, the crash is what is visible to the user, and he won't even
distinguish between different and identical crashes -- but that still is
one bug.

Code paths in Nautilus aren't as unique as one might prefer. Which makes
spotting duplicates (and the actual issue) harder. Also, there are a lot
of different low-lever libraries involved, not part of Nautilus. A crash
in gnome-vfs, pango or even a distro-specific GTK+ theme (yes, talking
from experience) will look just like Nautilus crashed yet again...

I am not aware of your goals, or your methods. I do, however, know for a
fact that it is not sufficient to look at the UI exposing part of the
stack to tell the culprit.

Sometimes it isn't even sufficient to compare the presumably crashing
code at the top of the stacktrace to identify duplicate issues. Unlike
most apps, Nautilus is of such type...


Please, don't get me wrong. I definitely do not mean to talk your
approach down. Neither your findings, nor your engagement. Personally, I
very much appreciate any effort to get madness like this under control
and to study the subject. :)

All I am trying to do here is to report from the perspective of a brave
and wounded soldier, who has been fighting front-line for a long
time. ;)  Please note, that I am not a Nautilus developer.


> Jason Fletchall and I didn't know that Nautilus 2.14 had a bad 
> reputation when we were producing this data, though it's pretty 
> unmistakable when you look at the plots.  I'm glad to hear that our 
> empirical data agrees with your informal sense of things.  That's one 
> thing I hope we can contribute: empirical evidence to support/refute 
> developers' intuitions about how apps are behaving in the field.
> 
> We welcome suggestions if you know of any other historical trends we 
> should look at.  We've actually got about four years' worth of data, 
> though most of our current plots don't go back that far.  And we can 
> look into much more than just crash rates, since CBI instruments *lots* 
> of different runtime code behaviors.

"In the field". That means the user visible part, counting duplicates,
instead of treating them as one single "bug"?


Another aspect -- at least when bugzilla is concerned -- are distro
releases. (Pre-)releases and a (rapidly) broadening user basis did
account for some major spikes (btw, particularly in this case, too). The
number of bugs was the same, reports however sky-rocketed to a point,
where we *had* to suppress known issues from even be filed into
bugzilla.

Granted, since you got your own instruments and likely know the exact
number of users (and maybe even usage time), this is nothing you need to
take into account -- unless, of course, you are going to use bugzilla as
a cross-reference / meta-data sorce (see below).


> So ... what should we look at next?

Keeping in mind that I still don't know about details of your project:

Investigating the actual crashes seems like a good idea. This at least
will help spot duplicate crashes, that happen to take down the app
often. Which IMHO is one bug. As apposed to other apps that crash in
distinct situations all over the place.

That should result in some better empirical data about the "buggy-ness"
of an app.


I believe the GNOME Bugsquad to be the best possible contact for
investigating any such details. There is quite some development going on
recently. Also, there are APIs [1] to use bugzilla to gather some meta
information.

HTH

  guenther


[1] Do not even attempt to harvest the web frontend.

-- 
char *t="\10pse\0r\0dtu\0  ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}