1 |
On Wed, 19 Jun 2013 12:45:05 +0200 |
2 |
Thomas Kahle <tomka@g.o> wrote: |
3 |
|
4 |
> On 06/19/2013 03:02 AM, Aaron W. Swenson wrote: |
5 |
> > On 2013-06-19 00:01, Jeroen Roovers wrote: |
6 |
> >> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - |
7 |
> >> - - - David Garcia, Marcelo Serrano Zanetti and Frank Schweitzer |
8 |
> >> Chair of Systems Design – www.sg.ethz.ch – ETH Zurich |
9 |
> >> |
10 |
> >> [...] |
11 |
> >> |
12 |
> >> http://arxiv.org/abs/1306.3612 |
13 |
> >> |
14 |
> >> |
15 |
> >> Merry reading, |
16 |
> >> jer |
17 |
> >> |
18 |
> > |
19 |
> > So, there ya go. Start taking happy pills and be more positive and |
20 |
> > motivating! |
21 |
> > |
22 |
> |
23 |
> I've not read their new paper but the first one was just ridiculous. |
24 |
> They drew an edge from A to B if A reassigned a bug to B and |
25 |
> intepreted this as "A knows that B is an expert on the subject". |
26 |
> That is just ridiculous since A is usually a bug wrangler and B is to |
27 |
> be found in metadata. For instance the emacs team which gets very |
28 |
> few bugs almost never appears as B (and certainly not A). This |
29 |
> introduces all sorts of bias. I think it would be fair to say that |
30 |
> their first study had very low predictive power and the effects they |
31 |
> saw were created by their method. |
32 |
|
33 |
Good to (finally) see some critical review on the "laymen's" side of |
34 |
the subject matter. I didn't vent any opinion on the last paper since, |
35 |
you know, they're doing science in their niche and I'm not going to |
36 |
tell them how to do it - social research has its fundamental problems |
37 |
and I have my own to deal with. |
38 |
|
39 |
In reading the second paper on the alleged decline of Gentoo's bug |
40 |
tracker performance, I went back to the first one, and I don't really |
41 |
see how performance degraded and never returned to form after that |
42 |
<sarcasm>cataclysmic event</sarcasm> in ~2008. |
43 |
|
44 |
I think it's fair to say things are done very differently now and that |
45 |
lots of bug reports now don't get CLOSED/INVALID, REOPENED, wrongly |
46 |
assigned, reassigned, re-reported, redefined, entangled with unrelated |
47 |
but similar bugs, and DUPLICATEd so much any more, so I've asked the |
48 |
researchers some questions[1] on this issue of measuring performance[2]. |
49 |
|
50 |
I should also point out that doing social science properly is rather |
51 |
tricky, and that not getting entangled in the historiography of the |
52 |
subject matter is important. Lots of things went on in 2008 - good and |
53 |
bad things, and the community was under a lot of stress. And while I am |
54 |
at it: do note that nobody was interviewed this time around. The second |
55 |
paper regards Gentoo's poor bug tracker performance after Alice's |
56 |
retirement as a well-established fact found in the first paper. |
57 |
|
58 |
|
59 |
Regards, |
60 |
jer |
61 |
|
62 |
|
63 |
[1] And within hours I have three replies from the same person in my |
64 |
inbox! I'll probably follow up on this. |
65 |
[2] Frankly it stings a little to read about my vain efforts to improve |
66 |
bug wrangling, but conversely I might simply blame the research |
67 |
metrics. :) |