1 |
21.08.2013 17:38, Wyatt Epp пишет: |
2 |
> Fundamentally, I see this as a problem of tooling. |
3 |
|
4 |
|
5 |
I think that no tool can cover all cases of checking that software |
6 |
WORKS. I mean - in generic, for all kinds of software. You can guarantee |
7 |
if it builds, if it follow some QA rules about CFLAGS/LDFLAGS/whatever, |
8 |
but there is only one 100% way to guarantee that software works - run |
9 |
and do something useful :-). |
10 |
|
11 |
Yeah, i know about testsuites, that can help in that case. Unfortunately: |
12 |
|
13 |
1) they do not cover all cases, but that's not so important; |
14 |
2) often they are badly broken; |
15 |
3) not every program carries test suite. |
16 |
|
17 |
So, we stuck at automation in run-time checks right at the beginning :-). |
18 |
|
19 |
But yeah, we could automate build-time checks, surely. |
20 |
|
21 |
|
22 |
> |
23 |
> I'd like to point out something that jumped out to me as a red flag |
24 |
> earlier (not to pick on you specifically, Tom; this is just the |
25 |
> cleanest example I saw), and turn it into an example: |
26 |
> |
27 |
> On Wed, Aug 21, 2013 at 4:51 AM, Tom Wijsman <TomWij@g.o> wrote: |
28 |
>> |
29 |
>> Well, they are listed there; but it's quite some work to actually go |
30 |
>> through that list, that is, manually check the bugs of ~2000 packages |
31 |
>> as well as file a STABLEREQ bug, takes quite a while... |
32 |
>> |
33 |
> This right here is a real problem. Any time you're talking about |
34 |
> doing anything on this scale "manually", you've already lost the |
35 |
> battle. You need a tool to minimise the overhead of time and |
36 |
> cognitive load. What would that tool look like? Think about the |
37 |
> steps involved and how you can reduce them to only the parts that |
38 |
> absolutely require decisions on your part. |
39 |
> |
40 |
>>> At least in the areas I usually work, I have found a combination of |
41 |
>>> the automatic stabilisation requests and imlate have definitely cut |
42 |
>>> back on the bitrot. |
43 |
>> |
44 |
>> A single unimportant bug can prevent the automatic STABLEREQ bug from |
45 |
>> getting filed; as for imlate, not everyone seems to know that tool, not |
46 |
>> everyone seems to run it. Attention for some stabilizations is lost... |
47 |
>> |
48 |
> First off, why do developers not know about the tools? How can this |
49 |
> be addressed? For a start, I'd suggest making sure the tools are at |
50 |
> least mentioned in the docs. Somewhere other than the amd64 Arch |
51 |
> Testing guide from 2006. [1] That's the only concrete (i.e. actually |
52 |
> in the DOCS rather than the ML archives) documentation I've found so |
53 |
> far, and it only references the file, rather than the tool. Maybe in |
54 |
> the devmanual Tools Reference? [2] |
55 |
> |
56 |
|
57 |
Good point, i agree. |
58 |
|
59 |
> But, imlate is a good example of a tool that could ease the time cost |
60 |
> of grindy crap. You showed before that it can get an ordinary count |
61 |
> bounded on n days. That's handy, but only a little. Build out: |
62 |
> - How many of those stablereq bugs reference versions no longer in the |
63 |
> tree? Those can probably get closed. |
64 |
> - How many have newer STABLE versions in the tree in the same slot? |
65 |
> Probably fine to close those, too. |
66 |
> - Of the remaining, how many have patches or ebuilds attached? Those |
67 |
> may be solved problems waiting for closure; shortlist them. |
68 |
> - How many are packages with newer versions that have been in the tree |
69 |
> for >30 days? Consider changing the target version, then. |
70 |
> - How many have open blockers, and what are those blockers (w/link and |
71 |
> summary)? Scan for low-hanging fruit jumping out in that list. |
72 |
> - Get views by category; are there categories where updates are more |
73 |
> important? Things in @system, and things with security concerns |
74 |
> (stuff in net-*) should probably get higher priority; games... |
75 |
> probably less so. |
76 |
> - Are there bugs with certain keywords in the body that should raise |
77 |
> priority? Things like "security" or "overflow" might be good |
78 |
> candidates. |
79 |
> - Are there bugs with certain keywords in the body that indicate it'll |
80 |
> be really easy to decide? e.g. "trivial" or "minor" might turn up some |
81 |
> of those super-small version bumps that you pretty much know aren't |
82 |
> going to affect stability. |
83 |
> |
84 |
> These are just examples off the top of my head, and by no means |
85 |
> bulletproof, but these are in the class of improvements that have ROI |
86 |
> because they reduce a task that previously took developer time to one |
87 |
> that takes CPU time. CPU time is essentially free compared to the |
88 |
> value of dev time. |
89 |
|
90 |
That's what i called constuctive criticism. Congratulations :-) |
91 |
|
92 |
> And I'm not saying more recruiting shouldn't happen, but relying on it |
93 |
> is no better than hoping at $deity for bugs to close themselves. ;) |
94 |
> |
95 |
> Cheers, |
96 |
> Wyatt |
97 |
> |
98 |
> [0] Okay, maybe in the "glory days" when we were higher up on |
99 |
> Distrowatch and thing were really kicking. (I know, I know, "DW isn't |
100 |
> representative", but really? Sabayon is doing better than we are, |
101 |
> now?) |
102 |
> [1] http://www.gentoo.org/proj/en/base/amd64/tests/index.xml?part=1&chap=2 |
103 |
> [2] http://devmanual.gentoo.org/tools-reference/index.html |
104 |
> |
105 |
|
106 |
|
107 |
-- |
108 |
Best regards, Sergey Popov |
109 |
Gentoo developer |
110 |
Gentoo Desktop-effects project lead |
111 |
Gentoo Qt project lead |
112 |
Gentoo Proxy maintainers project lead |