Gentoo Archives: gentoo-dev

From: Ryan Hill <dirtyepic@g.o>
To: gentoo-dev@l.g.o
Subject: [gentoo-dev] The importance of test suites
Date: Sun, 21 Feb 2010 05:23:47
Message-Id: 20100220220832.343f7084@gentoo.org
1 (this isn't directed at any one person or group or any recent incident, this
2 has been bugging me for years)
3
4 I have one simple request. When you make a non-trivial change to an ebuild -
5 a patch, a version bump, anything that can effect the behaviour of the
6 package - please run the test suite. If it fails, fix it. Or restrict it.
7 Or even make it non-fatal if there's no other choice. If you can reproduce
8 failures outside of portage, report them upstream. Failures indicate either
9 a broken package or a broken test suite and either way it's in your best
10 interests to get them fixed.
11
12 Remember that for anyone running FEATURES=test a failure breaks the build*.
13 You wouldn't commit something that doesn't compile (hopefully :P), so why
14 is this any different? There is no point in even having test suites if
15 everyone just disables them in frustration because every third package fails.
16
17 I apologize for the rant, but when I do testing for gcc-porting I rely
18 heavily on tests to catch runtime issues. And every release cycle I end up
19 spending way too much time trying to figure out why a test is failing, only
20 to find that there's been an bug open about it for two years with no
21 activity.
22
23
24 * I know about test-fail-continue, but I've found that it just causes me
25 file fewer bug reports because they don't annoy me as much. ;)
26
27 --
28 fonts, by design, by neglect
29 gcc-porting, for a fact or just for effect
30 wxwidgets @ gentoo EFFD 380E 047A 4B51 D2BD C64F 8AA8 8346 F9A4 0662

Replies

Subject Author
Re: [gentoo-dev] The importance of test suites "Paweł Hajdan
Re: [gentoo-dev] The importance of test suites Tobias Klausmann <klausman@g.o>