Re: [gentoo-dev] Re: Handling of tests

[Rick \\\"Zero_Chaos\\\" Farina]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 05/01/2013 09:18 PM, Ryan Hill wrote:
> On Wed, 1 May 2013 10:14:02 +0200
> Ralph Sennhauser <sera@g.o> wrote:
>
>> On Tue, 30 Apr 2013 21:25:40 -0600
>> Ryan Hill <dirtyepic@g.o> wrote:
>>
>>> I'm also going to rename the "test" flag to "regression-test" or
>>> something similar to get it out of FEATURES="test" control.  The
>>> testsuite is a huge time-suck and only useful to developers IMO
>>> (always expected to fail and primarily meant to be used to check for
>>> regressions between patchsets).  I'm a big supporter of
>>> FEATURES="test" by default and I think this is a small step towards
>>> that.
>>
>> This step is so tiny that we wont ever reach the goal like this.
>
> I was hoping it would set a precedent and then people would start thinking of
> splitting up test into categories, maybe even start a thread about it ;).
>
>> Let's
>> start to properly classify test into categories, like for instance
>>
>> - expected to be run (cheap, no silly deps)
>> - good thing if run (still reasonable wrt resources) (current src_test)
>> - if you are the maintainer or simply curious. (boost, jtreg and
>>   friends)
>
> Something like "dev-test" or "qa-test"?  I can think of a couple packages..
>
>> ... and improve on how to configure Portage whether to run tests of any
>> given category.
>
> Yeah I'd love to be able to do something like emerge TESTS="dev qa
> system -extradeps -expensive" @world.

Honestly, IMHO, I think breaking it down to more than "test" and "qa" is
excessive.  I certainly wouldn't block anyone that wishes to do that
work, but I think we all realize that enabling tests (and especially
some of the really awesome QA tests Diego does in the tinderbox) are
really expensive time wise.  The average user should never have this
enabled, and honestly, most devs shouldn't have it enabled either as
they simply won't have the hardware to run the tests every time (I know
I've commited things from my chromebook without excessive testing...).

I have a little cron script that builds all the packages I'm marked
maintainer of every night with FEATURES=test and would be easy enough to
adapt to looking at herd instead of maintainer.

I don't claim to be the expert on testing, but in case anyone finds
something like this useful... here:

#!/bin/sh

emerge --sync -q

#maintainer check (excluding madwifi)
MAINTAINER_OF="$(fgrep 'zerochaos@g.o'
/usr/portage/*/*/metadata.xml | cut -d/ -f4-5 | grep -v madwifi)"

#build test
FEATURES="test" emerge --deep --oneshot -q ${MAINTAINER_OF}
if [ $? -ne 0 ]; then
        echo "build failed, see above"
fi

#repoman check
for atom in ${MAINTAINER_OF}; do
        cd /usr/portage/${atom}
        repoman -qd full > /dev/null
        if [ $? -ne 0 ]; then
                repoman -qd full
        fi
done

Thanks,
Zero


PS> I welcome suggestions for this script, new functionality is always
welcome

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJRgfRKAAoJEKXdFCfdEflKJPYP/2qUMB5T3KZ1jT2dusdgg3KA
yuPAc6xL9VNg25pnVYl14JoS66z1hKpisSCCZeeGT7uqRDEkh+jseEaZIOhrektb
Zgn3WXmekmU5pc9wicUW4aYVzNyc9PBbw2NcBWq1HwgfMcwBoX87FxVECw0L71dO
hNOWQbJ1amGGoLXiMmAI/S4CpkZcfX3WZv0R2oosKK/Iu34gZeHGk2w5ui4tuY6g
sZmKawjXTFzbluIcaZlUPyajBjyZEXx0Vsp3uYUY1lStUNG45q5jaz3/83V+NkF2
Lu0FHrXrSW97f19k2HkcTdl4icF48k6bmIbw22xZ1lII+rnV/3uqsok/UayRyF1c
w6EIFnS37M/MhDknJ9R65P18v/SWAP2MLnfhcIqFDWs6jXZqq8vpBOfW+waEWsKK
8qflxpU08joUBsUSBcz+Y8s6JBxbWVdlY+f+jLsP6kPUH5otHKym7vB2P0nqCGQQ
SayT8fpK5izX4UraNhpuOiDV8nH7cvD1h3t6MRlSzRshj7UWyw4nYIkhqFZJhFzW
g6hfmLUGqHeHxRKMyImVLY4+PST820s+5mTTG57YuXmzD/nYU9N2Yor61FtgLR26
975mApTT4cunHdiUudyGbeaE0j0f/wOe1CAivzTCmDi+tM4dYR78XscPRZCsrk2+
JeNw6XtA5A6jvVk12jYN
=+Zzv
-----END PGP SIGNATURE-----