Re: [gentoo-dev] Re: Handling of tests (was: GCC USE flag changes)

[Ralph Sennhauser]

On Wed, 1 May 2013 19:18:52 -0600
Ryan Hill <dirtyepic@g.o> wrote:

> On Wed, 1 May 2013 10:14:02 +0200
> Ralph Sennhauser <sera@g.o> wrote:
>
> > On Tue, 30 Apr 2013 21:25:40 -0600
> > Ryan Hill <dirtyepic@g.o> wrote:
> >
> > > I'm also going to rename the "test" flag to "regression-test" or
> > > something similar to get it out of FEATURES="test" control.  The
> > > testsuite is a huge time-suck and only useful to developers IMO
> > > (always expected to fail and primarily meant to be used to check
> > > for regressions between patchsets).  I'm a big supporter of
> > > FEATURES="test" by default and I think this is a small step
> > > towards that.
> >
> > This step is so tiny that we wont ever reach the goal like this.
>
> I was hoping it would set a precedent and then people would start
> thinking of splitting up test into categories, maybe even start a
> thread about it ;).

Hehe, yes, you forced me to speak up with trying to set a precedent I
think will get in the way of solving it in a more complete way.

Though for that we have to agree on

 - split is desirable
 - which categories and how to classify tests
 - how to implement the splitting (EAPI support?)

[...]

> > ... and improve on how to configure Portage whether to run tests of
> > any given category.
>
> Yeah I'd love to be able to do something like emerge TESTS="dev qa
> system -extradeps -expensive" @world.
>
>

I was thinking about a /etc/portage/package.test that works more like
package.use. So most users will want to have something like:

  # package.test
  */* cheap

Others might use:

  # package.test
  */* cheap normal
  */*::sunrise expensive

  my-pkg/foo *

  # broken test suite, bug XXX
  =dev-foo/bar-1.1 -*

This would also be pretty similar to what your regression-tests useflag
for gcc would have been. Even though Portage allows you to configure
FEATURES=test on a per package basis since a couple years it doesn't
seem to have become a common practice. While Portages mechanism is
powerful it might be just to complex or tedious for the average user.


As for your example command line 'emerge TESTS="dev qa system
-extradeps -expensive" @world', could you elaborate on what the
categories dev qa system are about?