[gentoo-dev] Re: FEATURES=test, sys-devel/gcc ignored test failures

[Ryan Hill]

On Tue, 22 Mar 2011 13:09:46 +0100
"Paweł Hajdan, Jr." <phajdan.jr@g.o> wrote:

> On 3/21/11 11:02 PM, Ryan Hill wrote:
> > It does to me, I use them all the time. ;)  The important part is that we
> > install the test results, which can then be used for regression testing when
> > rolling patchsets.
>
> I see, it makes sense. I guess you're comparing the test results before
> and after rolling patchsets and look for regressions.
>
> > I think that glibc and gcc tests and other testsuites that nearly always
> > fail shouldn't be run for the average user but should still be easily
> > accessible in a standard way.  I think we need a more finely grained test
> > setup, where we can say tests are "expensive" or "interesting only to
> > developers" or "known to fail", and let people opt-in to these on a
> > per-package basis. Right now you always have to opt-out using
> > package.use.mask which "works" but is unintuitive.
>
> My main point is that the developer profile has FEATURES=test, and also
> arch testers and developers run with FEATURES=test. Being able to
> quickly rebuild gcc, glibc and others is a win.

Yes, I'm agreeing with you.  I'd like these off by default too.  We need a
standard way of enabling them however.  USE="test-dev" or something.  I
complain about this about once a year or so. ;)  Maybe I should just do it.

In the meantime:

echo -e 'sys-libs/glibc test\nsys-devel/gcc test' \
  >> /etc/portage/profile/package.use.mask
 
> I'm trying to understand the problem better - do you know what causes
> those test failures? I don't expect a "complete" answer because that'd
> probably be a half of actually fixing the failures.

The GCC testsuite isn't designed to pass.  It's designed to be a
regression test.  Check before and after you apply a patch, make sure you
don't cock things up.  From http://gcc.gnu.org/install/test.html: "It is
normal for some tests to report unexpected failures. At the current time the
testing harness does not allow fine grained control over whether or not a test
is expected to fail."  Look at http://gcc.gnu.org/buildstat.html and you'll
see this in practice.

One thing I know of that causes a bunch of failures is the fact that we
enable -Wformat, -Wformat-security, and -Wtrampolines by default.  Any
additional output during a test = fail.  I patched these recently for 4.5
though so they shouldn't be a problem going forward.


-- 
fonts, gcc-porting,                  it makes no sense how it makes no sense
toolchain, wxwidgets                           but i'll take it free anytime
@ gentoo.org                EFFD 380E 047A 4B51 D2BD C64F 8AA8 8346 F9A4 0662