Re: [gentoo-dev] RFC: Removing src_test from www-client/chromium

[Tom Wijsman]

On Sun, 01 Jun 2014 15:41:35 +0200
""Paweł Hajdan, Jr."" <phajdan.jr@g.o> wrote:

> On 5/31/14, 8:30 PM, Tom Wijsman wrote:
> > On Sat, 31 May 2014 19:50:20 +0200
> > ""Paweł Hajdan, Jr."" <phajdan.jr@g.o> wrote:
> >> This is one of my points: I don't remember a single chromium bug
> >> filed in Gentoo that would be caught by a test or that a failing
> >> test actually detected.
> >
> > Your point covers the lack of tests, or tests that are non-fatal;
> > however, it doesn't cover tests that are fatal, what if they fail?
>
> I'm confused by the distinction of fatal and non-fatal tests. Neither
> upstream nor the Gentoo chromium package makes that distinction.

Tests that break parts of your browser; you don't notice such tests in
what was said, until one day such a test does break one or another way.

> >> By the way, I don't remember seeing many reports about font issues
> >> or tab crashes. Please make sure to file them when they occur, or
> >> just point me to them in case I somehow missed them.
> >
> > They usually go straight to upstream, though I've managed to somehow
> > fix it up; as for Gentoo, some people create forum threads about
> > them.
>
> I can't speak for other people, but please consider reporting issues
> to Gentoo first. Our bug queue is under 30 bugs, while upstream is
> several thousand. Once we can confirm a bug clearly belongs to
> upstream, we can tell the reporter to file bug upstream or do that
> ourselves, but keeping Gentoo out of the loop seems to increase the
> time needed to fix a bug.

This confuses me; your thread opener seems to suggest you have too much
bugs, whereas this one seems to suggest you don't have enough bugs.

What's the purpose of disabling src_test if bug count isn't a problem?
Iotw, why are you making a project-internal decision here?

(Your last two paragraphs may respond to this; in which case, nvm)

> > (One was due to a library compiled with a less common flag, the
> > other due to fontconfig being a regression magnet; both fun to
> > debug, the former a test wolud've caught, the latter is due to the
> > lack thereof)
>
> If there's something that could be changed e.g. in chromium's
> dependencies, please let me know. There are cases where we require
> certain USE flags to be set on dependencies for things to work
> properly.

Wasn't the case of a different version or USE flag state; but that is
the case one or another day, I'll let you know.

> About the issue that a test would have caught: was that a chromium
> test? If so, which one?

Unable to tell, since I don't run them.

> >>> While I don't run tests myself; the need for them is clear, for
> >>> those that aim for more production ready systems (eg. university
> >>> network PCs).
> >>
> >> This seems too theoretical to me. I'd be fine with someone
> >> volunteering to maintain chromium's src_test in Gentoo. Unless we
> >> have such a person though, it seems to mostly take valuable focus
> >> away from bugs that definitely *do* affect our users, for no
> >> provable benefit for Gentoo.
> >
> > What about provable benefit for upstream? Does upstream /dev/null
> > them?
>
> Effectively yes. For an example see
> https://bugs.gentoo.org/show_bug.cgi?id=497512 and
> https://groups.google.com/a/chromium.org/d/msg/chromium-dev/OdX7ShsOqsM/-R9sexJAEa4J
>
> The failure is not Gentoo-specific, and is not a bug in code but
> problem with the test (it makes assumptions about internal glibc
> implementation). It actually fails on the latest Ubuntu LTS Trusty
> Tahr, which means the test will have to be fixed or disabled
> upstream. But 6 months of no reaction is not really a good sign IMHO.
>
> Paweł

Yeah; if failing tests on distributions aren't getting fixed by
upstream, then there's indeed no point to keep them running.

Though; on the other hand, one has to consider that this acts like a
priority queue and therefore tests that fail on most distributions would
get fixed before tests that fail on just one or two distributions.

It's a tricky decision to drop them; but it's not an irreversible
decision, thus a reevaluation in 5 years from now could be possible. If
that reevaluation then shows a responsive upstream, reconsider src_test.

Don't mind me, I've played devil's advocate to explore the reasoning;
go ahead if you want to, given it barely result in fatal test failures.

-- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : TomWij@g.o
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D