[gentoo-dev] [RFC] Providing consistent means to enable tests requiring Internet access

[Michał Górny]

Hi, everyone.

Per long-standing unwritten (or rather, written but not really
officially approved) policy unmasked ebuilds are not allowed to fiddle
with Internet. While this normally makes a lot of sense (except for
special cases like live ebuilds), this also means that for some poorly
written packages we end up either disabling a fair number of tests, or
even restricting tests completely.

The Internet-based tests are of course mostly unreliable in the long
run, and should normally be replaced by some kind of mocking, local
servers etc., we simply do not have the manpower to fix all
the packages. All we can do is disable them.

Sadly, for some packages this means that we're left with no tests at
all. As developers, we can play around to run the tests manually, or
comment out needed ebuild bits for extra local testing. However, I think
it would make our work easier if we had a more uniform solutions for
detecting whenever the developer needs to disable networked tests,
and how to enable them.

The obvious solution would be to use a global USE flag with explanatory
description for that. For example:

internet-test - Enable running tests that access the Internet. Those
tests can be unreliable, result in data transfer fees and cause privacy
concerns (potentially exposing which packages are being installed). Use
at your own discretion.


The advantages of that would be:

a. tests requiring Internet are exposed in the standard ebuild metadata,
making it easy to grab it using standard tools,

b. those tests can easily be enabled, and that fact is recorded
in the installed package metadata,

c. the flag can easily be used in RESTRICT="" constraint to easily
disable all the tests.

The disadvantage is that we're introducing yet another special flag that
does not affect installed files.


What do you think? Any other ideas?


-- 
Best regards,
Michał Górny