Re: [gentoo-project] Making Gentoo more fun

[Rich Freeman]

On Sat, Feb 17, 2018 at 4:07 PM, Daniel Robbins <drobbins@××××××.org> wrote:
>
> First, if chromium isn't shipping a stable upstream release, then the
> chromium or "browser-kit" kit can select a version that has undergone some
> testing and is known to be reliable, and make sure this ends up in the
> "prime" kit. The latest dev releases can be added to the "dev" kit of
> 'browser-kit'.
>

I think I need to clarify what I meant regarding QA.  In Gentoo we
have QA standards that are universal, and other standards that govern
keywording in particular.

If we went to an overlay-based system we could either hold all the
overlays to the same standards, or allow them to vary.  If we allow
them to vary then we need ways to categorize the various QA standards
so that people can control what they're getting.

This has nothing to do with what upstream considers stable/beta/etc.
To help reduce confusion I will change "stable" to "supported."

For example, for an overlay to be considered "supported" it might be
required to:
1.  Address security bugs in accordance with policies like: "A Gentoo
package found on 5%+ of user installs that allows remote execution of
arbitrary code must be fixed within 1 day."  (Which is a Gentoo policy
currently [1].)
2.  Not keyword packages as stable for 30 days except as required for
security vulnerabilities.
3.  Not remove the highest-numbered stable-keyworded version of a package.
4.  Never contain errors detectable by repoman.

On the other hand, an overlay considered experimental might not have
to follow any of those rules.

Suppose the chromium team has a bleeding-edge repository where they
often have repoman errors, but it is tagged experimental.  This would
not be a problem.  However, devs who want to use chromium without the
repoman errors might be forced to fork this and create a "supported"
chromium overlay.  This may or may not actually contain stable
keywords.

Now, we can of course choose to simply impose QA standards top-down on
all overlays that are a part of this scheme and avoid some of this.
Maybe core packages would have higher standards than non-core, but
that would be it, and nobody would be allowed to have things like
repoman errors or persistent security vulnerabilities in one of these
new overlays.  However, this does reduce the flexibility (and
complexity) of such a scheme.

I'd also note that we could apply these kinds of QA labels to overlays
informationally.  By this I mean that as far as policy is concerned
everything would be held to a defined standard, however some process
would provide additional descriptive labels to overlays to indicate
whether they exceed or fall short of the standard.  For example, an
automated process might review all security bugs in the last six
months and measure their timings and grade the overlay against our
goals.  While this could be used to prod projects that fall short it
could just as easily be aimed more that users who would be interested
in knowing that a particular overlay delivers a very high level of
quality so that they can use this information to guide their own risk
management.

Finally, as an example of a somewhat cheaper buffer that already
exists in Gentoo I'd consider the stable mirror:
https://github.com/gentoo-mirror/gentoo

The stable branch (which is the default) on this only pulls from the
gentoo-x86 master when it passes all repoman checks.  When somebody
makes a broken commit it simply pauses the stable branch until it is
fixed, and users of this branch would never see the problem.  This
isn't really the same thing, but it is an example of a cheap solution.


1 - https://www.gentoo.org/support/security/vulnerability-treatment-policy.html

-- 
Rich