Re: [gentoo-dev] Improving the stabilisation process - part 1

[Jason Zaman]

On Fri, Nov 25, 2016 at 06:41:20AM +1100, Michael Palimaka wrote:
> As I am sure everyone is aware by now, stabilisation requests on many
> architectures take a long time to be actioned. There are many factors
> contributing to this, but today I'd like to address three specific
> problems that unnecessarily delay developers actioning those requests.:
>
> 1. There's no easy way to pick atoms out of a stabilisation bug.
> Currently, atoms can appear, in a varying formations, in multiple
> locations: in the bug title, spread across multiple comments, in an
> attachment, [...].
>
> 2. There's no way to determine if a stabilisation request is valid (will
> pass repoman) until someone actually tries it.
>
> 3. There's no easy way to identify the level of testing required for any
> given request.
>
> To address this, I am proposing some changes to Bugzilla, some
> automation, and some improved tools.
>
>
> Bugzilla changes
> ================
>
> I'd like to introduce two new fields:
>
> 1. "Runtime testing required" - a drop-down list with three values:
>         * (unset) - the default. The stabilising developer will use
> their best judgement as to what testing is required
>         * Yes - the maintainer is specifically requesting runtime testing
>         * No - the maintainer is happy to stabilise with only a build
> test (for example, trivial revision bumps, libraries with good test
> coverage, or otherwise at the maintainer's discretion)
>
> 2. "Atoms to stabilise" - a textarea containing a list of fully
> qualified atoms. Each atom may optionally be followed by a
> whitespace-delimited list of architectures for which that atom should be
> stabilised. If an atom is not followed by any list of architectures, it
> is assumed it should be stabilised for all architectures CC'd on the bug.
>
> Example atom list from a bug with amd64, arm, and x86 in CC:
>
> =app-foo/bar-1.2.3           # will be stabilised on amd64, arm, and x86
> =app-foo/baz-2.3.4 amd64 x86 # will be stabilised on only amd64 and x86
>
> I'd also like to introduce a flag called "repoman-tested". This will be
> used identify stabilisation requests that have been automatically
> repoman checked by a bot (described below).
>
> To prevent cluttering the Bugzilla interface, I suggest resurrecting the
> "Stabilisation" component and having these new fields appear only for
> bugs filed there and in "Security: Vulnerabilities" (so not to disrupt
> the security team's workflow).
>
>
> Automation
> ==========
>
> It's easy to forget to check that all the required dependencies are in
> stable before filing a stabilisation test, but this wastes the actioning
> developer's time. I have prepared a bot that repoman checks the list of
> atoms and flags the bug appropriately. This allows easy filtering out of
> broken requests.

This part would need to take into account DEPENDS ON in the bug too.
When I file my xfce lists I dep on the gnome stable bug since I need
the gtk stuff and if that isnt taken into account, repoman would just
die.
>
> Improved tools
> ==============
>
> To support the above, I am also preparing some tools to streamline the
> process further:
>
> * A script to, given a bug, produce an architecture-specific list of
> atoms ready to feed into package.accept_keywords / emerge
>
> * An updated version of app-portage/tatt (allowing easy testing of USE
> flag combinations, reverse dependencies, committing keyword changes etc.)
>
> * Your idea here
>
>
> Too long; didn't read
> =====================
>
> We add a new box to Bugzilla to hold atoms on stabilisation request bugs
> so it's easy to extract programmatically.
>
> We add a new box to Bugzilla to indicate if a stabilisation request
> requires runtime testing or not to better focus effort.
>
> If your stabilisation request is invalid a bot will let you know and it
> will be easy to skip your request until you fix it.
>
>
> This will also help pave the way for future enhancements such as
> triggering automated build and QA tests so that the human only has to do
> runtime testing.
>
> Thoughts?

Everything here sounds pretty good to me!

I just realized there is another rare issue that we may have to take
into account. Some sets of packages *must* be stablized in lockstep. For
regular sets of packages like eg xfce if you leave off a bunch of the
extras its no big deal.
Eg for SELinux, the policy packages must all be stabilized all at
once because they depend on each other (I think perl is like this too?).

We would need a way for maintainers to ask for testing without actually
committing. The maintainer can wait till everything is done and commit
everything at once himself. Some flag to make the tatt script skip the
step would be enough I think.

-- Jason