Re: [gentoo-dev] Improving the stabilisation process - part 1

[M. J. Everitt]

On 24/11/16 19:41, Michael Palimaka wrote:
> As I am sure everyone is aware by now, stabilisation requests on many
> architectures take a long time to be actioned. There are many factors
> contributing to this, but today I'd like to address three specific
> problems that unnecessarily delay developers actioning those requests.:
>
> 1. There's no easy way to pick atoms out of a stabilisation bug.
> Currently, atoms can appear, in a varying formations, in multiple
> locations: in the bug title, spread across multiple comments, in an
> attachment, [...].
>
> 2. There's no way to determine if a stabilisation request is valid (will
> pass repoman) until someone actually tries it.
>
> 3. There's no easy way to identify the level of testing required for any
> given request.
>
> To address this, I am proposing some changes to Bugzilla, some
> automation, and some improved tools.
>
>
> Bugzilla changes
> ================
>
> I'd like to introduce two new fields:
>
> 1. "Runtime testing required" - a drop-down list with three values:
>         * (unset) - the default. The stabilising developer will use
> their best judgement as to what testing is required
>         * Yes - the maintainer is specifically requesting runtime testing
>         * No - the maintainer is happy to stabilise with only a build
> test (for example, trivial revision bumps, libraries with good test
> coverage, or otherwise at the maintainer's discretion)
>
> 2. "Atoms to stabilise" - a textarea containing a list of fully
> qualified atoms. Each atom may optionally be followed by a
> whitespace-delimited list of architectures for which that atom should be
> stabilised. If an atom is not followed by any list of architectures, it
> is assumed it should be stabilised for all architectures CC'd on the bug.
>
> Example atom list from a bug with amd64, arm, and x86 in CC:
>
> =app-foo/bar-1.2.3           # will be stabilised on amd64, arm, and x86
> =app-foo/baz-2.3.4 amd64 x86 # will be stabilised on only amd64 and x86
>
> I'd also like to introduce a flag called "repoman-tested". This will be
> used identify stabilisation requests that have been automatically
> repoman checked by a bot (described below).
>
> To prevent cluttering the Bugzilla interface, I suggest resurrecting the
> "Stabilisation" component and having these new fields appear only for
> bugs filed there and in "Security: Vulnerabilities" (so not to disrupt
> the security team's workflow).
>
>
> Automation
> ==========
>
> It's easy to forget to check that all the required dependencies are in
> stable before filing a stabilisation test, but this wastes the actioning
> developer's time. I have prepared a bot that repoman checks the list of
> atoms and flags the bug appropriately. This allows easy filtering out of
> broken requests.
>
>
> Improved tools
> ==============
>
> To support the above, I am also preparing some tools to streamline the
> process further:
>
> * A script to, given a bug, produce an architecture-specific list of
> atoms ready to feed into package.accept_keywords / emerge
>
> * An updated version of app-portage/tatt (allowing easy testing of USE
> flag combinations, reverse dependencies, committing keyword changes etc.)
>
> * Your idea here
>
>
> Too long; didn't read
> =====================
>
> We add a new box to Bugzilla to hold atoms on stabilisation request bugs
> so it's easy to extract programmatically.
>
> We add a new box to Bugzilla to indicate if a stabilisation request
> requires runtime testing or not to better focus effort.
>
> If your stabilisation request is invalid a bot will let you know and it
> will be easy to skip your request until you fix it.
>
>
> This will also help pave the way for future enhancements such as
> triggering automated build and QA tests so that the human only has to do
> runtime testing.
>
> Thoughts?
>
+1

Doing a great job there, kensington :]

Will grab a paint-brush for the 'shed, soon as my own have completed
erection....