[gentoo-dev] Stabilisation procedure

[Michael Palimaka]

I've prepared a document that tries to consolidate the various bits of
information regarding the stabilisation procedure into one document /
policy. Please review.

The live version currently lives at
https://wiki.gentoo.org/wiki/User:Kensington/Stabilisation_procedure and
I've included the raw text below for convenience.




This article describes the procedure for moving an ebuild from testing
to stable.

== Responsibility ==

The primary purpose of the stabilisation process is to integrate a
testing ebuild into the stable tree. This can involve maintaining the
consistency of the dependency graph, basic compatibility checks with
other packages, and smoke testing of the package itself.

Stabilisation is not intended to relieve a package maintainer of their
responsibility to ship a quality package - the primary responsibility of
ensuring that a package is a good stable candidate remains with the
person approving the stabilisation request. The stabilisation process
does not include more than basic functionality checks unless explicitly
requested.

== Configuration ==

It is preferred that testing take place on a real system, inside a
chroot, or within another type of non-virtualised container.
Virtualisation may be acceptable in situations where it is not possible
or practical to test on real hardware.

The testing system must only have stable packages installed, with no
testing packages keyworded or unmasked. It should be up-to-date, and it
is recommended to have as few packages installed as possible.

{{path|make.conf}} should have settings similar to the following:
{{FileBox|filename=/etc/portage/make.conf|lang=bash|1=
# CFLAGS should be reasonable
CFLAGS="-march=native -O2 -pipe -frecord-gcc-switches"
CXXFLAGS="${CFLAGS}"

# Defining all four *FLAGS variables enables a portage QA check
reporting when these variables are not respected
FFLAGS="${CFLAGS}"
FCFLAGS="${CFLAGS}"

# Enables a portage QA check to report when LDFLAGS is not respected
LDFLAGS="${LDFLAGS} -Wl,--hash-style=gnu"

# collision-protect - prevent a package from overwriting files it does
not own
# ipc-sandbox - prevent host IPC access (requires Linux and namespace
support in kernel)
# network-sandbox - prevent network access during merge (requires Linux
and network namespace support in kernel)
# sandbox - ensure package does not write directly to live system
# split-log - store logs created by PORTAGE_ELOG_SYSTEM="save" in
category subdirectories
# split-elog - store build logs in category subdirectories
# strict - have portage react strongly to conditions that have the
potential to be dangerous
# test - run package tests, or alternatively test-fail-continue
# userfetch - drop privileges during fetching
# userpriv - drop privileges during merge
# usersandbox - enable sandbox when userpriv is enabled
FEATURES="collision-protect ipc-sandbox network-sandbox sandbox
split-log split-elog strict test userfetch userpriv usersandbox"

# Display selected types of messages again when emerge exits, and save
them to disk
PORTAGE_ELOG_CLASSES="log warn error qa"
PORTAGE_ELOG_SYSTEM="echo save"
}}

== Testing ==

Each package in Gentoo is different and therefore requires a slightly
different approach to stabilisation. Consider the following guidelines
for each class of package, and use common sense when in doubt.

=== General ===

==== USE flags ====

While it is preferable to test every USE flag combination, this is not
always possible or appropriate. The package may have a large number of
USE flags, a long compile time, or the stabilisation in question may
just not call for it.

In cases where all USE flags combinations are not being tested, it is
still recommended to test:
* with all USE flags enabled
* with all USE flags disabled
* the default USE flag settings

==== Runtime testing ====

Consider the level of runtime testing that is required for the target
package. Remember, the focus of stabilisation is to integrate a testing
ebuild into the stable tree and not to identify routine bugs or
regressions - that is the purpose of the package's waiting time in ~arch.

The level of runtime testing required will vary wildly based on a
variety of factors. Consider the following examples:

* Multiple days of "normal use" testing may be appropriate for a new
version of {{package|sys-libs/glibc}}
* Basic functionality testing, such as browsing some web pages, may make
sense for a new version of {{package|www-client/firefox}}
* Passing tests might be enough for {{package|dev-python/yenc}}
* A leaf package such as {{package|kde-apps/kcalc}} may not require any
runtime testing at all

=== Libraries ===

A new library version may introduce incompatibles with reverse
dependencies. Where there's a risk of such breakage, each stable reverse
dependency must be rebuilt. Beware of reverse dependencies that only use
the library conditionally (eg. <code>USE="png"</code>).

=== Kernel ===

Kernel packages referenced in the handbook have certain exemptions from
the usual stabilisation policy, so stabilisation requests are normally
only filed for the first version in a long term stable branch
(subsequent versions can be stabilised at the discretion of the maintainer).

First, test all available kernel options:

{{Cmd|cd /usr/src/example-sources-1.2.3
|make allyesconfig
|make # add '-j' as appropriate for your hardware}}

If that succeeds, build with your normal configuration:

{{Cmd|make distclean
|make menuconfig
|make
|make modules_install # if you use modules}}

After reboot, check <code>dmesg</code> for anything strange and use the
system as normal, trying to get a bit of uptime.

If stabilising a special feature variant such as
{{package|sys-kernel/hardened-sources}}, try to test those features.

=== Toolchain ===

New versions of toolchain packages can often introduce major changes and
widespread breakage into the tree. The purpose of a stabilisation
request for a toolchain package is to test the package itself on each
architecture - not to detect build failures in miscellaneous packages.
It is expected that such failures are managed and resolved by the
maintainer (normally through tracker bugs and tinderboxing) prior to
filing a stabilisation request.

Once the normal testing is successful, rebuild <code>@system</code> (or
<code>@world</code> if the hardware permits) and once successful,
observe the system in normal operation for abnormalities.

== QA violations ==

Most of these violations will be detected automatically using the
testing tool, but are also described here for completeness.

* Does not respect CC
* Does not respect CFLAGS
* Does not respect LDFLAGS
* Bundled symbols
* Insecure symbols
* Installs documentation outside of /usr/share/doc/${PF}
* ELF files found in /usr/share
* ...
* ...

== Architecture-specific notes ==

A number of items described in earlier sections, such as checking of
reverse dependencies and miscellaneous QA checks, are
architecture-neutral. At a stabilisation level, the primary
responsibility for carrying out these checks rests on the first
architecture to stabilise an ebuild. Subsequent architectures may assume
that these checks have been completed and skip them if they wish.

=== amd64 ===
* Any developer may perform {{keyword|amd64}} stabilisations - it is not
necessary to be on the arch team
* <code>multilib-strict</code> must be added to <code>FEATURES</code>

=== arm ===
The [[Project:ARM|ARM project]] supports four {{keyword|arm}} variants -
armv4, armv5, armv6, and armv7. In addition to regular testing, the
package must be build tested on each variant. If access to each physical
variant is not possible, <code>CFLAGS="-march=$arch"</code> is acceptable.

=== x86 ===
* Any developer may perform {{keyword|x86}} stabilisations - it is not
necessary to be on the arch team
* It is acceptable to stabilise in an {{keyword|x86}}
[[Project:AMD64/32-bit Chroot Guide|chroot]] on {{keyword|amd64}}
* It is generally acceptable to stabilise a package with only a build
test on {{keyword|x86}} if it is already stable on {{keyword|amd64}}

== Acknowledgements ==
Most of this guide was shameless stolen from many sources, including but
not limited to:
* Agostino Sarubbo
* Various arch teams