Re: [gentoo-dev] Changing policy about -Werror

[Alon Bar-Lev]

Hi,

I was the one that (re)trigger this discussion, I thank bircoph for
opening this up.

First, let me apologize that I did not test the capi USE for long
time, as this option is not used for long time by users, I am also
apologize of treating bug from anyone (may it be user, developer or
even qa) at the same SLA. It took one hour from report to fix
including evaluation of the issue and submitting the fix to upstream.
The package is non-stable for four months and stable in amd64 for a
while without users reporting that because as expected feature is
seldom used. Please avoid flaming me for this and address the
principal subject at hand as it is important.

I would like to suggest a modify our policy with the following
exception clause: Package maintainer may decide to keep upstream
-Werror as long as he is willing to resolve all issues resulting from
-Werror as if it was a blocker bug.

The package maintainer decision may be based on the package state and
the relationship with upstream, but basically, it is his decision as
long as issues are fixed in timely manner, it is his overhead after
all.

I am maintaining selected packages in which upstream is fully aware of
-Werror implications, accepting any patches to resolve issues found by
anyone. My judgment as Gentoo developer for these selected packages is
to absorb additional overhead in maintaining these packages while
helping upstream and users to enjoy higher quality. I've never assumed
that an effort individual project is willing to invest is something
that overrule by any other project as long as users receives a great
service (bugzilla stats are available).

I believe that a maintainer in Gentoo is a special role, we not only
helping users, but we also help upstream to improve their packages. We
are unique as permutations and architectures that are used by Gentoo
users are so diverse that we find issues that nobody else finds. In
addition to these, we also use latest and greatest toolchains, tools
and utilities, this triggers issues at Gentoo even before other
distributions.
Gentoo non-stable users are a great asset as they are willingly
helping to find these issues, with the understanding that their system
may break. A good relationship between Gentoo downstream maintainer
and upstream actually helps to find fixes long before other
distributions are affected. Even when I was in Red Hat, my policy was
Gentoo first, as a package that is built in Gentoo without tweaks will
be built successfully in all distributions (except of Java maven
packages in which we are far behind).

Over the years, many Gentoo developers, I included, have built
relationship with most active upstreams in my slot to push Gentoo
interests into upstream, I invite you to portage tree to see in how
many packages we drag patches from one version to another or to
evaluate ebuild tweaks. This mutual respect also suggests that if
upstream has a -Werror policy, in which every issue as minor as it can
be must be taken care of before package is considered to be usable, I
as downstream maintainer should help and apply the policy to help
upstream to improve its policy.

More and more upstream developers are aware of static code analysis
benefits, they use every source of information possible, opening
coverity to opensource made a great difference, the -Werror is yet
another tool to find unexpected issues. The collective mindset was
progressed greatly since 2009 where flameeyes opened bug#260867. At
least once per 10 years one should re-evaluate his policies, the
industry is changing.

When upstream policy is to have zero warnings policy, suppressed by
explicit suppression (code or compile argument), and when upstream is
friendly and actively following the policy of investigating each
incident and fix it properly, we can help for selected packages.

ARGUMENT: If a package compiled in the past using toolchain X then it
must continue to do so with any future toolchain.

I do not understand when "build" is the test for runtime, for 30 years
I tell my developers and students that "It compiles" and "It works"
are two different statements. One should only be thankful for any tool
to detect issues hidden within code, stop and re-evaluate when new
issues are found.

Let's take two recent examples from my queue:

bug#665464 in which there was unused return code variable, this made
me look into source code of upstream from master back in time until
code was introduced to find out when return code was used and why it
was removed, reaching to a conclusion that indeed return code should
be ignored and submitting a patch to upstream to validate that,
upstream confirmed and merged. Imagine what would happen if I would
have found that it is a real issue and should be addressed? Both users
and upstream would have benefit from finding and fixing the issue.

bug#664198 in which -Werror found mismatched memcpy, this was an
actual bug that must be fixed.

Based on the above we have in recent month one false positive and one
positive issues. These issues in most cases found by our non stable
user community, either these who are using individual packages as non
stable or these that are using a non-stable toolchain. Many of these
issues are triggering automation such as the one that Toralf Förster
is running that helps to improve Gentoo directly and the entire open
source eco-system indirectly. Stable users are seldom affected from
-Werror issues as our non stable process usually enables to resolve
these issues before stabilization.

ARGUMENT: Old packages in tree will not be built with newer toolchain
if -Werror is enabled.

The role of package maintainer is to make sure that everything in tree
is working regardless of -Werror, he has the choice either to remove
old incompatible packages or patch them to remove  -Werror.

There are different reasons why a package will not be built, for
example when nothrow destructors were introduced, we must had fixed
many packages to meet this new policy to avoid undesired program
termination.

The policy I take is to leave only recent stable package for each
slot, as there usually sufficient time when stabilization is starting
and ending.

ARGUMENT: -Werror cause even more issues when cross compiling a package

When -Werror is carefully maintained by both upstream and downstream,
and when cross compile is supported by both upstream and downstream,
there is no reason why -Werror have any side effect compared to
native. Every issue resulted out of -Werror must be evaluated and
resolved per the upstream policy of supporting -Werror.

Package maintainer has always the option to remove -Werror if package
is causing maintenance overhead which cannot be resolved in decent
resources.

ARGUMENT: User do not care about helping upstream to improve their packages

I believe Gentoo users do care about upstream, Gentoo unstable users
are unstable exactly for this purpose. For sure, Gentoo users
understand that the more Gentoo patches software the less support they
can receive directly from upstream.

It is sufficient to perform autoreconf to replace upstream tools to
void the warranty of upstream.

Users who is using Gentoo are advanced users compared to other
distributions and are fully aware of the upstream quality, in many
cases they help us to convince upstream to be receptive.

ARGUMENT: Maintainer must join toolchain team and test everything with
every release candidate of the compiler

The Gentoo unstable process is exactly designed for this phase,
without joining other project a developer can test his packages with
the recent version of dependent packages.

ARGUMENT: Any maintainer can add this to his CFLAGS him/herself if
s/he wishes so.

Unlike binary semi-commercial distributions, Gentoo do not provide an
automation infrastructure for developers to use. A Gentoo developer is
using his own system and resources to test packages at best effort,
delegating the rest of the permutations for unstable users, arch teams
and then to stable users.

Don't get me wrong, I would love to have an environment provided by
infra to allow me to run an ebuild candidate for all architectures and
all USE flag combinations in a single batch and to trigger rebuild
when every non-stable dependency is introduced or when @system is
modified.

I am thankful for Toralf Förster efforts to help us be closer to automation.

However, enabling the flag only at the architecture in which the
developer has access to has a very little value, the -Werror is a life
saver when it is causing build failure on environment the developer do
not have access to.

ARGUMENT: Add -Werror when a specific USE flag is set

This USE flag will probably not be enabled for most of users, and
enabled only after a damage happened.

One can expect automation to enable this flag, however, if we already
have automation then automation will assist of resolving the -Werror
issues so that issues will not be manifested later in pipeline.

We can conditionally patch the package in non-stable and stable,
however, any patch is evil and fork us from upstream as I described
above.

ARGUMENT: You have no idea how much unneccessary pain -Werror caused
when gcc started warning on "misleading indentation".

Whoever maintain -Werror packages as downstream or upstream has
idea... Please avoid these statements.

Even misleading indentation may be result of incorrect patch merge or
visually undetected branch that is not being executed at the right
flow, we are human after all.

Based on the above, I would like us to allow maintainer to decide when
and if he would like to maintain a package with -Werror based on by
statement at the top.

Regards,
Alon