Re: [gentoo-project] call for agenda items -- council meeting 2021-05-09

[Sam James]

> On 27 Apr 2021, at 19:56, Andreas K. Huettel <dilfridge@g.o> wrote:
>
> I'd like to kick off a discussion whether LTO should be considered "supported". With that I essentially mean that bugs involving LTO should be considered valid, and fixes (be it only stripping -flto from flags, or similar solutions) should be committed to the tree.
>

Forgive me for giving a tiny bit of unstructured opinion about USE=lto, before I dive into the actual proposal:

1) I’m really happy to use LTO whenever it is supported upstream (just like -O3, etc) but I don't use it out of thin air.

2) For that reason, I personally like it when USE=lto exists even when no specific build system hacks are required (because it tells me “upstream will help with bugs” and so on) but I completely understand this is
  a bit at odds with what we usually do, and therefore is something I just need to get used to not having.

Of course, this problem goes away if we’re going to generally encourage tinderboxes and general LTO usage, just like we did with as-needed.

> I would like to clarify this before possibly suggesting an initiative to make the Gentoo repository LTO-safe (similar to what we did years ago with --as-needed).

I’d be interested in if slyfox or Soap had any input on heuristics to help determine if something is likely to be unsafe. LTO is really good at “provoking” undefined behaviour. Build completing means very little in terms of success here.

I don’t really want to go around running UBSAN on everything to know it’s safe to use it. Polynomial-C mentioned data corruption at runtime with some packages in #gentoo-dev the other day too (this kind of experience is very real and we need to mitigate it).

Obviously that would be a good candidate for stripping out LTO, but how are we supposed to notice this stuff if it only happens under certain circumstances?

My rough plan would be:
- Coordinate via e.g. wiki pages (and IRC as usual)
- *Strong* focus on packages with test suites so that we can get some idea of whether it’s working correctly with LTO. Let’s ignore those without tests in the first round(s).
- Provide some rough documentation for developers on how to build with UBSAN which we can use at least for critical applications
- For codebases which are known to be “rough” (and we would include feedback from the LTO overlay [0] here), we’d possibly filter LTO flags proactively (at least if they’re critical packages).

>
> Background is, just about every binary distribution out there builds with LTO by default now. It's not so great if we then keep telling people "LTO is dangerous".

Right. Fedora are doing this and Clear Linux has been doing this for a very long time too. What I find interesting is that I’ve never actually come
across any patches in either to fix LTO issues, which either means I’m (un)lucky or they’re not hitting issues so often?

Obviously, we end up hitting more than other people because of often exotic configurations on the user side, but it is what it is.

This is one of those situations where reaching out to some folks we know in other distros for some (unstructured) thoughts might not be a bad idea - just
to find out some e.g. heads up on problematic codebases.

Best,
sam

[0] https://github.com/InBetweenNames/gentooLTO

>
> Cheers,
> Andreas
>
> (Yes I'm aware of the LTO overlay. It may be a great source.)
>
> --
> Andreas K. Hüttel
> dilfridge@g.o
> Gentoo Linux developer
> (council, toolchain, base-system, perl, libreoffice)