[gentoo-dev] Re: LLVM build strategy

[Sam James <sam@gentoo.org>]

Sam James <sam@gentoo.org> writes:

> Michał Górny <mgorny@gentoo.org> writes:
>
>> Hello,
>>
>> Given that the number of LLVM packages is growing, and probably will
>> grow again (I'm introducing "offload" right now, expect at least MLIR
>> soon, there are open requests for flang, polly...), I'd like to propose
>> creating dedicated categories for these packages and moving them there.
>>
>> [...]
>>
>> WDYT?
>
> I fear this sort of assumes we won't switch to monobuild any time soon.

(I'm not even sure I'm advocating a move, it's more that I'd like to
flesh out the arguments for it so I have it clear in my mind.)

>
> I keep thinking [0] about how sustainable our current setup is:
> * Fedora moved away from it for >=18 [1].
> * As we saw with offload, it broke a few times in just a week. So it's
>   only Gentoo who cares about this configuration AFAIK.
> * It's not working great if we're already not able to easily package
>   mlir, flang, or polly.
>
> Violet attempted working on a merge before [2].
>
> The trade-offs I see are:
> * Increased disk space requirement for building LLVM
>
>   I think that's a legitimate concern although perhaps not that big of a
>   deal given it is, after all, a whole toolchain. GCC takes quite a bit
>   to build too.
>
> * Build time
>
>   Build time for mono LLVM would increase as we're building more
>   components at least for some users.
>
>   But the time added by
>   building more components (see below) should be balanced out by ccache if
>   doing development and also, importantly, not needing to force on all
>   targets anymore (they keep growing).
>
>   The cumulative time should be the same (although maybe a bit less
>   given the targets change) given that most people need the
>   same set of components because of mesa, firefox, or other things which
>   need libclang.
>
> * Moving to an upstream supported-configuration
>
>   We may have a bit of pain in moving to it and getting used to it, but
>   we're no longer swimming upstream and being the only people caring
>   about our choice of build configuration (and regularly having to
>   explain and justify it to upstream).
>
>   Upstream also recommend doing a bootstrap build. We don't and can't do
>   that right now.
>
>   Folks upstream state at every opportunity that they don't care about
>   standalone builds,
>   e.g. https://discourse.llvm.org/t/rfc-do-something-with-the-subproject-tarballs-in-the-release-page/75024/2.
>
> * Better support for LLVM as a system toolchain
>
>   The current setup doesn't work well for people using LLVM as a system
>   toolchain (because some of the components *must* be upgraded together),
>   it doesn't work well for people who want to use mlir/flang/polly, and it
>   doesn't work well for users on constrained hardware because we have to
>   force on all targets. It also prohibits more optimisation, PGO, and
>   bootstrapping it to test reliability.
>
>   (This is why I'm not too sympathetic to claims that the monobuild is
>   mostly for binary distributions, because we're actually *more*
>   vulnerable to issues as a result of it being split when building from
>   source if using the LLVM toolchain.)
>
> * Maintaining older LLVMs
>
>   It's easier for older LLVMs to be either maintained by somebody else
>   (for e.g. Rust's purposes) or in an overlay if it's a single package
>   rather than many.
>
>   This is also true for e.g. testing old snapshots or keeping binary
>   packages to downgrade to once they got cleaned up.

Another issue with monobuild would be handling the case where
USE-depends-on-other-USE and you either have a lot of REQUIRED_USE, or
USE that do nothing. This is more of an issue for the runtimes, I think.

We would also have an issue where we either have to default-enable
Clang, or require many, many users to enable it manually/via autounmask
(as opposed to now where it gets dragged in as a dep).

> [...]