LLVM build strategy (was: Re: [gentoo-dev] [RFC] New categories for LLVM)

[Sam James <sam@gentoo.org>]

Michał Górny <mgorny@gentoo.org> writes:

> Hello,
>
> Given that the number of LLVM packages is growing, and probably will
> grow again (I'm introducing "offload" right now, expect at least MLIR
> soon, there are open requests for flang, polly...), I'd like to propose
> creating dedicated categories for these packages and moving them there.
>
> [...]
>
> WDYT?

I fear this sort of assumes we won't switch to monobuild any time soon.

I keep thinking [0] about how sustainable our current setup is:
* Fedora moved away from it for >=18 [1].
* As we saw with offload, it broke a few times in just a week. So it's
  only Gentoo who cares about this configuration AFAIK.
* It's not working great if we're already not able to easily package
  mlir, flang, or polly.

Violet attempted working on a merge before [2].

The trade-offs I see are:
* Increased disk space requirement for building LLVM

  I think that's a legitimate concern although perhaps not that big of a
  deal given it is, after all, a whole toolchain. GCC takes quite a bit
  to build too.

* Build time

  Build time for mono LLVM would increase as we're building more
  components at least for some users.

  But the time added by
  building more components (see below) should be balanced out by ccache if
  doing development and also, importantly, not needing to force on all
  targets anymore (they keep growing).

  The cumulative time should be the same (although maybe a bit less
  given the targets change) given that most people need the
  same set of components because of mesa, firefox, or other things which
  need libclang.

* Moving to an upstream supported-configuration

  We may have a bit of pain in moving to it and getting used to it, but
  we're no longer swimming upstream and being the only people caring
  about our choice of build configuration (and regularly having to
  explain and justify it to upstream).

  Upstream also recommend doing a bootstrap build. We don't and can't do
  that right now.

  Folks upstream state at every opportunity that they don't care about
  standalone builds,
  e.g. https://discourse.llvm.org/t/rfc-do-something-with-the-subproject-tarballs-in-the-release-page/75024/2.

* Better support for LLVM as a system toolchain

  The current setup doesn't work well for people using LLVM as a system
  toolchain (because some of the components *must* be upgraded together),
  it doesn't work well for people who want to use mlir/flang/polly, and it
  doesn't work well for users on constrained hardware because we have to
  force on all targets. It also prohibits more optimisation, PGO, and
  bootstrapping it to test reliability.

  (This is why I'm not too sympathetic to claims that the monobuild is
  mostly for binary distributions, because we're actually *more*
  vulnerable to issues as a result of it being split when building from
  source if using the LLVM toolchain.)

* Maintaining older LLVMs

  It's easier for older LLVMs to be either maintained by somebody else
  (for e.g. Rust's purposes) or in an overlay if it's a single package
  rather than many.

  This is also true for e.g. testing old snapshots or keeping binary
  packages to downgrade to once they got cleaned up.

At the moment, I fear us choosing the non-recommended path gives us very
little, and causes a bunch of problems in return.

I don't particularly care much personally about using LLVM as the system
toolchain and if that's the only reason to move to monobuild, I'd be far
less worried about it (even if I think using it as a system toolchain is
reasonable for someone to work on).

But if it's getting in the way of packaging things like flang and it's
100% on us to make standalone work - and keep it working - is it really
worth it?

I will note that it seems like the breakage has reduced upstream in the
last few cycles, although the offload case was pretty bad just in the
last week. On the other hand, if you think the problems with mlir and
friends are easily-solvable, then we may be fine as we are?

IIRC, Violet ended up running into issues with getting the CMake to
properly handle both shared and static libraries to install with the
monobuild, so maybe LLVM's CMake is just broken in both directions and
we just have to pick whichever one is least-worst for us (which might be
the standalone builds as we do now)?

[0] https://github.com/gentoo/gentoo/pull/32716#issuecomment-1713689410
[1] https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/23QT5ISGGOQUOMTOOLMUBV6MSBB2C2C6/#PDAXH7PZTA7SFSSKF65KIXYLPYHAI3SS
[2] https://github.com/gentoo/gentoo/pull/32716

sam