Re: LLVM build strategy (was: Re: [gentoo-dev] [RFC] New categories for LLVM)

[Violet Purcell <vimproved@inventati.org>]

On Sun, Dec 08, 2024 at 04:53:58AM +0000, Sam James wrote:
> I fear this sort of assumes we won't switch to monobuild any time soon.
> 
> I keep thinking [0] about how sustainable our current setup is:
> * Fedora moved away from it for >=18 [1].
> * As we saw with offload, it broke a few times in just a week. So it's
>   only Gentoo who cares about this configuration AFAIK.
> * It's not working great if we're already not able to easily package
>   mlir, flang, or polly.
> 
> Violet attempted working on a merge before [2].

Hey! Thanks for bringing up my work on this. I've also been continuing
this work in an overlay which I use myself on some of my machines:
https://codeberg.org/vimproved/llvm-overlay.

> IIRC, Violet ended up running into issues with getting the CMake to
> properly handle both shared and static libraries to install with the
> monobuild, so maybe LLVM's CMake is just broken in both directions and
> we just have to pick whichever one is least-worst for us (which might be
> the standalone builds as we do now)?

LLVM's CMake is definitely just rather broken in many ways, yeah. The
biggest issue I experienced when working on this was with LLD libraries.
Currently, sys-devel/lld is built with -DBUILD_SHARED_LIBS=ON, which
builds some LLD libraries used by e.g. zig as shared libraries. However,
there's no way to do this in monobuild since passing
-DBUILD_SHARED_LIBS=ON is not desirable. The solution I settled on was
splitting a sys-libs/lld-libs package which builds LLD with
-DBUILD_SHARED_LIBS=ON and then only installs the shared libraries.

As for sys-libs/llvm-libs, this package is a result of me trying to
figure out a way to have a statically linked system clang (for
performance purposes) while still having LLVM functional as a library.
After a lot of sifting through CMake spaghetti I settled on splitting
the LLVM monobuild into two packages: sys-devel/llvm and
sys-libs/llvm-libs. sys-devel/llvm installs the toolchain parts of LLVM
(statically linked), and sys-libs/llvm-libs installs the library and
CMake parts of LLVM.

This overlay is just for my personal use, and I don't think the above
package split would be viable for ::gentoo however.

One more thing that wasn't mentioned here, LLVM monobuild would also
potentially unblock PGO/BOLT optimizations for clang, so overall it
could have some pretty good speed improvements for system LLVM toolchain
users.

- Violet