Re: LLVM build strategy (was: Re: [gentoo-dev] [RFC] New categories for LLVM)

["Michał Górny" <mgorny@gentoo.org>]

[-- Attachment #1: Type: text/plain, Size: 3932 bytes --]

On Sun, 2024-12-08 at 04:53 +0000, Sam James wrote:
> I fear this sort of assumes we won't switch to monobuild any time soon.

I don't see one precluding the other.  Categories are cheap.  Package
moves not necessarily, but switching to monorepo will be complete pain
whether one more package move is involved or not.

> I keep thinking [0] about how sustainable our current setup is:
> * Fedora moved away from it for >=18 [1].
> * As we saw with offload, it broke a few times in just a week. So it's
>   only Gentoo who cares about this configuration AFAIK.

It broke once, and only because the pull request merged preceded my
changes, and the author dealt with merge conflicts wrong.

That said, it's not like I didn't fix the monorepo build as well this
week, because it was broken from day one.

We're on our own either way.


> * Build time
> 
>   Build time for mono LLVM would increase as we're building more
>   components at least for some users.
> 
>   But the time added by
>   building more components (see below) should be balanced out by ccache if
>   doing development and also, importantly, not needing to force on all
>   targets anymore (they keep growing).

I don't see how we would avoid forcing targets if *external* projects
(wasn't the bug about Rust originally?) can still be broken if you
change targets.

>   The cumulative time should be the same (although maybe a bit less
>   given the targets change) given that most people need the
>   same set of components because of mesa, firefox, or other things which
>   need libclang.

So you spend hours building LLVM and Clang.  Then you spend hours
building everything again because one more packages needs LLD.  Then
more hours because you've decided to try LLDB.

I've been rebuilding three LLVM versions recently because of cpp-httplib
changing subslot multiple times recently.  With the proposed change, I'd
be rebuilding everything instead.

In fact, I've already started considering splitting llvm-debuginfod.

> At the moment, I fear us choosing the non-recommended path gives us very
> little, and causes a bunch of problems in return.

If you consider being able to have a really working LLVM package "very
little", so be it.

If you choose to go for monorepo, I'm stepping down, because
I definitely won't be able to deal with this shit without being able to
split it into smaller parts.

I don't like the idea that any minor patch (think of compiler-rt that
regularly needs to be updated for newer glibc) will require spending
hours rebuilding everything.  In multiple LLVM versions.  For every
single person, including all the people who don't build compiler-rt
at all.

I don't like the idea of not being able to run parts of test suite
without resorting to ugly hacks.  I don't like the idea of spending
hours building everything before I'm even able to start running tests,
just to learn that LLVM is broken and there's no point in even starting
to build the rest.  Or having the test suite fail after a few hours
because of some minor configuration issue (like the one we'd had with
libcxx, and I've hit that one three times), and having to start
everything over again.


And ccache is not a solution.  It's a cheap hack, and a costly one. 
Maintaining a cache for this thing requires tons of wasted disk space. 
And unless you go out of the way to reconfigure it, building 2-3 LLVM
versions will normally push all previous objects of the cache, so it
won't work for most of the people at all.  Provided they go out of their
way to configure it in the first place.

In the end, LLVM is a project primarily maintained by people working for
shitty corporations that only care about being able to build their
shitty browser written in bad C++.  It sucks we ended up having to
maintain it, but that's not our choice.

-- 
Best regards,
Michał Górny


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 512 bytes --]