Re: [gentoo-alt] clang-5 on macOS

[Michael Weiser]

Hi Fabian,

On Tue, Dec 26, 2017 at 11:36:52AM +0100, Fabian Groffen wrote:

> > After upgrading to macOS 10.13, a couple of packages stopped compiling
> > with clang 3.9.1. The errors came from system headers. It looked like
> > Apple had again used some funky new clang feature in them which 3.9.1
> > didn't support.
> Yes, I noticed that too, see:
> https://bugs.gentoo.org/632500

Ah, right, totally forgot about bison: I grabbed the workaround from
Homebrew some time back
(https://github.com/Homebrew/homebrew-core/issues/18591).

> > Getting clang-5 up and running wasn't all that bad: I needed to clone
> > the libcxx and libcxxabi ebuilds for version 5.0.1, adjust eclass
> > cmake-utils and reintroduce some prefix provisions to clang and
> > compiler-rt ebuilds that didn't make the move from the llvm ebuild.
> Oh, I experienced failures to compile it on a 10.13 profile.

Considering that the only real difference I can see between profiles is
the MACOS_DEPLOYMENT_TARGET, it might have been the rpath problem. It
failed to compile for me too until I removed
CMAKE_BUILD_WITH_INSTALL_RPATH from the cmake config. This indication is
that some binary (most likely llvm-tblgen) is called from the build
directory and croaks on the old LLVM libs it loads from EPREFIX/usr/lib
with missing symbols.

> > Does anyone remember why CMAKE_BUILD_WITH_INSTALL_RPATH was necessary
> > for cmake ebuilds?
> I don't recall, seems like this indeed shouldn't be forced.

I'll just go ahead and drop it from cmake-utils.eclass completely and
see what breaks.

> > @rpath. So I had to force an *install* rpath of
> > ${EPREFIX}/usr/lib/llvm/${SLOT}/lib.
> I assume this is install_name_tool-ing.

Well, in a way: The libraries' ids get to be @rpath/<lib> as well. This
way, when linking into a binary, they don't bring the path to themselves
with them any more. The benefit is that an LLVM/clang installation can
be fully relocatable. The drawback is, that you need to muck around with
DYLD_{,FALLBACK_}LIBRARY_PATH, link every binary that links against
them with an rpath where they're on or compile a static rpath into them
as I did. The latter still doesn't propagate into a binary that links
against them. So you can dlopen() them and they'll find their
dependencies. But linking against them still requires putting an rpath
into the binary they're linked into. It's all somewhat sad.

I now realize that this might have been exactly the intention of the
cmake settings, especially CMAKE_INSTALL_NAME_DIR. But a.) we'd need to
adjust that for the llvm slot directory and b.) we'd need to disable it
for the build directory.

BTW: AFAIK all this isn't done using install_name_tool but directly when
linking the binaries. According to
https://cmake.org/cmake/help/v3.0/variable/CMAKE_BUILD_WITH_INSTALL_RPATH.html,
cmake normally links libs and executables twice: Once upon build with a
additions to rpath that make them consistently use the newly compiled
libraries in the build tree and then again upon install with the actual
install rpath. This would also change the library id (install name).

Here's what it looks like on my system right now:

# otool -D EPREFIX/usr/lib/llvm/5/lib/libLLVMSupport.dylib 
EPREFIX/usr/lib/llvm/5/lib/libLLVMSupport.dylib:
@rpath/libLLVMSupport.dylib
# otool -L EPREFIX/usr/lib/llvm/5/lib/libLLVMSupport.dylib 
EPREFIX/usr/lib/llvm/5/lib/libLLVMSupport.dylib:
	@rpath/libLLVMSupport.dylib (compatibility version 0.0.0, current version 0.0.0)
	EPREFIX/usr/lib/libncurses.6.dylib (compatibility version 6.0.0, current version 6.0.0)
	EPREFIX/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.0.0)
	@rpath/libLLVMDemangle.dylib (compatibility version 0.0.0, current version 0.0.0)
	EPREFIX/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1.0.0)
# otool -l EPREFIX/usr/lib/llvm/5/lib/libLLVMSupport.dylib  
EPREFIX/usr/lib/llvm/5/lib/libLLVMSupport.dylib:
[...]
Load command 3
          cmd LC_ID_DYLIB
      cmdsize 56
         name @rpath/libLLVMSupport.dylib (offset 24)
Load command 12
          cmd LC_LOAD_DYLIB
      cmdsize 56
         name /usr/lib/libSystem.B.dylib (offset 24)
   time stamp 2 Thu Jan  1 01:00:02 1970
      current version 1252.0.0
compatibility version 1.0.0
Load command 13
          cmd LC_LOAD_DYLIB
      cmdsize 56
         name @rpath/libLLVMDemangle.dylib (offset 24)
   time stamp 2 Thu Jan  1 01:00:02 1970
      current version 0.0.0
compatibility version 0.0.0
Load command 14
          cmd LC_LOAD_DYLIB
      cmdsize 72
         name EPREFIX/usr/lib/libc++.1.dylib (offset 24)
[...]
Load command 16
      cmd LC_DATA_IN_CODE
  cmdsize 16
  dataoff 1040248
 datasize 440
Load command 17
          cmd LC_RPATH
      cmdsize 56
         path EPREFIX/usr/lib/llvm/5/lib (offset 12)
              ^ by default this is @loader_path/../lib or @executable_path/../lib

> > As long as neither USE tapi nor lto are enabled on binutils-apple, they
> > should be fine. Symlinks to size and nm will break and will need
> > adjustment to point to the new llvm install though.
> Hmm...

I've put the current version of the updated ebuild into a new bug
#642292. Rationale for the changes are somewhat scattered in the bug
comments, ebuild, patch headers and comments in the patches. Let me know
if you'd like me to collect them in one place. :)

The main change is that I added a function and a shim binary to cctools
that goes looking for llvm tools and libs in the slot directories for
major versions 10 to 4 and EPREFIX/usr/bin directly. Since cctools
dlopen libLTO and exec() the tools, that should make them very robust
against updates.

ld64 is another story because it directly links against libLTO. If only
libLTO had a proper version number, portage could help by preserving it
on update. As of now, only the different slot paths of llvm major
versions might provide some protection against breakage.

> > Any comments on my approach would be highly appreciated. Patches
> > attached for documentation.
> Seems like you got yourself pretty deep into it!  clang-5 is the blocker
> for supporting high sierra IMO.  Can you now build 5 using 3.9.1?

I have indeed successfully built llvm and clang 5.0.1 using 3.9.1 with
the above changes in my existing prefix. I am now trying to get all
packages updated to the current portage tree, switch profile and CHOST
to 10.13 and recompile the whole thing again. This will tell, if it's
self-hosting. After that I can try downgrading llvm/clang and upgrading
it again to see what breaks.

> I think the cmake patches are ok, I'll look into the ebuilds you
> mention.  I'll sync whatever I can find in this area.

Actually, the cmake patch might be incorrect since it got lost on emaint
sync and I had to reconstruct it from memory. I'm sure about
CMAKE_BUILD_WITH_INSTALL_RPATH, CMAKE_INSTALL_RPATH,
CMAKE_INSTALL_RPATH_USE_LINK_PATH but I think I also dropped
CMAKE_INSTALL_NAME_DIR for llvm and clang.
-- 
bye, Michael