Re: [gentoo-alt] clang-5 on macOS

[Fabian Groffen]

On 26-12-2017 00:17:55 +0100, Michael Weiser wrote:
> Happy Christmas!
>
> Is anyone (else) working to get clang-5 running in prefix on macOS?
>
> After upgrading to macOS 10.13, a couple of packages stopped compiling
> with clang 3.9.1. The errors came from system headers. It looked like
> Apple had again used some funky new clang feature in them which 3.9.1
> didn't support.

Yes, I noticed that too, see:
https://bugs.gentoo.org/632500

> (Disclamer: I left prefix untouched when updating. It's still running
> with CHOST=x86_64-apple-darwin16 and profile
> prefix/darwin/macos/10.12/x64.)
>
> Getting clang-5 up and running wasn't all that bad: I needed to clone
> the libcxx and libcxxabi ebuilds for version 5.0.1, adjust eclass
> cmake-utils and reintroduce some prefix provisions to clang and
> compiler-rt ebuilds that didn't make the move from the llvm ebuild.

Oh, I experienced failures to compile it on a 10.13 profile.

> The cmake problem is somewhat tricky to explain: A special prefix
> handling is supposed to force all cmake-based builds to link against the
> libraries in prefix. This is done by setting an rpath and forcing cmake
> to use that rpath whenever possible. Normally, cmake will link the
> binaries for the build directory with an rpath that prefers libraries in
> the build directory. This is disabled by setting
> CMAKE_BUILD_WITH_INSTALL_RPATH.
>
> For LLVM and clang builds this causes a problem: Newly compiled binaries
> immediately get linked with the forced rpath. But the build calls some
> of them while still in the build directory and before installation of
> their libraries (especially llvm-tblgen and clang-tblgen). So they pick
> up the old installed versions of the libraries via the rpath and croak
> on missing symbols. I guess, other packages would face the same problem,
> if they called dynamically linked binaries from the build directory -
> which is somewhat rare.
>
> Does anyone remember why CMAKE_BUILD_WITH_INSTALL_RPATH was necessary
> for cmake ebuilds?

I don't recall, seems like this indeed shouldn't be forced.

> Disabling this setting led to another problem: LLVM and clang have all
> their libs reference each other using @rpath. This is done relative to
> the executable they're linked into which gets an rpath of
> @executable_path/../lib for that purpose. This is fine as long as both
> LLVM and clang are built in one sitting. With the new concept of first
> building and installing LLVM and then building clang, the call to
> clang-tblgen in the build directory again croaks. But this time it is
> because dynamic LLVM libraries that get pulled in by the dynamic LLVM
> libraries it's linked against can't be found because they're not on its
> @rpath. So I had to force an *install* rpath of
> ${EPREFIX}/usr/lib/llvm/${SLOT}/lib.

I assume this is install_name_tool-ing.

> With this I got llvm, clang and their other components installed. clang
> defaulted to /usr/include for includes at first and compiler-rt compiled
> against the Xcode SDK. With some research in the 3.9.1 ebuilds that was
> quickly remedied.
>
> The only real fallout came from the fact that my binutils-apple had LTO
> and tapi support enabled. Both broke horribly which rendered ld64
> defunct.
>
> As long as neither USE tapi nor lto are enabled on binutils-apple, they
> should be fine. Symlinks to size and nm will break and will need
> adjustment to point to the new llvm install though.

Hmm...

> I think I got LTO support sorted in an updated binutils-apple-8.2.1
> ebuild. I have also taken care of LLVMs move into a slotted
> $EPREFIX/usr/lib/llvm which should make them more robust to future
> updates and moves.
>
> But tapi is a real mess which shines a spotlight on a more fundamental
> problem: Apple has yet to release the source of Xcode 9. So tapi-1.30
> won't compile with clang-5.0.1 any more. A tapi-2.0.0 open source drop
> on GitHub (https://github.com/ributzka/tapi) requires Apple-specific
> additions to LLVM (ObjCMetadata). Those from the last released Apple
> clang-8.0.0 are slightly incompatible to llvm-5.0.0 *and* this
> tapi-2.0.0. I am currently digging my way through that.
>
> In case anyone wonders: tapi is the library that supports SDK .tbd stub
> files. So if ld64 has no tapi support it can't use recent Apple SDKs,
> which for the purposes of a prefix install is no big deal, I guess. It
> might even serve as a flag that some package is trying to use an SDK.
>
> Any comments on my approach would be highly appreciated. Patches
> attached for documentation.

Seems like you got yourself pretty deep into it!  clang-5 is the blocker
for supporting high sierra IMO.  Can you now build 5 using 3.9.1?

I think the cmake patches are ok, I'll look into the ebuilds you
mention.  I'll sync whatever I can find in this area.

Thanks,
Fabian


> --- portage/eclass/cmake-utils.eclass.orig	2017-12-25 22:56:39.589527684 +0100
> +++ portage/eclass/cmake-utils.eclass	2017-12-25 23:09:19.207489713 +0100
> @@ -588,9 +588,25 @@
>  			SET(CMAKE_PREFIX_PATH "${EPREFIX}/usr" CACHE STRING "" FORCE)
>  			SET(CMAKE_SKIP_BUILD_RPATH OFF CACHE BOOL "" FORCE)
>  			SET(CMAKE_SKIP_RPATH OFF CACHE BOOL "" FORCE)
> +			_EOF_
> +
> +			case ${PN} in
> +				llvm|clang)
> +					cat >> "${build_rules}" <<- _EOF_ || die
> +					SET(CMAKE_INSTALL_RPATH "${EPREFIX}/usr/lib/llvm/${SLOT}/$(get_libdir)" CACHE STRING "" FORCE)
> +					_EOF_
> +					;;
> +
> +				*)
> +					cat >> "${build_rules}" <<- _EOF_ || die
>  			SET(CMAKE_BUILD_WITH_INSTALL_RPATH TRUE CACHE BOOL "")
>  			SET(CMAKE_INSTALL_RPATH "${EPREFIX}/usr/lib;${EPREFIX}/usr/${CHOST}/lib/gcc;${EPREFIX}/usr/${CHOST}/lib;${EPREFIX}/usr/$(get_libdir);${EPREFIX}/$(get_libdir)" CACHE STRING "" FORCE)
>  			SET(CMAKE_INSTALL_RPATH_USE_LINK_PATH TRUE CACHE BOOL "" FORCE)
> +					_EOF_
> +					;;
> +		esac
> +
> +		cat >> "${build_rules}" <<- _EOF_ || die
>  			SET(CMAKE_INSTALL_NAME_DIR "${EPREFIX}/usr/lib" CACHE STRING "" FORCE)
>
>  			ENDIF (NOT APPLE)

> --- portage/sys-devel/clang/clang-5.0.1.ebuild	2017-12-20 23:24:32.000000000 +0100
> +++ portage/sys-devel/clang/clang-5.0.1.ebuild	2017-12-23 21:38:22.000000000 +0100
> @@ -110,6 +110,13 @@
>  	fi
>  }
>
> +src_prepare() {
> +	default
> +
> +	eapply "${FILESDIR}"/${PN}-5.0.1-darwin_prefix-include-paths.patch
> +	eprefixify lib/Frontend/InitHeaderSearch.cpp
> +}
> +
>  multilib_src_configure() {
>  	local llvm_version=$(llvm-config --version) || die
>  	local clang_version=$(ver_cut 1-3 "${llvm_version}")

> --- a/lib/Frontend/InitHeaderSearch.cpp
> +++ b/lib/Frontend/InitHeaderSearch.cpp
> @@ -233,6 +233,7 @@
>      case llvm::Triple::Bitrig:
>        break;
>      default:
> +      AddPath("@GENTOO_PORTAGE_EPREFIX@/usr/include", System, false);
>        // FIXME: temporary hack: hard-coded paths.
>        AddPath("/usr/local/include", System, false);
>        break;
> @@ -505,6 +506,7 @@
>    // Add the default framework include paths on Darwin.
>    if (HSOpts.UseStandardSystemIncludes) {
>      if (triple.isOSDarwin()) {
> +      AddPath("@GENTOO_PORTAGE_EPREFIX@/Frameworks", System, true);
>        AddPath("/System/Library/Frameworks", System, true);
>        AddPath("/Library/Frameworks", System, true);
>      }

> --- portage/sys-libs/compiler-rt/compiler-rt-5.0.1.ebuild	2017-12-20 21:40:08.000000000 +0100
> +++ portage/sys-libs/compiler-rt/compiler-rt-5.0.1.ebuild	2017-12-23 21:56:33.000000000 +0100
> @@ -77,6 +77,13 @@
>  		-DCOMPILER_RT_BUILD_XRAY=OFF
>  	)
>
> +	if use prefix && [[ "${CHOST}" == *-darwin* ]] ; then
> +		mycmakeargs+=(
> +			# disable use of SDK for the system itself
> +			-DDARWIN_macosx_CACHED_SYSROOT=/
> +		)
> +	fi
> +
>  	if use test; then
>  		mycmakeargs+=(
>  			-DLIT_COMMAND="${EPREFIX}/usr/bin/lit"


-- 
Fabian Groffen
Gentoo on a different level