Re: [gentoo-dev] Support for multiple ABIs for amd64 (64bit,32bit) in multilib overlay

[Mike Frysinger]

On Monday 12 October 2009 15:49:48 Thomas Sachau wrote:
> Mike Frysinger schrieb:
> > On Sunday 16 August 2009 08:37:37 Thomas Sachau wrote:
> >> -for the portage version: It is also in the multilib overlay, but in a
> >>  different branch called portage-multilib. To use this, you should read
> >> the instructions at [1] (doc/portage-multilib-instructions). This one
> >> should also mainly work, but there is probably a good amount of packages
> >> in the main tree, which may refuse to work with it.
> >
> > the abi-wrapper doesnt look terribly appealing.  why dont we use
> > broken/custom -config handling as incentive to convert packages to .pc
> > files.  pkg-config handles ABI/cross-compile splitting cleanly and
> > transparently.
>
> I am totally unfamiliar with pkg-config, so that would take some time or a
>  helping hand.

going by the pkg-config wrapper we use for cross-compiling, the multilib code 
should unset:
	PKG_CONFIG_ALLOW_SYSTEM_CFLAGS
	PKG_CONFIG_ALLOW_SYSTEM_LIBS

then it should export export PKG_CONFIG_LIBDIR to the ABI-specific pkgconfig 
dir.  so it should be something akin to:
	export PKG_CONFIG_LIBDIR=/usr/$(get_libdir)/pkgconfig

any package that adds things to PKG_CONFIG_PATH (grep /etc/env.d/) will be a 
problem here, but it seems only qt-3 currently falls into this category.  the 
easiest thing will probably be to just disallow this behavior completely.

then pkg-config should automatically locate the right abi-specific .pc files 
and filter the related abi-specific flags.

> > bash-4 is stable, so we could start depending on it.
>
> It still has 3 unstable KEYWORDS including mips.

mips doesnt count -- it's moving all to ~arch, and you've counted the bsd 
arches which are never stable (currently).

> > you dont save/restore CPPFLAGS
>
> Are there any initial values it should get?

the current profiles dont set any, but it should be treated as a precious 
toolchain variable same as any other compiler flag

another quick look at _setup_abi_env() looks like it needs work:
 - LD should not default to `ld`
 - the -L paths to system dirs in LDFLAGS should not be there -- the toolchain 
can handle these just fine
 - missing CPPFLAGS handling
 - see previous comments re-pkg-config
 - you dont declare pyver local
 - the abi if check is uncommon syntax.  "! [[ a == b ]]" -> "[[ a != b ]]"

> > is there documentation that covers the proposed changes/design to portage
> > ? i'm not looking for high level (it runs src_compile twice).  i dont
> > recall seeing details posted to the portage or gentoo mailing lists ...
> > it's hard to review `git diff portage-svn..master`.
>
> There is currently no documentation for the actual code version.

what about documentation for ebuild writers ?  only thing i see is how to 
install the new version of portage.

> In short, there are those changes:
>
> 1. ability to run phases multiple times in ebuild.sh:
>
> change from
> ...
> dyn_unpack()
>     src_unpack
>
> dyn_compile()
>     src_compile
>
> dyn_install()
>     src_install
> ...
>
> to basicly
>
> ...
> dyn_unpack()
>     for each ABI
>         call set_abi
>             create ABI workdir
>             set ABI envronment variables
>         call src_unpack
>
> dyn_compile()
>     for each ABI
>         call set_abi
>             use workdir for current ABI
>             set ABI environment variables
>         call src_compile
>
> dyn_install()
>     for each ABI
>         call set_abi
>             use workdir for current ABI
>             set ABI environment variables
>         call src_install
>         call _finalize_abi_install
>             create gentoo-multilib headers (if needed)
> ...

so basically what we already do in like glibc and sandbox.  at the base level, 
should be fine since it's been working so far with these few ebuilds.

how do you control whether the multilib headers are needed ?  it'll probably 
make sense in general, but there are def some packages where this will only 
get in the way (the toolchain ones).

how do you differentiate between packages where multi ABIs make no sense ?  
for example, most packages that dont install any libraries but just binaries.  
let's use the simple package app-text/tree.

a lot of this multilib code should probably continue to live in the tree as 
it's a pretty big base of code to formalize that could do with fixes over 
time.  i.e. we figure out that certain paths / define protections dont work so 
well, so changing them to something else would require PMS changing !?  there 
has been talk before about pushing a lot of basic stuff to the tree so things 
dont have to be encoded in the PMS.

how are packages handled that can only be used via 1 ABI ?  any of the 
packages listed in the amd64 no-multilib package.mask for example.  while 
these are mostly binary-only, this isnt a binary-specific issue.  there are a 
number of packages which only work on x86/ppc but could easily work in a 
multilib amd64/ppc64 as a 32bit binary (source code sucks, is heavily asm, 
something else).

> 2. Adding the internal lib32 useflag and usedeps with some workarounds

what exactly does this "lib32" do ?  naming USE flags according to specific 
ABI implementations is a bad idea.  you have to forget special casing anything 
to "lib32 vs lib64".  amd64, while the most common, is hardly extensible.  we 
must handle multiple ABIs which easily might have the same bitsize.
-mike