Re: [gentoo-dev] Support for multiple ABIs for amd64 (64bit,32bit) in multilib overlay

[Thomas Sachau]

Mike Frysinger schrieb:
> On Sunday 18 October 2009 14:46:07 Thomas Sachau wrote:
>> Mike Frysinger schrieb:
>>> another quick look at _setup_abi_env() looks like it needs work:
>>>  - LD should not default to `ld`
>> Whats your suggestion?
>
> the majority of the time, the compiler driver (i.e. `gcc`) should be used for
> linking.  very few packages should invoke the linker directly.  that is why
> currently the toolchain-func.eclass has tc-getLD return `ld` -- a few packages
> need it, but not most.  if we're going to be forcing the setting of the LD env
> var all the time, then it needs to default to ${CC}.  packages that need funky
> behavior should still work as they will be calling $(tc-getLD) anyways.

Dropped $LD for now, seems to work fine for me.

>
>>>  - the -L paths to system dirs in LDFLAGS should not be there -- the
>>> toolchain can handle these just fine
>> Last time i tried without, some packages failed to compile, will test it
>>  again to check, if its still needed
>
> if things are failing, then we should look at gcc/binutils to make sure they
> use the right default search paths when given -m32/-m64

I currently dont export any explicit LDFLAGS, but instead only save the value at the end of each
phase, seems to work fine for me.

>
>>> how do you control whether the multilib headers are needed ?  it'll
>>> probably make sense in general, but there are def some packages where
>>> this will only get in the way (the toolchain ones).
>> What do you mean here? If the diff between ABIs makes sense to install
>>  seperate versions?
>
> some packages (like glibc and linux-headers) already handle the multilib
> situation.  forcing the unnecessary Gentoo layer into the stack can easily
> break things.

For glibc, it is avoided since it has the "multilib" useflag. If this is enabled, it indicates for
me that the package does handle everything for itself, so multilib-portage does handle it as if it
would be a normal package without multi-ABI request.
Since linux-headers do also some special multilib handling, could you also add a "multilib" useflag
for them? Currently i have an exception for them in my code to prevent problems for other packages.

I assume that ever package not having a multilib useflag does not any multilib-specific action. To
check, if the header files differ per ABI, i save them for both ABIs, then diff them and create
ABI-specific header files with a wrapper for all header files, that differ. The rest is just
installed as usual.

>>> how do you differentiate between packages where multi ABIs make no sense
>>> ? for example, most packages that dont install any libraries but just
>>> binaries. let's use the simple package app-text/tree.
>> I dont differentiate. Currently i build for every ABI, if lib32 useflag is
>>  set and multilib useflag is not set. The idea behind it is to allow the
>>  installation of additional 32bit binaries, if requested.
>
> that's is an immense waste of time.  if we ran all the source phases for a
> single ABI in one go, it should be easy to dynamically detect when a multilib
> multipass is necessary (by looking at library paths in $D).  and for the odd
> package out, create a hook of some sort (EMULTIABI=true) to force behavior.
>
> i dont think there is any inherit reliance on running the multilib pass on
> each src step every time (other than that was easiest to implement) ?

For packages with header files, i need to run all phases for both ABIs to check, if the header files
are ABI specific.
So it would require a check for installed libs and installed header files. And its more work since
it requires both changes to the ebuild and emerge command.

>
>>> a lot of this multilib code should probably continue to live in the tree
>>> as it's a pretty big base of code to formalize that could do with fixes
>>> over time.  i.e. we figure out that certain paths / define protections
>>> dont work so well, so changing them to something else would require PMS
>>> changing !?  there has been talk before about pushing a lot of basic
>>> stuff to the tree so things dont have to be encoded in the PMS.
>> How do you want to do this? Require package managers to inherit some
>>  file/eclass?
>
> considering this requires changes to the PM already, i dont see why not.  and
> it wouldnt really be an inherit in the current sense of the word.  more like a
> simple source.

I am ok with this suggestion.

>
>>> how are packages handled that can only be used via 1 ABI ?  any of the
>>> packages listed in the amd64 no-multilib package.mask for example.  while
>>> these are mostly binary-only, this isnt a binary-specific issue.  there
>>> are a number of packages which only work on x86/ppc but could easily work
>>> in a multilib amd64/ppc64 as a 32bit binary (source code sucks, is
>>> heavily asm, something else).
>> The binary-only ones are easy. Since they dont interact with the env, they
>>  can be installed as usual. If they depend on a new enough package manager
>>  with multilib support, they could also depend on the useflag for the
>>  additional 32bit libs they need.
>
> if it's a binary package, we know the ABI of it ahead of time.  so if the
> package declared the binary ABI that it uses, then portage could handle the
> rest (making sure the deps are resolved against the ABI that it requires).  we
> dont want to rewrite every binary ebuild to include an explicit [foo] ABI flag
> on each of its deps.

This would require additional vars for multilib handling, which would require inclusion in EAPI-3 or
in a future EAPI, if the current process with EAPIs is continued.

With the current implementation, i try to stay EAPI-independent, so the changes can be implemented
without having to wait for aproval of another EAPI.

>
>>>> 2. Adding the internal lib32 useflag and usedeps with some workarounds
>>> what exactly does this "lib32" do ?  naming USE flags according to
>>> specific ABI implementations is a bad idea.  you have to forget special
>>> casing anything to "lib32 vs lib64".  amd64, while the most common, is
>>> hardly extensible.  we must handle multiple ABIs which easily might have
>>> the same bitsize.
>> "lib32" is added to IUSE, you can enable as as every other useflag.
>>  Internally, portage does add [lib32?] usedeps to all dependencies. So if
>>  you enable the lib32 useflag, portage will require this useflag for all
>>  dependencies too. I dont mind renaming it, if there is some other sane
>>  naming for it.
>
> i think every package will need tagging for each ABI, not just relative to the
> default one.  so the profile on an amd64 multilib would declare that it wants
> both x86 and x86_64 binaries by default.  in the normal case, the PM can then
> automatically resolve all dependencies as requiring all ABIs.  if a binary
> package is emerged, the ebuild itself declares the ABIs it provides, and then
> the PM only resolves the dependencies for the ABIs the package provides.

In general, this looks ok. As said above, i just want to prevent the multilib-implementation from
being delayed until another EAPI is created, aproved and implemented.

-- 
Thomas Sachau

Gentoo Linux Developer