Re: [gentoo-dev] Re: [RFC] multilib-build.eclass and restricting unsupported ABIs

[Alexis Ballier]

On Mon, 4 Mar 2013 21:49:48 +0100
Michał Górny <mgorny@g.o> wrote:

> On Mon, 4 Mar 2013 11:02:40 +0100
> Alexis Ballier <aballier@g.o> wrote:
>
> > On Sun, 3 Mar 2013 23:25:03 +0100
> > Michał Górny <mgorny@g.o> wrote:
> >
> > > On Sun, 3 Mar 2013 18:18:12 +0100
> > > Alexis Ballier <aballier@g.o> wrote:
> > >
> > > > On Sun, 3 Mar 2013 17:58:26 +0100
> > > > Michał Górny <mgorny@g.o> wrote:
> > > >
> > > > > What do we need that wrapper for? What does the wrapper do?
> > > > > Does it just rely on custom 'ABI' variable?
> > > >
> > > > yes -- it must perform some checks though.
> > >
> > > What kind of checks?
> >
> > you are called with ABI=sth argv[0] = your name
>
> I'm afraid that's the first potential point of failure. Relying
> on argv[0] is a poor idea and means that any application calling
> exec() with changed argv[0] on a wrapped binary will fail terribly.

Go tell that to those that wrote busybox, tex or git: They all use
that argv trick :) I'm sure there are plenty of other examples and
don't consider that a poor idea. Note that the wrapper does not lie on
argv: it just makes the binary think it has its original name (by
passing argv as is) while it has been moved to allow parallel
installation of different flavors of it.

> > if argv[0] = abiwrapper -> print some information and exit
> > getenv ABI -> if nothing then set ABI=$DEFAULT_ABI (hardcoded at
> > buildtime of the wrapper)
> > execvp(argv[0]_$ABI, argv)
> > if execvp returns: print a warning, execvp argv[0]_$DEFAULT_ABI
> >
> >
> > python-wrapper.c is very likely to have such a logic already.
> >
> > btw, IMHO ABI is a too common name for such a variable, I'd better
> > name it something like _GENTOO_MULTILIB_ABI so that collisions are
> > much less likely.
>
> I'm afraid you are actually starting to go the other way.
>
> Global wrapper means that it is potentially useful to our users.
> However I don't really see people who want to compile 32-bit
> executables think of setting some custom variable like ABI.

They will not: Users will call the name-suffixed version directly. If
they call the non-suffixed version then the above logic will run the
DEFAULT_ABI version. There's no variable to set.
The variable is more so that ebuilds/eclasses can set the variable and
be done. Exactly like EPYTHON I think.


[...]
> > That's why I asked for examples :)
> > qmake may do it, I don't think its sane, but that's life for now.
> > having glxinfo for each abi is useful from a user perspective (it
> > does not need the wrapper a priori though)
>
> Yep, I intended to just have the additional variant of glxinfo
> directly callable, with a name chosen specifically by the X11 team.
> Wrapper would be more confusing than beneficial here IMO.

Having a wrapper or not, for the end user that does not know the
internal variable name, the end-result will be exactly the same :)

[...]
> > See it something like python-wrapper. EPYTHON=python3.2 python ->
> > runs python3.2 :)
>
> Err, python-wrapper respects quite complex logic involving EPYTHON,
> and eselect-python. We don't really want people to have eselect-abi,
> do we?

No we don't. The wrapper has nothing to do with that.

[...]
> > To some extent that's what happened to python too :) As a python
> > maintainer, you could share your thoughts on the topic. python
> > slotting was intended to make switching between python versions
> > easy but has been needing wrappers for the python binary.
>
> I'm doing just that. Any kind of wrapping is an increasing mess. I'm
> still trying to find out good solutions for Python wrapping but
> there's no such thing. It's always about choosing one evil over the
> other.

I don't understand here. What is python-wrapper good for if that's not
to be able to chose which python to run within ebuilds? If that's just
for eselecting the active version, a symlink is likely to be enough.
python-wrapper is an elegant idea for not having to invent a new way
and change everything else: Within an ebuild, setting EPYTHON lets you
be certain that everything will be run by the version you want. You do
not have to mess with packages calling 'python' directly.

[...]
> > packages
> > calling directly binaries having ABI specific output will be broken
> > for multilib too (and I don't know of anyone checking for this
> > while the other two have been long standing issues we tried to
> > fix). We can fix this, but the fact is that we need multi-binary
> > support for users, then the only choice to make is if we want to
> > provide a wrapper so that we do not need to fix build systems or if
> > we want to fix them. The latter is likely preferred but I do not
> > know what kind of work it will involve. It'd help if tommy could
> > provide a list of binaries he needed to wrap through the abiwrapper.
>
> We don't want it for users in an automagic and fragile way. And
> ebuilds we can fix while migrating.

I think you misunderstood the idea of the wrapper: It seems more robust
than anything else.

Consider the following scenario: 
The binary foo transforms a .foo file into a .h file. .foo files define
types foo32 and foo64. The foo utility, while processing its .foo
files, #define them to, respectively, long and long long on x86
and int and long on amd64. Its output depends on the ABI, so we install
foo and foo_x86 for an amd64/x86 multilib system.

Now I have a package that uses .foo files: it installs several
libraries and binaries and seems to accept FOO='name of the foo utility'
variables. Good, so we set FOO=foo_${ABI} or FOO=foo depending on what
we are building for. All seems good. But one of all its libraries
has an hardcoded foo call. Your package will build fine but x86
binaries using this library will except the foo_x86 types of abi,
however this library was built with foo_amd64 types: the foo64
types will actually be 32bits for this sole library. Calling a
function with a foo64 argument from this library will corrupt the stack
and you will get weird crashes, usually very hard to debug.


This is an extreme and stupid example, but without any wrapper and with
ABI-specific output used within ebuilds and build systems, you have to
be *very* careful that the correct tool is *always* called.
The wrapper is only a way to ensure that after all :)


Alexis.