Re: [gentoo-alt] AIX linking adventure

[Michael Haubenwallner]

On 02/10/11 01:57, Perry Smith wrote:
>> On 02/09/11 17:47, Perry Smith wrote:
>>> Lets start over.  I don't think I'm making any progress but I'll give it one last go.
>>>                        ***Import File Strings***
>>> INDEX  PATH                          BASE                MEMBER
>>> 0      /usr/local/rvm/rubies/ruby-1.9.1-p378/lib:/usr/local/lib/gcc/powerpc-ibm-aix5.3.0.0/4.3.1:/usr/local/lib/gcc/powerpc-ibm-aix5.3.0.0/4.3.1/../../..:/usr/lib:/lib
>>> 1                                    librtl.a            shr.o
>>> 2      /usr/lib                      libc.a              shr.o
>>> 3      /usr/lib                      libpthread.a        shr_comm.o
>>> 4      /usr/lib                      libpthread.a        shr_xpg5.o
>>> 5      /usr/local/rvm/rubies/ruby-1.9.1-p378/lib libruby.so
>> Is this with our without your intervention in ruby's build process and/or toolchain setup?
>
> This is after my toolchain changes.

How does this look like without your changes?

> The 1st entry is what I was calling "relative" but as you point out, that isn't the right term.
> "affected by run path search" is too long for me. :-)
>
> The 2nd through 5th items are those I was calling "absolute" but really they should be
> called "not affected by run path search".  Perhaps "fixed" in this case would work?
> non-LIBPATH-able?

IIRC, "hardcoded" is what libtool uses for that kind of paths,
but I'm fine with "fixed" too.

>> How exactly did you touch either the build process or the toolchain setup to "allow" that?
>
> I put my ld in /usr/local/aixbin/ld and I put /usr/local/aixbin as the first element of PATH.
> So my ld gets called instead of the real ld.  My ld monkeys with things.

Which compiler did you use - gcc or xlc?
Does xlc search for 'ld' using PATH?
Usually, gcc does not, you'll have to configure it to use "/usr/local/aixbin/ld".

>>> Lets pause here and see if we can come to an agreement on these two items.
>>> 1) objets have only absolute paths after they are installed
>>
>> Still not clear to me:
>> Do you actually /prefer/ hardcoded paths (without runpath search)? Why?
>>
>> Although runpath search is affected by LIBPATH, I'm just fine with that.
>> This is the outcome of using "-L" and "-l" linker flags together.
>
> I thought this was one of your objectives based on the fact that java sets LIBPATH
> to something weird and it breaks everything.

Nope. The problem with LIBPATH set by java was that there is nothing
like "soname". Iff my binary had searched for the /file/ "libiconv.so.2"
(either archive or not) instead of "libiconv.a(libiconv.so.2)", even with
LIBPATH set to "/usr/lib" it would not have found "/usr/lib/libiconv.a"
(without "libiconv.so.2" member), but continued along LIBPATH + runpath
until it found the "libiconv.so.2" file in its usual place.

>> Seems we have to agree on some terms to be used first.
>
> Yes.
>
> I like "fixed" for entries not affected by runtime search.

"fixed" = "hardcoded" (see above).

> The other tact would be to note that PATH is either empty or not.
>
> "A path dependency with no path" or "a dependency with a path" ?

Which "PATH" ("path") do you refer to here that can be either empty or not?
There is the "PATH" environment variable, which does not affect runpath search at all.
There is the "LIBPATH" environment variable, which does.
There is the encoded "hardcoded path" (="fixed") for one library, not allowing runpath search.
There is the encoded "runpath", used (after LIBPATH) for any library without a "hardcoded path".

> "binary" instead of "object" sounds better to me.

ok.

>> What else to say:
>> When libtool knows for a platform that linking libraries results in hardcoded
>> paths, or there is no such thing like LIBPATH environment variable to temporarily
>> override the runpath, it actually /does/ "relink" the binaries during installation.
>
> I understood that part -- but
>
>> OTOH, when the /absolute/path/to/the/library.so found at linktime is hardcoded into
>> the resulting binary, installing into DESTDIR is broken - which is a prerequisite
>> for package managers like portage to work properly.
>
> Didn't understand this part.  In particular, from the comma to the end ", installing ..."

Ohw, ok - this is the core requirement for proper package managing:

Consider these raw build steps for an usual package, which does provide
"${prefix}/lib/library.so", and "${prefix}/bin/mybinary" linked against it:

1. Configure:
   $ ./configure --prefix="/final"
2. Compile:
   $ make
3. Install (to allow for a binary package):
   $ make install DESTDIR="/image"
   The important thing here is to /not/ copy anything to "/final" at the
   build machine, but still the binaries copied to "/image/final" have
   to be able to run when moved to "/final" by the package manager,
   eventually on another machine with (ideally) identical setup.
4. Create the binary package:
   $ cd "/image"
   $ find "final" > "filelist-this-package-does-install"
   $ tar cfj "binpackage.tar.bz2" "filelist-this-package-does-install" "final"
5. optionally ship the binary package to another host with (ideally) identical setup.
6. See if the binary package would fit:
   $ tar xfj "binpackage.tar.bz2" "filelist-this-package-does-install"
   $ for f in $(<"filelist-this-package-does-install")
     do
       if [[ -e /${f} ]]
       then
         do-we-update "${f}" && continue
         echo "Installing this package conflicts with existing '/${f}'"
       fi
     done
7. Merge the binary package to the live filesystem:
   $ cd "/"
   $ tar xfj "binpackage.tar.bz2" "final"
8. Use the package:
   $ "/final/bin/mybinary"

During step 3, the shared library gets installed as "/staging/final/lib/library.so".
Relinking "mybinary", the linker has to read that file, while encoding "/final/lib"
(without "/staging") into "mybinary" as the search path for "library.so".

Usually that is done this way, encoding the final "runpath":
$ ld -o "/staging/final/bin/mybinary" -L "/staging/final/lib" -lrary -blibpath:"/final/lib"

Because this won't work as expected, as the "hardcoded" path would contain "/staging":
$ ld -o "/staging/final/bin/mybinary" "/staging/final/lib/library.so"

Except... when an "Import File" is found at "library.so", referring to "#! /final/lib/...".

However, when "library.so" is a /standalone/ Import File,
it has to refer to something like "#! /final/lib/library.so.0",
so the loader actually can load a shared object rather than an Import File.

When "library.so" is an archive file /containing/ both the Import File "shr.imp" and the shared object "shr.o",
the Import File has to refer to "#! /final/lib/library.so(shr.o)".

For completion, "library.so.0" is called the "soname" in ELF world,
but that's a different (although related) topic.

/haubi/
-- 
Michael Haubenwallner
Gentoo on a different level