Gentoo Archives: gentoo-alt

From: Perry Smith <pedzsan@×××××.com>
To: gentoo-alt@l.g.o
Subject: Re: [gentoo-alt] AIX linking adventure
Date: Thu, 10 Feb 2011 14:15:49
Message-Id: 5E62A3DE-AC26-4E02-B92D-431C8B3EE154@gmail.com
In Reply to: Re: [gentoo-alt] AIX linking adventure by Michael Haubenwallner
On Feb 10, 2011, at 3:06 AM, Michael Haubenwallner wrote:

> > On 02/10/11 01:57, Perry Smith wrote: >>> On 02/09/11 17:47, Perry Smith wrote: >>>> Lets start over. I don't think I'm making any progress but I'll give it one last go. >>>> ***Import File Strings*** >>>> INDEX PATH BASE MEMBER >>>> 0 /usr/local/rvm/rubies/ruby-1.9.1-p378/lib:/usr/local/lib/gcc/powerpc-ibm-aix5.3.0.0/4.3.1:/usr/local/lib/gcc/powerpc-ibm-aix5.3.0.0/4.3.1/../../..:/usr/lib:/lib >>>> 1 librtl.a shr.o >>>> 2 /usr/lib libc.a shr.o >>>> 3 /usr/lib libpthread.a shr_comm.o >>>> 4 /usr/lib libpthread.a shr_xpg5.o >>>> 5 /usr/local/rvm/rubies/ruby-1.9.1-p378/lib libruby.so >>> Is this with our without your intervention in ruby's build process and/or toolchain setup? >> >> This is after my toolchain changes. > > How does this look like without your changes?
I can't recall 100%. I can go back and do another build. In particular, I can't recall who had the problem if it was ruby itself or libruby.so that had the problem. In general, there were no "hardcoded" elements and the 0th element was simply: /usr/local/rvm/rubies/ruby-1.9.1-p378/lib:/usr/lib:/lib Let me know if that is sufficient. If not, I can do another build without my mod's.
> >> The 1st entry is what I was calling "relative" but as you point out, that isn't the right term. >> "affected by run path search" is too long for me. :-) >> >> The 2nd through 5th items are those I was calling "absolute" but really they should be >> called "not affected by run path search". Perhaps "fixed" in this case would work? >> non-LIBPATH-able? > > IIRC, "hardcoded" is what libtool uses for that kind of paths, > but I'm fine with "fixed" too.
Lets use existing terms if they exist. hardcoded isn't too long. what is the opposite? non-hardcoded?
> >>> How exactly did you touch either the build process or the toolchain setup to "allow" that? >> >> I put my ld in /usr/local/aixbin/ld and I put /usr/local/aixbin as the first element of PATH. >> So my ld gets called instead of the real ld. My ld monkeys with things. > > Which compiler did you use - gcc or xlc?
gcc 4.3.1
> Does xlc search for 'ld' using PATH?
Good question. I don't know.
> Usually, gcc does not, you'll have to configure it to use "/usr/local/aixbin/ld".
collect2 (from gcc) calls ld. I'm using my "stock" gcc build and its working for me.
> >>>> Lets pause here and see if we can come to an agreement on these two items. >>>> 1) objets have only absolute paths after they are installed >>> >>> Still not clear to me: >>> Do you actually /prefer/ hardcoded paths (without runpath search)? Why? >>> >>> Although runpath search is affected by LIBPATH, I'm just fine with that. >>> This is the outcome of using "-L" and "-l" linker flags together. >> >> I thought this was one of your objectives based on the fact that java sets LIBPATH >> to something weird and it breaks everything. > > Nope. The problem with LIBPATH set by java was that there is nothing > like "soname". Iff my binary had searched for the /file/ "libiconv.so.2" > (either archive or not) instead of "libiconv.a(libiconv.so.2)", even with > LIBPATH set to "/usr/lib" it would not have found "/usr/lib/libiconv.a" > (without "libiconv.so.2" member), but continued along LIBPATH + runpath > until it found the "libiconv.so.2" file in its usual place.
Hmmm... ok. I had forgotten that detail.
> >>> Seems we have to agree on some terms to be used first. >> >> Yes. >> >> I like "fixed" for entries not affected by runtime search. > > "fixed" = "hardcoded" (see above). > >> The other tact would be to note that PATH is either empty or not. >> >> "A path dependency with no path" or "a dependency with a path" ? > > Which "PATH" ("path") do you refer to here that can be either empty or not?
In the dump -H output above, there is a column called PATH. That PATH. I apologize for being so consistently vague.
> There is the "PATH" environment variable, which does not affect runpath search at all. > There is the "LIBPATH" environment variable, which does. > There is the encoded "hardcoded path" (="fixed") for one library, not allowing runpath search. > There is the encoded "runpath", used (after LIBPATH) for any library without a "hardcoded path". > >> "binary" instead of "object" sounds better to me. > > ok. > >>> What else to say: >>> When libtool knows for a platform that linking libraries results in hardcoded >>> paths, or there is no such thing like LIBPATH environment variable to temporarily >>> override the runpath, it actually /does/ "relink" the binaries during installation. >> >> I understood that part -- but >> >>> OTOH, when the /absolute/path/to/the/library.so found at linktime is hardcoded into >>> the resulting binary, installing into DESTDIR is broken - which is a prerequisite >>> for package managers like portage to work properly. >> >> Didn't understand this part. In particular, from the comma to the end ", installing ..." > > Ohw, ok - this is the core requirement for proper package managing: > > Consider these raw build steps for an usual package, which does provide > "${prefix}/lib/library.so", and "${prefix}/bin/mybinary" linked against it: > > 1. Configure: > $ ./configure --prefix="/final" > 2. Compile: > $ make > 3. Install (to allow for a binary package): > $ make install DESTDIR="/image" > The important thing here is to /not/ copy anything to "/final" at the > build machine, but still the binaries copied to "/image/final" have > to be able to run when moved to "/final" by the package manager, > eventually on another machine with (ideally) identical setup.
Does the package (need to) work from the /image area or is this just a temporary place used to create the tar file? In particular, what about paths hard coded into the binaries? (Not the paths in the binary's header but just paths hard coded in the code like where to find my lisp files, etc. I know that *usually* there are options to change them but there is still some default set at compile time.
> 4. Create the binary package: > $ cd "/image" > $ find "final" > "filelist-this-package-does-install" > $ tar cfj "binpackage.tar.bz2" "filelist-this-package-does-install" "final" > 5. optionally ship the binary package to another host with (ideally) identical setup. > 6. See if the binary package would fit: > $ tar xfj "binpackage.tar.bz2" "filelist-this-package-does-install" > $ for f in $(<"filelist-this-package-does-install") > do > if [[ -e /${f} ]] > then > do-we-update "${f}" && continue > echo "Installing this package conflicts with existing '/${f}'" > fi > done > 7. Merge the binary package to the live filesystem: > $ cd "/" > $ tar xfj "binpackage.tar.bz2" "final" > 8. Use the package: > $ "/final/bin/mybinary" > > During step 3, the shared library gets installed as "/staging/final/lib/library.so". > Relinking "mybinary", the linker has to read that file, while encoding "/final/lib" > (without "/staging") into "mybinary" as the search path for "library.so".
Shoot... you lost me. I think maybe /staging is the same as /image above?
> > Usually that is done this way, encoding the final "runpath": > $ ld -o "/staging/final/bin/mybinary" -L "/staging/final/lib" -lrary -blibpath:"/final/lib" > > Because this won't work as expected, as the "hardcoded" path would contain "/staging": > $ ld -o "/staging/final/bin/mybinary" "/staging/final/lib/library.so" > > Except... when an "Import File" is found at "library.so", referring to "#! /final/lib/...". > > However, when "library.so" is a /standalone/ Import File, > it has to refer to something like "#! /final/lib/library.so.0", > so the loader actually can load a shared object rather than an Import File. > > When "library.so" is an archive file /containing/ both the Import File "shr.imp" and the shared object "shr.o", > the Import File has to refer to "#! /final/lib/library.so(shr.o)". > > For completion, "library.so.0" is called the "soname" in ELF world, > but that's a different (although related) topic.
Ok. I have more thoughts on this too. Lets not go there just yet. I'm still lagging just a little. One question: with each of your experiments, do you dump out the binary's "header" with dump -H? The reason I ask is I would expect linking with an import file basically just put that piece, the #!<whatever> into the binary's header at link time and then what happens at run time is the same as if you had linked against a library. I remember toying with the import file inside the archive (and getting frustrated at the lack of documentation) but I can't remember what I found.

Replies

Subject Author
Re: [gentoo-alt] AIX linking adventure Michael Haubenwallner <haubi@g.o>