Gentoo Archives: gentoo-alt

From: Perry Smith <pedzsan@×××××.com>
To: gentoo-alt@l.g.o
Subject: Re: [gentoo-alt] AIX linking adventure
Date: Tue, 08 Feb 2011 18:36:35
In Reply to: Re: [gentoo-alt] AIX linking adventure by Michael Haubenwallner
On Feb 8, 2011, at 2:49 AM, Michael Haubenwallner wrote:

> Hi Perry! > > On 02/08/2011 01:35 AM, Perry Smith wrote: >> This is *not* ready for prime time but I thought I would mention it here. >> >> I wrote a front end to AIX's ld command as well as a relink command I'm calling "rl". > > Actually, in Prefix we already do wrap AIX' ld in sys-devel/binutils-config > to get the runpaths (the -blibpath argument) right. > > At the moment I'm trying another thing to get full "soname" support, > both with and without libtool, using Import Files. There is an RFC[1] > for the way I want to have shared libraries being created on AIX. > > However, I'm still unsure if we actually should use the '# autoload' thingy, > but ignore the auto-dependency problem[2] instead. > > [1] > [2]
I am not on the libtool list. I did not read that thread yet but I will here shortly. I need some "netiquette" help. Should I plop my thoughts / questions about that thread into this thread or start a new thread? I get confused either way :-) (Maybe I'm special that way.)
> > Additionally, for packages not using libtool, there's another ld wrapper in > sys-devel/native-cctools waiting for checkin here to support the '-soname' flag > known on ELF (Linux), creating the shared libraries according to above way[1]. > Eventually, this second wrapper would be merged into binutils-config's wrapper. > >> You can see them here: >> >> > > Commenting on 'Readme.markdown' found there: > Why do you move '-blibpath' arguments to '-L' arguments? > IMO, the other way round would make more sense actually. > Why do you need 'rl' (relink) at all?
For ruby, their libpath was a complete mistake. They did not include the path to gcc's library. They did not have a -L /path/to/where/they/would/install but they did have -blibpath:/path/to/where/they/would/install:/usr/lib:/lib and somehow thought that libpath was going to fix it. I guess it would have if it wasn't for the fact that the executable could not find gcc's library. And... the executable would never work in the directory where it was built in. It would work only after the package was installed (moved). I don't know how they were going to do the "test" or "check" phase which I'm sure Ruby has. I took their mistake as a symptom as a general practice. i.e. the general population somehow thinks that libpath substitutes for -L and it does not. I'm interested to hear of your experience but I can't think of a time where open source understood what libpath (inside the object) could/should be used for. The Ruby guys did have a fairly reasonable fear. They did not want to have just -L . and -L .., etc and not set the libpath because the -L . adds that to the internal libpath and that creates what they called a security risk. But, the problem is that the dot needs to be in the internal libpath while the executable is in the directory it was built in. The purpose for rt is after the install. The first link of ruby leaves the path to as relative (no path component). This allows ruby to execute in the directory that it was built in (before rt is run) as well as its final destination (after rt is run) because both dot and the final destination directory are in the internal libpath. This is going to be a general problem. You want the executables to work before being installed. Often there are check or test processes that use this feature. This implies that finding the libraries is going to need to be done via relative paths. But after ruby (the package) is installed, I wanted to fix the path to to the place that it was installed. I could not guess where it was going to be installed during the "build" phase (the first compile / link). I had to wait until it was installed and then use libpath (which I had to fix for Ruby via adding the -L) to find where finally got installed at. I also had a rather self imposed limitation to work around. I am trying to get "rvm" to work and rvm thinks it knows how to unpackage, configure, build, and install without any user intervention. To get that to work, I had to hook in at the ld point and then, after the fact, fix the resulting files after they are installed.
> >> I've used this only for Ruby -- and it was in fact Ruby that caused me to do this. >> Their out of the box system does not work at all for AIX if the prefix directory is not the same as GCC's. >> WIth this, I can now use rvm on AIX and do "rvm install 1.9.1" and it will complete successfully without my fingers leaving my hands. > > Haven't used/installed Ruby on any platform yet: What is 'rvm' ?
rvm is "Ruby Version Manager". rvm allows completely independent versions of Ruby (1.8.7, 1.9.1, MacRuby, etc). Each version can have its own independent sets of gems (called gemsets) so an application can use its own version of Ruby and its own set of gems. I can set up an environment on my development laptop and then fairly easily replicate this environment on my staging server and production server. It solves many problems I have when trying to have three or four Rails apps on the same host. (sorry for the long winded reply).
> >> I mentioned this idea on this list before but it didn't fly very well. >> I'm not sure if it was completely understood. > > I fail to find that post you're referring to. Has it been within one > of the threads we already created together, and I've overlooked it?
I *think* this is the post but it got munged: My archive is this (snipped out). The time stamp is Tue, Dec 14, 2010 at 6:25 PM (to the gentoo-alt list)
> Also, the guy I'm talking to mentioned rtl_enable -s. I've not used it but it seems like it might make life a lot easier. > > You can play with it some but if you point it at a shared object, it creates a script, an import file, and an export file. Together, they will recreate the shared object. You can then edit the files, for example, adding absolute paths, if you want to change the header of the shared object file. > > I am not familiar with portage at all but I've seen that it patches various things like the configure script, etc. It might be easier / safer to leave the scripts alone (leave libtool do whatever it wants) and then use rtl_enable -s, edit the resulting files, and rebuild the shared object the way you want it. >
> >> This is more of a proof of concept at this stage. I'm going to putter around and try and use it on other packages. > > For packages not aware of AIX at all: > How would this require their build systems to be patched? > >> "rl" could be enhanced to take "from" and "to" directories. >> If you wanted to change files installed in /some/old/directory and put them into /some/new/directory, rl could easily modified do that. > > In which context do you need this use case?
With putting absolute paths into the objects, we are effectively nullifying the possible benefit of the LIBPATH environment. I think we both agree that that benefit is questionable but others might like it. So if someone wants to move where I built a package from /usr/local to /opt, the first step would be to change the internal absolute paths to each dependency. There would be other obstacles too like any internal paths of the executables but often there are command line arguments to change those. e.g. you can give emacs directions on where to find its lisp files but you can not magically tell the loader where to find emacs' dependent libraries. A user once could via LIBPATH but that isn't going to work now with absolute internal paths. rl gives that possibly useful option back to the user.
> And how would you do that on Linux?
I don't know squat about Linux. Part of my frustration is everyone appears to want to make AIX fit into the Linux way. My objective is subtly but significantly different. I want to get open source packages onto AIX.
> > /haubi/ > -- > Michael Haubenwallner > Gentoo on a different level >


Subject Author
Re: [gentoo-alt] AIX linking adventure Michael Haubenwallner <haubi@g.o>