Gentoo Archives: gentoo-alt

From: Perry Smith <pedzsan@×××××.com>
To: gentoo-alt@l.g.o
Subject: Re: [gentoo-alt] AIX linking adventure
Date: Tue, 08 Feb 2011 18:36:35
Message-Id: 0C8FD112-2752-4D38-94C9-11F3FF700923@gmail.com
In Reply to: Re: [gentoo-alt] AIX linking adventure by Michael Haubenwallner
1 On Feb 8, 2011, at 2:49 AM, Michael Haubenwallner wrote:
2
3 > Hi Perry!
4 >
5 > On 02/08/2011 01:35 AM, Perry Smith wrote:
6 >> This is *not* ready for prime time but I thought I would mention it here.
7 >>
8 >> I wrote a front end to AIX's ld command as well as a relink command I'm calling "rl".
9 >
10 > Actually, in Prefix we already do wrap AIX' ld in sys-devel/binutils-config
11 > to get the runpaths (the -blibpath argument) right.
12 >
13 > At the moment I'm trying another thing to get full "soname" support,
14 > both with and without libtool, using Import Files. There is an RFC[1]
15 > for the way I want to have shared libraries being created on AIX.
16 >
17 > However, I'm still unsure if we actually should use the '# autoload' thingy,
18 > but ignore the auto-dependency problem[2] instead.
19 >
20 > [1] http://lists.gnu.org/archive/html/libtool/2011-01/msg00023.html
21 > [2] http://archives.gentoo.org/gentoo-alt/msg_b7aec69c70e257206ac49f74a41c9af6.xml
22
23 I am not on the libtool list. I did not read that thread yet but I will here shortly.
24
25 I need some "netiquette" help. Should I plop my thoughts / questions about that
26 thread into this thread or start a new thread? I get confused either way :-) (Maybe
27 I'm special that way.)
28
29 >
30 > Additionally, for packages not using libtool, there's another ld wrapper in
31 > sys-devel/native-cctools waiting for checkin here to support the '-soname' flag
32 > known on ELF (Linux), creating the shared libraries according to above way[1].
33 > Eventually, this second wrapper would be merged into binutils-config's wrapper.
34 >
35 >> You can see them here:
36 >>
37 >> https://github.com/pedz/aixbin
38 >
39 > Commenting on 'Readme.markdown' found there:
40 > Why do you move '-blibpath' arguments to '-L' arguments?
41 > IMO, the other way round would make more sense actually.
42 > Why do you need 'rl' (relink) at all?
43
44 For ruby, their libpath was a complete mistake. They did not include
45 the path to gcc's library. They did not have a -L
46 /path/to/where/they/would/install but they did have
47 -blibpath:/path/to/where/they/would/install:/usr/lib:/lib and somehow
48 thought that libpath was going to fix it. I guess it would have
49 if it wasn't for the fact that the executable could not find gcc's
50 library. And... the executable would never work in the directory
51 where it was built in. It would work only after the package was
52 installed (moved). I don't know how they were going to do
53 the "test" or "check" phase which I'm sure Ruby has.
54
55 I took their mistake as a symptom as a general practice. i.e.
56 the general population somehow thinks that libpath substitutes for
57 -L and it does not. I'm interested to hear of your experience but
58 I can't think of a time where open source understood what libpath
59 (inside the object) could/should be used for.
60
61 The Ruby guys did have a fairly reasonable fear. They did not want
62 to have just -L . and -L .., etc and not set the libpath because the -L .
63 adds that to the internal libpath and that creates what they called a
64 security risk. But, the problem is that the dot needs to be in the internal
65 libpath while the executable is in the directory it was built in.
66
67 The purpose for rt is after the install. The first link of ruby
68 leaves the path to libruby.so as relative (no path component). This
69 allows ruby to execute in the directory that it was built in (before rt is run)
70 as well as its final destination (after rt is run) because both dot and
71 the final destination directory are in the internal libpath.
72
73 This is going to be a general problem. You want the executables to
74 work before being installed. Often there are check or test processes
75 that use this feature. This implies that finding the libraries is going
76 to need to be done via relative paths.
77
78 But after ruby (the package) is installed, I wanted to fix the path
79 to libruby.so to the place that it was installed. I could not guess
80 where it was going to be installed during the "build" phase (the
81 first compile / link). I had to wait until it was installed and
82 then use libpath (which I had to fix for Ruby via adding the -L)
83 to find where libruby.so finally got installed at.
84
85 I also had a rather self imposed limitation to work around.
86 I am trying to get "rvm" to work and rvm thinks it knows how to
87 unpackage, configure, build, and install without any user
88 intervention. To get that to work, I had to hook in at the ld
89 point and then, after the fact, fix the resulting files after they
90 are installed.
91
92
93 >
94 >> I've used this only for Ruby -- and it was in fact Ruby that caused me to do this.
95 >> Their out of the box system does not work at all for AIX if the prefix directory is not the same as GCC's.
96 >> WIth this, I can now use rvm on AIX and do "rvm install 1.9.1" and it will complete successfully without my fingers leaving my hands.
97 >
98 > Haven't used/installed Ruby on any platform yet: What is 'rvm' ?
99
100 rvm is "Ruby Version Manager". rvm allows completely independent
101 versions of Ruby (1.8.7, 1.9.1, MacRuby, etc). Each version can have
102 its own independent sets of gems (called gemsets) so an application
103 can use its own version of Ruby and its own set of gems. I can set up
104 an environment on my development laptop and then fairly easily replicate
105 this environment on my staging server and production server. It solves
106 many problems I have when trying to have three or four Rails apps
107 on the same host. (sorry for the long winded reply).
108
109 >
110 >> I mentioned this idea on this list before but it didn't fly very well.
111 >> I'm not sure if it was completely understood.
112 >
113 > I fail to find that post you're referring to. Has it been within one
114 > of the threads we already created together, and I've overlooked it?
115
116 I *think* this is the post but it got munged:
117
118 http://archives.gentoo.org/gentoo-alt/msg_d16e4dce43974e6926eedf04a3b8fb36.xml
119
120 My archive is this (snipped out).
121
122 The time stamp is Tue, Dec 14, 2010 at 6:25 PM (to the gentoo-alt list)
123
124 > Also, the guy I'm talking to mentioned rtl_enable -s. I've not used it but it seems like it might make life a lot easier.
125 >
126 > You can play with it some but if you point it at a shared object, it creates a script, an import file, and an export file. Together, they will recreate the shared object. You can then edit the files, for example, adding absolute paths, if you want to change the header of the shared object file.
127 >
128 > I am not familiar with portage at all but I've seen that it patches various things like the configure script, etc. It might be easier / safer to leave the scripts alone (leave libtool do whatever it wants) and then use rtl_enable -s, edit the resulting files, and rebuild the shared object the way you want it.
129 >
130
131
132 >
133 >> This is more of a proof of concept at this stage. I'm going to putter around and try and use it on other packages.
134 >
135 > For packages not aware of AIX at all:
136 > How would this require their build systems to be patched?
137 >
138 >> "rl" could be enhanced to take "from" and "to" directories.
139 >> If you wanted to change files installed in /some/old/directory and put them into /some/new/directory, rl could easily modified do that.
140 >
141 > In which context do you need this use case?
142
143 With putting absolute paths into the objects, we are effectively nullifying the possible benefit of the LIBPATH environment. I think we
144 both agree that that benefit is questionable but others might like it. So if someone wants to move where I built a package from
145 /usr/local to /opt, the first step would be to change the internal absolute paths to each dependency. There would be other obstacles too
146 like any internal paths of the executables but often there are command line arguments to change those. e.g. you can give
147 emacs directions on where to find its lisp files but you can not magically tell the loader where to find emacs' dependent libraries.
148 A user once could via LIBPATH but that isn't going to work now with absolute internal paths. rl gives that possibly useful option back
149 to the user.
150
151 > And how would you do that on Linux?
152
153 I don't know squat about Linux. Part of my frustration is everyone appears to want to make AIX fit into the Linux way.
154
155 My objective is subtly but significantly different. I want to get open source packages onto AIX.
156
157 >
158 > /haubi/
159 > --
160 > Michael Haubenwallner
161 > Gentoo on a different level
162 >

Replies

Subject Author
Re: [gentoo-alt] AIX linking adventure Michael Haubenwallner <haubi@g.o>