1 |
On Feb 10, 2011, at 3:06 AM, Michael Haubenwallner wrote: |
2 |
|
3 |
> |
4 |
> On 02/10/11 01:57, Perry Smith wrote: |
5 |
>>> On 02/09/11 17:47, Perry Smith wrote: |
6 |
>>>> Lets start over. I don't think I'm making any progress but I'll give it one last go. |
7 |
>>>> ***Import File Strings*** |
8 |
>>>> INDEX PATH BASE MEMBER |
9 |
>>>> 0 /usr/local/rvm/rubies/ruby-1.9.1-p378/lib:/usr/local/lib/gcc/powerpc-ibm-aix5.3.0.0/4.3.1:/usr/local/lib/gcc/powerpc-ibm-aix5.3.0.0/4.3.1/../../..:/usr/lib:/lib |
10 |
>>>> 1 librtl.a shr.o |
11 |
>>>> 2 /usr/lib libc.a shr.o |
12 |
>>>> 3 /usr/lib libpthread.a shr_comm.o |
13 |
>>>> 4 /usr/lib libpthread.a shr_xpg5.o |
14 |
>>>> 5 /usr/local/rvm/rubies/ruby-1.9.1-p378/lib libruby.so |
15 |
>>> Is this with our without your intervention in ruby's build process and/or toolchain setup? |
16 |
>> |
17 |
>> This is after my toolchain changes. |
18 |
> |
19 |
> How does this look like without your changes? |
20 |
|
21 |
I can't recall 100%. I can go back and do another build. In particular, I can't recall |
22 |
who had the problem if it was ruby itself or libruby.so that had the problem. In general, |
23 |
there were no "hardcoded" elements and the 0th element was simply: |
24 |
|
25 |
/usr/local/rvm/rubies/ruby-1.9.1-p378/lib:/usr/lib:/lib |
26 |
|
27 |
Let me know if that is sufficient. If not, I can do another build without my mod's. |
28 |
|
29 |
> |
30 |
>> The 1st entry is what I was calling "relative" but as you point out, that isn't the right term. |
31 |
>> "affected by run path search" is too long for me. :-) |
32 |
>> |
33 |
>> The 2nd through 5th items are those I was calling "absolute" but really they should be |
34 |
>> called "not affected by run path search". Perhaps "fixed" in this case would work? |
35 |
>> non-LIBPATH-able? |
36 |
> |
37 |
> IIRC, "hardcoded" is what libtool uses for that kind of paths, |
38 |
> but I'm fine with "fixed" too. |
39 |
|
40 |
Lets use existing terms if they exist. |
41 |
hardcoded isn't too long. what is the opposite? non-hardcoded? |
42 |
|
43 |
> |
44 |
>>> How exactly did you touch either the build process or the toolchain setup to "allow" that? |
45 |
>> |
46 |
>> I put my ld in /usr/local/aixbin/ld and I put /usr/local/aixbin as the first element of PATH. |
47 |
>> So my ld gets called instead of the real ld. My ld monkeys with things. |
48 |
> |
49 |
> Which compiler did you use - gcc or xlc? |
50 |
|
51 |
gcc 4.3.1 |
52 |
|
53 |
> Does xlc search for 'ld' using PATH? |
54 |
|
55 |
Good question. I don't know. |
56 |
|
57 |
> Usually, gcc does not, you'll have to configure it to use "/usr/local/aixbin/ld". |
58 |
|
59 |
collect2 (from gcc) calls ld. I'm using my "stock" gcc build and its working for me. |
60 |
|
61 |
> |
62 |
>>>> Lets pause here and see if we can come to an agreement on these two items. |
63 |
>>>> 1) objets have only absolute paths after they are installed |
64 |
>>> |
65 |
>>> Still not clear to me: |
66 |
>>> Do you actually /prefer/ hardcoded paths (without runpath search)? Why? |
67 |
>>> |
68 |
>>> Although runpath search is affected by LIBPATH, I'm just fine with that. |
69 |
>>> This is the outcome of using "-L" and "-l" linker flags together. |
70 |
>> |
71 |
>> I thought this was one of your objectives based on the fact that java sets LIBPATH |
72 |
>> to something weird and it breaks everything. |
73 |
> |
74 |
> Nope. The problem with LIBPATH set by java was that there is nothing |
75 |
> like "soname". Iff my binary had searched for the /file/ "libiconv.so.2" |
76 |
> (either archive or not) instead of "libiconv.a(libiconv.so.2)", even with |
77 |
> LIBPATH set to "/usr/lib" it would not have found "/usr/lib/libiconv.a" |
78 |
> (without "libiconv.so.2" member), but continued along LIBPATH + runpath |
79 |
> until it found the "libiconv.so.2" file in its usual place. |
80 |
|
81 |
Hmmm... ok. I had forgotten that detail. |
82 |
|
83 |
> |
84 |
>>> Seems we have to agree on some terms to be used first. |
85 |
>> |
86 |
>> Yes. |
87 |
>> |
88 |
>> I like "fixed" for entries not affected by runtime search. |
89 |
> |
90 |
> "fixed" = "hardcoded" (see above). |
91 |
> |
92 |
>> The other tact would be to note that PATH is either empty or not. |
93 |
>> |
94 |
>> "A path dependency with no path" or "a dependency with a path" ? |
95 |
> |
96 |
> Which "PATH" ("path") do you refer to here that can be either empty or not? |
97 |
|
98 |
In the dump -H output above, there is a column called PATH. That PATH. I apologize for being |
99 |
so consistently vague. |
100 |
|
101 |
> There is the "PATH" environment variable, which does not affect runpath search at all. |
102 |
> There is the "LIBPATH" environment variable, which does. |
103 |
> There is the encoded "hardcoded path" (="fixed") for one library, not allowing runpath search. |
104 |
> There is the encoded "runpath", used (after LIBPATH) for any library without a "hardcoded path". |
105 |
> |
106 |
>> "binary" instead of "object" sounds better to me. |
107 |
> |
108 |
> ok. |
109 |
> |
110 |
>>> What else to say: |
111 |
>>> When libtool knows for a platform that linking libraries results in hardcoded |
112 |
>>> paths, or there is no such thing like LIBPATH environment variable to temporarily |
113 |
>>> override the runpath, it actually /does/ "relink" the binaries during installation. |
114 |
>> |
115 |
>> I understood that part -- but |
116 |
>> |
117 |
>>> OTOH, when the /absolute/path/to/the/library.so found at linktime is hardcoded into |
118 |
>>> the resulting binary, installing into DESTDIR is broken - which is a prerequisite |
119 |
>>> for package managers like portage to work properly. |
120 |
>> |
121 |
>> Didn't understand this part. In particular, from the comma to the end ", installing ..." |
122 |
> |
123 |
> Ohw, ok - this is the core requirement for proper package managing: |
124 |
> |
125 |
> Consider these raw build steps for an usual package, which does provide |
126 |
> "${prefix}/lib/library.so", and "${prefix}/bin/mybinary" linked against it: |
127 |
> |
128 |
> 1. Configure: |
129 |
> $ ./configure --prefix="/final" |
130 |
> 2. Compile: |
131 |
> $ make |
132 |
> 3. Install (to allow for a binary package): |
133 |
> $ make install DESTDIR="/image" |
134 |
> The important thing here is to /not/ copy anything to "/final" at the |
135 |
> build machine, but still the binaries copied to "/image/final" have |
136 |
> to be able to run when moved to "/final" by the package manager, |
137 |
> eventually on another machine with (ideally) identical setup. |
138 |
|
139 |
Does the package (need to) work from the /image area or is this |
140 |
just a temporary place used to create the tar file? In particular, |
141 |
what about paths hard coded into the binaries? (Not the |
142 |
paths in the binary's header but just paths hard coded in the code |
143 |
like where to find my lisp files, etc. I know that *usually* there are |
144 |
options to change them but there is still some default set at compile |
145 |
time. |
146 |
|
147 |
> 4. Create the binary package: |
148 |
> $ cd "/image" |
149 |
> $ find "final" > "filelist-this-package-does-install" |
150 |
> $ tar cfj "binpackage.tar.bz2" "filelist-this-package-does-install" "final" |
151 |
> 5. optionally ship the binary package to another host with (ideally) identical setup. |
152 |
> 6. See if the binary package would fit: |
153 |
> $ tar xfj "binpackage.tar.bz2" "filelist-this-package-does-install" |
154 |
> $ for f in $(<"filelist-this-package-does-install") |
155 |
> do |
156 |
> if [[ -e /${f} ]] |
157 |
> then |
158 |
> do-we-update "${f}" && continue |
159 |
> echo "Installing this package conflicts with existing '/${f}'" |
160 |
> fi |
161 |
> done |
162 |
> 7. Merge the binary package to the live filesystem: |
163 |
> $ cd "/" |
164 |
> $ tar xfj "binpackage.tar.bz2" "final" |
165 |
> 8. Use the package: |
166 |
> $ "/final/bin/mybinary" |
167 |
> |
168 |
> During step 3, the shared library gets installed as "/staging/final/lib/library.so". |
169 |
> Relinking "mybinary", the linker has to read that file, while encoding "/final/lib" |
170 |
> (without "/staging") into "mybinary" as the search path for "library.so". |
171 |
|
172 |
Shoot... you lost me. I think maybe /staging is the same as /image above? |
173 |
|
174 |
> |
175 |
> Usually that is done this way, encoding the final "runpath": |
176 |
> $ ld -o "/staging/final/bin/mybinary" -L "/staging/final/lib" -lrary -blibpath:"/final/lib" |
177 |
> |
178 |
> Because this won't work as expected, as the "hardcoded" path would contain "/staging": |
179 |
> $ ld -o "/staging/final/bin/mybinary" "/staging/final/lib/library.so" |
180 |
> |
181 |
> Except... when an "Import File" is found at "library.so", referring to "#! /final/lib/...". |
182 |
> |
183 |
> However, when "library.so" is a /standalone/ Import File, |
184 |
> it has to refer to something like "#! /final/lib/library.so.0", |
185 |
> so the loader actually can load a shared object rather than an Import File. |
186 |
> |
187 |
> When "library.so" is an archive file /containing/ both the Import File "shr.imp" and the shared object "shr.o", |
188 |
> the Import File has to refer to "#! /final/lib/library.so(shr.o)". |
189 |
> |
190 |
> For completion, "library.so.0" is called the "soname" in ELF world, |
191 |
> but that's a different (although related) topic. |
192 |
|
193 |
Ok. I have more thoughts on this too. Lets not go there just yet. I'm still |
194 |
lagging just a little. |
195 |
|
196 |
One question: with each of your experiments, do you dump out the |
197 |
binary's "header" with dump -H? The reason I ask is I would expect |
198 |
linking with an import file basically just put that piece, the #!<whatever> |
199 |
into the binary's header at link time and then what happens at run |
200 |
time is the same as if you had linked against a library. |
201 |
|
202 |
I remember toying with the import file inside the archive (and getting |
203 |
frustrated at the lack of documentation) but I can't remember what |
204 |
I found. |