1 |
Per subject, let's open a can full of worms. |
2 |
|
3 |
For the last few weeks I've been working on 64-bits non-Linux targets. |
4 |
sparc64-solaris at first appeared to be a piece of cake, |
5 |
powerpc64-apple-darwin8 and x86_64-apple-darwin9 tricky because of |
6 |
python, openssl, and most annoying odcctools not being 64-bits |
7 |
safe-code. With x86_64-pc-solaris2.10 I've almost reached the end (5 |
8 |
packages pending, mostly trivial) but on the way fixing bugs in |
9 |
packages, I found some horrible scenario forced upon us. Let's start |
10 |
off that x86_64-pc-solaris2.10 needed patches to both gcc and binutils |
11 |
because both do not recognise it as a valid target. By relatively |
12 |
simple patches they seem to work reasonably ok, though. |
13 |
|
14 |
Now the real problem is in the library search paths. When compiling |
15 |
wget on Solaris 10/x86_64 the linker started to complain that there were |
16 |
undefined symbols: intl_gettext. Shame on me, as I ignored those on |
17 |
sparc64-sun-solaris2.10 (you only get them with USE="nls"). |
18 |
The message is weird, and doesn't appear on i386-pc-solaris2.10. |
19 |
Further investigation shows that the linker pulls in |
20 |
/lib/64/libintl.so.1 (or it's realpath |
21 |
/lib/{amd64,sparcv9}/libintl.so.1) instead of libintl.so.8 as we have in |
22 |
prefix. Linking with -lgettextlib results in a warning that conflicts |
23 |
may arise since two libintl.so's are being included, as expected. |
24 |
(gettextlib was probably because of libtool correctly linked against the |
25 |
prefix libintl.so.8). |
26 |
|
27 |
Further investigations (gcc .... -v -Wl,-t) revealed that gcc calls |
28 |
collect2 nicely like ".../collect2 -L/usr/lib/amd64 -L/lib/amd64 ...", |
29 |
which -- indeed -- pushes the host "64-bits libs" in front of any other |
30 |
-L direction onto the linker. I considered that to be no good, and |
31 |
quite explanatory for wget linking to /lib/amd64/libintl.so.1 by |
32 |
default. After some searching, the MULTILIB_OSDIRNAMES variable |
33 |
appeared to be causing this. For the m32 targets of gcc on Solaris it's |
34 |
set to ".", for the m64 targets it's set to either amd64 or sparcv9. |
35 |
Since "gcc -m32 .... -v -Wl,-t" revealed no -L instructions for /usr/lib |
36 |
and /lib being passed onto collect2, I locally patched my GCC to have |
37 |
also in the 64-bits case MULTILIB_OSDIRNAMES to be ".". This is where |
38 |
the fun starts. |
39 |
|
40 |
Under Linux, where I've always happily assumed everything is ok, gcc |
41 |
injects -L/usr/lib/../lib64 and -L/lib/../lib64. This fits nicely with |
42 |
MULTILIB_OSDIRNAMES set to "../lib64". It is no good though, so my |
43 |
patch rips them out for Linux as well. Now I started compilling on |
44 |
Solaris, and at the final xgcc call to link together a new stage of the |
45 |
compiler, the GNU linker neatly segfaults. Before it does, it complains |
46 |
that i386 objects aren't compatible with x86_64 ones. Looking into what |
47 |
gcc passes on to collect2, indeed no -L/usr/lib/amd64 is passed, so yay, |
48 |
that worked, but those paths /need/ to be searched in the end, of |
49 |
course. So, work to do for the binutils-wrapper script, that's what we |
50 |
have it for in the end, right? |
51 |
|
52 |
Well, not entirely it seems. |
53 |
|
54 |
First, we never ever injected -L/usr/lib{,64} and -L/lib{,64} into the |
55 |
search path. Binutils folks apparently stick to the standard there, so |
56 |
the 64-bits libpaths are not in the default search path. Right they |
57 |
are, because neither does the runtime linker/kernel do so. Great. Then |
58 |
why did my amd64 linux box at work run so well? On Fedora multi-lib, |
59 |
/lib is the 32-bits dir, unlike under Gentoo, where /lib is always |
60 |
linked to the native bits libdir. To all joy, there is no runpath to |
61 |
/lib64 in bin/bash, and yet ldd happily shows it links against |
62 |
/lib64/libc.so.6. Conclusion, the runtime linker/kernel *does* use |
63 |
lib64 on Fedora at least for a 64-bits object. Getting sweaty yet? I |
64 |
do. |
65 |
|
66 |
My GCC on the Fedora box nicely passes /usr/lib/../lib64 as first |
67 |
argument to collect2. The ldwrapper adds -rpath=${EPREFIX}/usr/lib64. |
68 |
The compiler checks for includes in ${EPREFIX}/usr/include first. |
69 |
* compiler sees correct headers |
70 |
* linker sees uncorrect libs when resolving symbols |
71 |
* kernel sees correct libs, because of run_path directions in the object |
72 |
It probably never failed for me so bad, since Fedora is quite up-to-date |
73 |
with their stuff these days, so I never had clashes. This is scary of |
74 |
course. |
75 |
|
76 |
On Darwin, no -L paths are ever passed to collect2, but we would have |
77 |
noticed it there much sooner. This is where the multilib story comes |
78 |
in. The search order we basically need is: ${EPREFIX}-libs, /-libs. On |
79 |
Darwin no lib64, lib/64, or lib/sparc64 exists. Indeed, they use "FAT" |
80 |
objects, sort of archives where the kernel extracts the object for the |
81 |
arch it needs. Neat or not, it means on Darwin there is only "lib" as |
82 |
directory. We pass ${EPREFIX}-libs in the ldwrapper, and get /-libs |
83 |
from the linker and runtime linker's defaults. So Darwin is correct in |
84 |
this case, as any other 32-bits target. GCC doesn't pass -L/usr/lib as |
85 |
it has been told that's the default anyway. |
86 |
|
87 |
I implemented the 64-bits native targets for Solaris and Darwin as |
88 |
entire prefixes just with 64-bits code and a 64-bits defaulting |
89 |
compiler. I just didn't think of it actually, but I just use |
90 |
${EPREFIX}/{usr,}/lib for the libraries. E.g. no multilib profiles. |
91 |
For inside the prefix this all works well, however with my patched gcc, |
92 |
the 64-bits system libraries are no longer seen by the linker, as |
93 |
neither gcc nor the ldwrapper give -L directions to them for platforms |
94 |
where they are not /lib. Only since Darwin has FAT objects, it works |
95 |
fine there. Not for Solaris and Linux. Well, Linux, depends on the |
96 |
vendor. Would work for Gentoo, not for Fedora. |
97 |
|
98 |
In prefix, I copied an amd64 profile, and it is a multilib-profile. I |
99 |
don't know exactly why, but at the time I did it, I thought that was |
100 |
good. From a desktop perspective multilib is good for Linux users, as |
101 |
e.g. acroread doesn't come in a 64-bits version. However, should that |
102 |
be our concern? Anyway, the ldwrapper adds: |
103 |
ADDLIBDIR("/usr/lib64"); |
104 |
ADDLIBDIR("/usr/lib"); |
105 |
ADDLIBDIR("/lib64"); |
106 |
ADDLIBDIR("/lib"); |
107 |
For the prefix, and at runtime those libs are added if they exist on the |
108 |
system the wrapper runs on. Now for most platforms a bunch of these are |
109 |
not necessary. Darwin will never use the 64 variants, 32-bits installs |
110 |
neither. But why do we use lib64 at all? If we have a compiler that |
111 |
produces 64-bits code, shouldn't the prefix be fully 64-bits as well? |
112 |
The ldwrapper has to be made smarter for cross-compiling anyway, but |
113 |
shouldn't we just nuke lib64? ... or should we think about a "sane" way |
114 |
to overlay a 32-bits prefix with a 64-bits one? Personally I'm against |
115 |
that, better keep it clean, and add paths from both prefixes to your |
116 |
path. |
117 |
|
118 |
In any case, the ldwrapper will have to add /usr/lib64 and friends in |
119 |
case the target system is one that uses those... |
120 |
|
121 |
|
122 |
-- |
123 |
Fabian Groffen |
124 |
Gentoo on a different level |
125 |
-- |
126 |
gentoo-alt@g.o mailing list |