Gentoo Archives: gentoo-dev

From: Steven J Long <slong@××××××××××××××××××.uk>
To: gentoo-dev@l.g.o
Subject: [gentoo-dev] Re: .LIBPATTERNS harmful?
Date: Sun, 22 Apr 2012 22:10:51
Message-Id: jn1vim$fjc$1@dough.gmane.org
In Reply to: Re: [gentoo-dev] .LIBPATTERNS harmful? by Mike Frysinger
1 Mike Frysinger wrote:
2
3 > On Sunday 22 April 2012 00:44:11 Steven J Long wrote:
4 >> I can find nothing overriding it in portage, which makes sense, since in
5 >> general one cannot know if the package in question uses gmake
6 >> .LIBPATTERNS to link to locally-built libs. However I can't help thinking
7 >> of it as harmful for a package manager, since a command like ld would be
8 >> given a parameter of say, /usr/lib/libfoo.so, not -lfoo, meaning LDFLAGS
9 >> would be irrelevant for its lookup.
10 >
11 > .LIBPATTERNS only matters if you specify the -lfoo in the dependency, and
12 > then link in via an automatic make variable.
13 >
14 Indeed. But that is accepted, conventional usage of make: it's why $+
15 exists, for example.
16
17 > e.g. this:
18 > $ cat Makefile
19 > all: test
20 > test: -lm
21 > $ echo 'main(){}' > test.c
22 > $ make
23 > cc test.c /usr/lib/libm.so -o test
24 >
25 > so the easy answer is: don't add -lfoo flags as dependencies to make
26 > targets. if you want to have something link in a library, do:
27 > $ cat Makefile
28 > all: test
29 > test: LDLIBS += -lm
30 > $ make
31 > cc test.c -lm -o test
32 >
33 The problem with target-specific variables, is that they also apply to any
34 prerequisites (and their prerequisites..) that get updated. While LDLIBS is
35 only used for linking final executables, it's not a general solution, since
36 it's possible for an object file to depend on a built executable (eg if the
37 binary is used to build an input file, or is a test program, or happens to
38 be a prerequisite of another target in the chain):
39
40 $ echo 'main(){}' > bar.c
41 $ echo 'const char * foo(void) { return "fubar"; }' > foo.c
42 $ printf '%s\n' '#include <stdio.h>' 'const char * foo(void);' 'int main(){'
43 ' printf ("Result: %s\n", foo()); }' > test.c
44
45 $ cat Makefile
46 test: foo.o -lm
47 foo.o: bar
48 $ make
49 cc bar.c -o bar
50 cc -c -o foo.o foo.c
51 cc test.c foo.o /usr/lib/libm.so -o test
52
53 vs:
54 $ cat Makefile
55 test: foo.o
56 test: LDLIBS += -lm
57 foo.o: bar
58 $ make
59 cc bar.c -lm -o bar
60 cc -c -o foo.o foo.c
61 cc test.c foo.o -lm -o test
62
63 Furthermore, if we apply the method iteratively (let's say bar uses
64 pthreads) and add:
65 bar: LDLIBS += -lpthread
66 ..then we get:
67 cc bar.c -lm -lpthread -o bar
68 ..meaning that test's lib/s will be searched before those specified by bar.
69 For the gallery, this is all a no-no in general: just because a lib should
70 be linked into one object, does not mean it should link into another, unless
71 it's explicitly been specified. Libs are searched in the order they appear
72 on the command-line, so test's libs would interpose symbols in bar's link.
73
74 All this makes the method package-specific, so it has to be done (and
75 tested) on a case-by-case basis, and imo liable to random breakage due to
76 interposition on prerequisite linkages. Worse, the value is unpredictable,
77 since it varies according to which targets are being updated during the make
78 run (and have thus added to LDLIBS for their prerequisites.) While this
79 might not matter so much for distro-builds, it can mess up development
80 builds.
81
82 In testing how to fix this, I found running:
83 make .LIBPATTERNS=
84 ..gave me:
85 make: *** No rule to make target `-lpthread', needed by `bar'. Stop.
86 So I just added a .PHONY line:
87 $ cat Makefile
88 test: foo.o -lm
89 foo.o: bar
90 bar: -lpthread
91 .PHONY: -lm -lpthread
92
93 ..and oddly enough, I found this alone, was enough to disable the expansion
94 of those libs:
95 $ make
96 cc bar.c -lpthread -o bar
97 cc -c -o foo.o foo.c
98 cc test.c foo.o -lm -o test
99
100 I got the same result with: make .LIBPATTERNS:= so using both seems like the
101 best general solution, since we're then guaranteed make will do no libname
102 substitution, and we can use conventional -lfoo deps for external libs.
103
104 >> I'd hope upstream would accept them, since it makes cross-development
105 >> easier. (One definitely does not want make expanding -lname to a library
106 >> in /lib or /usr/lib in that case, and it's better to error out if the
107 >> library can't be found than link to host libs.)
108 >
109 > i've seen this usage in only one or two packages before. and when i
110 > notified the respective upstream, they weren't really doing it on purpose,
111 > so a simple patch (like i showed above) they were fine with merging.
112
113 Thanks, that's exactly the kind of knowledge I don't have, and it's good to
114 know that no-one really wants -lfoo looked up and substituted by make.
115
116 Adding -lname specified prereqs to .PHONY is a simple fix, although if
117 they're internal there won't be any lookup done by make. Personally I think
118 that's a good thing, but I don't know how it'd affect things; for instance a
119 package building okay now with a dep on an internal -lfoo which is expanded
120 to a target which can be built.
121
122 But as you state, you've not seen it done on purpose (and it would mess up
123 cross-compiles) so I guess the fix there is for the internal lib to be
124 specified as a filename, libfoo.so or .a (and perhaps -L .)?
125
126 In any event, we seem to agree that we don't want .LIBPATTERNS expansions
127 happening.
128
129 Personally, I feel setting .LIBPATTERNS empty by default, either via make
130 command-line (or setenv unless the package is marked as needing the
131 expansions) would be 'correct', in that it would pick up potential problems
132 straightaway, but I don't have the knowledge to assess the consequences. It
133 wouldn't affect packages using the LDLIBS+= target-specific setting, at
134 least, but would break packages using -lfoo prerequisites via automatic
135 variables, which haven't been patched to use LDLIBS+= or add .PHONY deps.
136
137 Might be something to consider for ebuild-developer mode, so new ebuilds
138 don't come in with the potential for borked linkage.
139
140 A case-by-case fix would be to add .LIBPATTERNS:= to the makefile when
141 adding .PHONY deps.
142
143 Anyhow, thanks for discussion and sharing your know-how; it means I now know
144 how to handle it in our builds at least (.PHONY for external libs specified
145 as -lfoo, .LIBPATTERNS set empty, filenames for internal libs and -L params
146 for their directories.)
147
148 Regards,
149 Steve.
150 --
151 #friendly-coders -- Where everybody knows your nickname ;-)