1 |
On 3/28/2021 16:05, William Hubbs wrote: |
2 |
> On Sat, Mar 27, 2021 at 10:51:11PM -0400, Joshua Kinard wrote: |
3 |
>> On 3/27/2021 20:32, William Hubbs wrote: |
4 |
>>> On Sat, Mar 27, 2021 at 05:43:34PM -0400, Joshua Kinard wrote: |
5 |
>>>> On 3/23/2021 07:31, Rich Freeman wrote: |
6 |
>>>>> On Mon, Mar 22, 2021 at 6:54 PM Andreas K. Huettel <dilfridge@g.o> wrote: |
7 |
>>>>>> |
8 |
>>>>>>>> Council decided years ago that we don't support separate /usr without |
9 |
>>>>>>>> an initramfs, but we haven't completed that transition yet. |
10 |
>>>>>>> |
11 |
>>>>>>> Which doesn't imply that we deliberately break things. |
12 |
>>>>>> |
13 |
>>>>>> That's right. Though we should at some point start thinking about an end of support for separate usr without initramfs. |
14 |
>>>>>> |
15 |
>>>>> |
16 |
>>>>> Just to clarify - it is already unsupported at a distro level. It is |
17 |
>>>>> just that some individual packages still work with it. |
18 |
>>>>> |
19 |
>>>>> The current Council decisions on the issue are (just providing for |
20 |
>>>>> general reference): |
21 |
>>>>> |
22 |
>>>>> - "Since that particular setup may already be subtly broken today |
23 |
>>>>> depending on the installed software, Council recommends using an |
24 |
>>>>> early boot mount mechanism, e.g. initramfs, to mount /usr if /usr |
25 |
>>>>> is on a separate partition." |
26 |
>>>>> Accepted unanimously. [1] |
27 |
>>>>> |
28 |
>>>>> - "The intention is to eventually not require maintainers to support |
29 |
>>>>> a separate /usr without an early boot mechanism once the Council |
30 |
>>>>> agrees that the necessary docs/migration path is in place." |
31 |
>>>>> Accepted with 4 yes votes, 1 no vote, 2 abstentions. [1] |
32 |
>>>>> |
33 |
>>>>> - "The Council agrees that all preparations for dropping support for |
34 |
>>>>> separate /usr without an initramfs or similar boot mechanism are |
35 |
>>>>> complete. A news item will be prepared, and users will be given one |
36 |
>>>>> month to switch after the news item has been sent." |
37 |
>>>>> Accepted with 5 yes votes, 1 no vote, 1 abstention. [2] |
38 |
>>>>> |
39 |
>>>>> Current policy documentation: |
40 |
>>>>> Developers are not required to support using separate /usr filesystem |
41 |
>>>>> without an initramfs. [3] |
42 |
>>>>> |
43 |
>>>>> 1 - https://projects.gentoo.org/council/meeting-logs/20130813-summary.txt |
44 |
>>>>> 2 - https://projects.gentoo.org/council/meeting-logs/20130924-summary.txt |
45 |
>>>>> 3 - https://projects.gentoo.org/qa/policy-guide/filesystem.html#pg0202 |
46 |
>>>> |
47 |
>>>> Is there a list of software/ebuilds that currently do this "subtle" handling |
48 |
>>>> of separate /usr w/o initramfs? |
49 |
>>>> |
50 |
>>>> I've got just my MIPS systems left that use a separate /usr and do not boot |
51 |
>>>> with initramfs because I build fully monolithic kernels and that makes the |
52 |
>>>> resulting vmlinux images run up against hard size limits in the SGI PROM |
53 |
>>>> (firmware, BIOS, etc) on these machines if I try to pack too large of an |
54 |
>>>> initramfs in. I can check for any software that may be switched over soon |
55 |
>>>> to a hard initramfs requirement and look at my options. |
56 |
>>> |
57 |
>>> One group of packages involved in this is any package that calls |
58 |
>>> gen_usr_ldscript. We have this function for a couple of reasons. |
59 |
>>> |
60 |
>>> Gentoo has a policy that bans *.a static libraries from being |
61 |
>>> installed in /lib*. I think this policy is unique to Gentoo, |
62 |
>>> because Most build systems I've seen do not |
63 |
>>> have separate paths for static vs dynamic libraries, so all libraries |
64 |
>>> from a package are installed in /usr/lib* or /lib*. This policy was |
65 |
>>> originally hard coded into portage some time back (I can find the commit |
66 |
>>> if you want it) before it was made a formal policy by the qa team. |
67 |
>>> It has since become a formal policy. |
68 |
>>> |
69 |
>>> Because of this policy, we have to tell upstream build systems to |
70 |
>>> install libraries in ${ED}/usr/$(get_libdir). This is fine unless you |
71 |
>>> have separate usr with no initramfs. In that case, you have binaries on |
72 |
>>> / that link to shared libraries on /usr. When we saw this happening, we |
73 |
>>> started manually moving shared libraries from /usr/lib* to /lib* if |
74 |
>>> someone needed the package at boot time. Then we hit bug 4411 [1] |
75 |
>>> where the linker was prioritizing static libraries in /usr/lib* over shared |
76 |
>>> libraries in /lib*. |
77 |
>>> |
78 |
>>> I don't know if we tried to address that upstream or not, but ultimately |
79 |
>>> gen_usr_ldscript came out of it. |
80 |
>>> |
81 |
>>> Several folks have wanted to get rid of this function for years [2]. |
82 |
>>> |
83 |
>>> I have looked into it before, and there are several things that can be done. |
84 |
>>> |
85 |
>>> We can have packages that currently build static libraries and |
86 |
>>> use gen_usr_ldscript stop building static libraries and use the |
87 |
>>> appropriate setting in their build systems to install libraries in |
88 |
>>> /$(get_libdir). This is the path OpenRC is taking per request of the qa |
89 |
>>> lead. The next release will not support the static-libs use flag. This |
90 |
>>> will actively break anyone who is expecting this use flag to exist for OpenRC. |
91 |
>>> |
92 |
>>> If a package does not build static libraries in the first place, there is |
93 |
>>> really no reason to call gen_usr_ldscript; the ebuild should be using |
94 |
>>> the upstream build system to put the libraries where they need to be. |
95 |
>> |
96 |
>> The number of packages calling gen_usr_ldscript looks to be fairly small. |
97 |
>> Creating a TRACKER bug and sub-bugs for checking or removing the need for |
98 |
>> gen_usr_ldscript might be a way to go to pair the list down and reduce the |
99 |
>> footprint: |
100 |
>> |
101 |
> |
102 |
> Yes, I can start this work. |
103 |
> |
104 |
>> |
105 |
>>> If static libraries are needed for a package that is using |
106 |
>>> gen_usr_ldscript to move shared libraries to /lib*, the only option you |
107 |
>>> have is to drop the gen_usr_ldscript call. If you do that, all of the |
108 |
>>> libraries will stay in /usr/lib*, but as soon as one package takes this |
109 |
>>> path, there will be active breakage for someone who is using a separate |
110 |
>>> /usr without an initramfs. |
111 |
>> |
112 |
>> Correct me if wrong, but static libraries are only needed during |
113 |
>> compilation, right? The *.a files are merged into the binary at link time |
114 |
>> and thus that binary can stand on its own regardless of whether the *.a |
115 |
>> files are available or not. They're not like shared libs which are needed |
116 |
>> by the loader to resolve symbols at run time. |
117 |
> |
118 |
> You are correct, and actually in Gentoo we prefer not to link to static |
119 |
> libraries, so we build them but we do not use them. |
120 |
> |
121 |
> *snip* |
122 |
> |
123 |
>> We're mostly talking about the small window during boot where, on a system |
124 |
>> with /usr on a separate disk partition, /usr might not be available until |
125 |
>> some userspace tool in /bin or /sbin runs to make it available. Running the |
126 |
>> system compiler during boot would be a rare and very unique scenario (not to |
127 |
>> mention a sign of questionable development processes). |
128 |
> |
129 |
> The problem is, there's a chicken-and-egg problem in the scenario where |
130 |
> / and /usr are on separate partitions, and this is why a number of linux |
131 |
> distros have moved to requiring an initramfs in this situation. |
132 |
> I'm linking systemd's description here, only because it is the best |
133 |
> writeup of the issue I've found [1]. |
134 |
> Anything that is needed in the early boot process requires all of its libraries, |
135 |
> dependent libraries, binaries, data files, etc to be on /, and this has |
136 |
> become a moving target. |
137 |
|
138 |
Yeah, I've read systemd's explanation, and generally disagree with it. They |
139 |
created the problem in the first place, then invented their own solution for |
140 |
it, and now everyone acts like they're the wise men on the mountain for it. |
141 |
|
142 |
I still don't see the connection to the static *.a libs and whether they're |
143 |
on /lib or /usr/lib, though. Unless we're implying that where the *.a's go, |
144 |
so too do the *.so's go, then THAT makes sense because *.so's ARE needed at |
145 |
program runtime, whereas *.a's are not. |
146 |
|
147 |
I wonder if we couldn't shovel all static libs off to a dedicated folder |
148 |
somewhere, like '/usr/lib/static/<pkg>/*.a', similar to the way debug files |
149 |
are now consolidated under '/usr/lib/debug'. Since they're only needed |
150 |
during a specific kind of compilation that we don't support out-of-the-box |
151 |
that happens long after the system is fully booted, stuffing them off |
152 |
somewhere unimportant would make some sense. Most modern software should be |
153 |
using shared libs by default, and if it ain't, that's either a bug or that |
154 |
software is for a very specific function (like a bootloader). |
155 |
|
156 |
|
157 |
>>> The most controversial thing to do would be the /usr merge. It would |
158 |
>>> affect far more users than the other paths for getting rid of |
159 |
>>> gen_usr_ldscript, but it would make all "separate /usr" concerns go |
160 |
>>> away along with giving us a number of other benefits. |
161 |
>>> |
162 |
>>> The short version is, anything we do to remove the gen_usr_ldscript |
163 |
>>> function will actively break some group of users. |
164 |
>> |
165 |
>> If I read the temperature right, it seems like there is desire to eliminate |
166 |
>> gen_usr_ldscript regardless of sep-usr or not. If so, then that seems the |
167 |
>> way to go. Make the eventual /usr merge a separate issue to tackle some |
168 |
>> time further down the road. |
169 |
> |
170 |
> The way I see it, when we start to remove the gen_usr_ldscript calls, |
171 |
> people using a sep-usr mount without an initramfs will run into one or |
172 |
> both of these issues: |
173 |
> |
174 |
> - they might have to increase the size of their root partition depending |
175 |
> on what gets added to /lib* |
176 |
> - if one package in that list drops gen_usr_ldscript without installing |
177 |
> libraries in /lib*, it will mean they need an initramfs. |
178 |
|
179 |
I tend to make my root partitions ~4GB, which has often been plenty of room |
180 |
for well over 15 years. But again, location of the *.a static libs is |
181 |
irrelevant during system boot. They are not needed nor referenced when a |
182 |
program executes. A statically-compiled program has all of its dependencies |
183 |
lumped inside of it, so you could put it pretty much anywhere on the |
184 |
filesystem and run it (ignoring for a moment 'noexec' potentially being |
185 |
set). Or even more to the point, you could e-mail a static binary to |
186 |
someone else on a different distro and it'll probably execute just fine. |
187 |
|
188 |
The important bit here is certain software packages defaulting their install |
189 |
into /usr that are needed to bring the system up (like encryption, lvm, |
190 |
etc). It's when using those setups that you pretty much have to use an |
191 |
initramfs, sometimes regardless if you have /usr on another partition or not. |
192 |
|
193 |
|
194 |
> From my pov, it is better to start using initramfs in that situation, |
195 |
> and if you do that you fix all of the other issues mentioned on |
196 |
> the page linked above. It feels like the usr merge is less of a big deal |
197 |
> once we know everyone who has /usr separate is using an initramfs. |
198 |
|
199 |
Yeah, and the problem there has been the generation of those initramfs |
200 |
archives. I'm stuck between a rock and a hard place because the classic |
201 |
tools like dracut target popular archs like amd64 or arm64, but not mips, |
202 |
and especially not 20+ year-old hardware, and go on to make assumptions |
203 |
under that pretense. Luckily, my MIPS machine setups are simple and really |
204 |
only rely on mdraid auto-assembling via 0.90 metadata (otherwise, I'd need |
205 |
an initramfs to get the things to boot, cause someone decided at some point |
206 |
that it ain't the kernel's job assemble the array, and newer metadata |
207 |
signatures enforce this...). |
208 |
|
209 |
Add on to that, my specific use-case, I really just need the tiniest binary |
210 |
that tells the kernel to go and mount /usr as an XFS partition on /dev/md2 |
211 |
(after array assembly). No one seems to have cobbled together an initramfs |
212 |
generator that simple. Instead, the Linux world went straight for the Rube |
213 |
Goldberg versions of initramfs. |
214 |
|
215 |
-- |
216 |
Joshua Kinard |
217 |
Gentoo/MIPS |
218 |
kumba@g.o |
219 |
rsa6144/5C63F4E3F5C6C943 2015-04-27 |
220 |
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943 |
221 |
|
222 |
"The past tempts us, the present confuses us, the future frightens us. And |
223 |
our lives slip away, moment by moment, lost in that vast, terrible in-between." |
224 |
|
225 |
--Emperor Turhan, Centauri Republic |