Gentoo Archives: gentoo-dev

From: Jaco Kroon <jaco@××××××.za>
To: gentoo development <gentoo-dev@l.g.o>
Subject: Re: [gentoo-dev] separate /usr without initramfs
Date: Fri, 25 Oct 2019 19:53:07
Message-Id: 6c42a495-4397-50f9-ed46-804a39db61af@uls.co.za
In Reply to: [gentoo-dev] separate /usr without initramfs by William Hubbs
1 Hi,
2
3 On 2019/10/25 20:14, William Hubbs wrote:
4 > Hey all,
5 >
6 > I have been advised to bring this topic back to the list before taking
7 > any action, so here it is.
8 >
9 > First, I need to clarify what I'm *NOT* talking about.
10 >
11 > This discussion has nothing to do with whether or not you have the
12 > split-usr use flag turned on; all of us officially have that on because
13 > /bin, /lib* and /sbin are directories in the official Gentoo setup. In
14 > other words, I am *not* talking about forcing the /usr merge.
15 >
16 > Unfortunately, the concept of separate usr has gotten wrapped up in the
17 > split-usr use flag and doesn't have to be. For the record, I mean something
18 > very specific when I say "separate usr". I am talking about the situation
19 > where /usr is a mount point separate from /, so in this thread, let's stick
20 > to "separate usr" for that situation. I am *not* even saying that using
21 > separate usr is wrong or unsupported. You can even run separate usr with
22 > split-usr turned off if you would like to do so.
23 >
24 > Now for the use case I want to talk about, and that is using separate
25 > /usr without using an initramfs to boot your system and pre-mount /usr.
26 >
27 > If you do this, many things are broken, and this is why the binary
28 > distros all use an initramfs if you do this. This configuration is also
29 > unsupported officially in Gentoo [1] [2], and it is not shown as the
30 > example setup in our handbook.
31 >
32 > I want to hear from people who have / and /usr on separate partitions
33 > and who are not using an initramfs.
34 >
35 > If you are in this group, I have a very specific question. Why aren't
36 > you using an initramfs?
37
38 Because until recently it wasn't an issue.  So for me the final kicker
39 was still not a separate /usr.  For my use case everything except a big
40 warning about md5sum not being available during boot works.  This is NOT
41 desktop setup for MOST of my systems.  On the few that are I don't
42 generally have things like bluetooth keyboards etc that I need available
43 during early(ish) boot or anything crazy.  So the argument that "many
44 things break" may be accurate, but I've yet to find a concrete example
45 that bugs me enough to care.
46
47 There is basically one thing that I found that broke "out of the box"
48 with a separate /usr - and that's if I have one of those stupid LTE
49 modem things that pretend to be a disk drive/cdrom or something similar
50 until you tell it to switch to network mode. The same thing that, for
51 me, breaks with suspend resume (after resume I have to kill
52 modem_manager, and start it *before* replugging the modem or it simply
53 will not work - another discussion).
54
55 So frankly, I just don't see the benefit.  The reason I've eventually
56 bothered with setting up and creating an initramfs now was because of
57 /lib/firmware which I need available during module load (pre
58 /etc/init.d/localmount) so that firmware is available as soon as amdgpu
59 and i915 loads or else I end up with a borked screen.  Only uefi fb
60 compiled into the kernel (required to get hand-over from grub2 ... at
61 least, the only way I could get it to work smoothly).
62
63 The initramfs we ONLY pull in on systems where it's required (one
64 currently, the machine I'm typing this on).
65
66 Why do I have /lib/firmware on a separate partition now?  Because 512MB
67 for / used to be plenty, and I have MANY history systems out there which
68 is non-trivial to make partitioning changes to (I keep / in a raw
69 partition).  Now just /lib/firmware is larger than that.
70
71 Like William, / and /boot are definite partitions, and I want to keep
72 these small.  Everything else is LVM.  Most systems have >50% of VG
73 space unallocated because people always overestimate what they really need.
74
75 Separate partitions means I can set up separate mount options
76 (nosuid,nodev etc ...)
77
78 Like William, I feel more cogs means more opportunities for breakage. I
79 roll my own vanilla kernel, with one or two of my own patches.  I roll
80 my own init script for initramfs.  Why - because system bootability is
81 of utmost importance.  So I absolutely have to KNOW that it'll work, and
82 when it doesn't that I can walk someone through fixing it over the
83 phone. Never used the default gentoo initramfs.  genkernel always pulls
84 in stuff I don't want nor need.  Sometimes it's just simpler I guess. 
85 But that's the thing - Gentoo gives me CHOICE.  Which I don't get from
86 other distributions.  These are not the kind of things I like to leave
87 to chance, or in the hands of a continuously changing tool (eg, dracut
88 as mentioned by William).
89
90 Recently we ran into a bug causing filesystem corruption on /usr/portage
91 (for some reason it was always under /usr/portage). With /usr on a
92 separate partition we could recover that by setting appropriate boot
93 options *remotely*.  No need to drive out.  Some of these systems were
94 >100km away.
95
96 So another motivation for separate / and /usr for me is that / is much
97 more read-heavy than /usr, and as such, with the lower write ratio lower
98 risk of corruption.  Better ability to recover.
99
100 As to why not use the initramfs - simple:  It's not needed.
101
102 The only potential advantage in my opinion is if you can build a
103 recovery system in there that's small enough, and contains all
104 conceivable tools required to recover from just about any boot failure. 
105 Work-in-progress I guess.
106
107 I'm not sure that answers your question, and this is one of those
108 debates that can run in circles for hours, days and even weeks. So I
109 hope I at least managed to give you some insight into my thinking and
110 reasoning.
111
112 Kind Regards,
113 Jaco