[gentoo-dev] /usr vs. initramfs redux

[Robin H. Johnson]

I've mainly said out of this discussion until now, because I've been
quite busy.

The root problem here is that there are starting to be a lot of cases
where rule run by udev require that /usr [1] and potentially /var [2] or
more are available when the udev rule runs.

To the best of my knowledge, udev has no way of knowing that the
relevant directory structures might not yet be available.

Moving the binaries out of /usr into / is a band-aid at best, and
potentially a giant nightmare because we have to trace down every binary
run by every rule. Then data-files also need to be considered.

There are only two solutions available, one of which has two variants.
1. Allow udev to delay running a rule until the volumes are available.
2. Ensure /usr and /var are available when udev starts, two possible
   ways:
2.1. Merge /usr and /var to /.
2.2. Initramfs to mount them.

Gregkh et al can speak to #1, and I think they will concur with me that
it's not a reasonable change to put in udev (it belongs in whatever is
starting the services and mounting those directory structures).

That leaves the only reasonable solution as #2. In terms of minimal
impact, I propose that we offer users with a static system an absolutely
minimal initramfs, that _just_ mounts the required directories. No
modules, no LVM, no MD, no crypto etc - if you want that functionality,
go and use genkernel or dracut. If your fstab contains a line like:
/dev/sdXN /usr ...
Then this initramfs is for you.

The minimal initramfs would do the following.

1. Mount devtmpfs/sysfs/procfs as needed to access devices.
2. Mount real_root to /newroot
3. Read /newroot/etc/initramfs.mount and /newroot/etc/fstab
4.1. If /newroot/etc/initramfs.mount does not exist
     Assume it contains only: /usr /var
5. Mount the combined items from said files
6. pivot_root.

This initramfs explicitly needs to be rebuilt extremely seldom, because
it reads everything from the real root. (Having the debug interface like
genkernel initramfs would be nice in case the user breaks their fstab,
but isn't required).

There's a couple of other tweaks/improvements that might be useful, like
supporting UUIDs and LABELs for devices.

[1]:
Besides the binaries run from /usr, I've seen broken rules because they
failed USB/PCI identification, because /usr/share/misc/pci.ids and
/usr/share/misc/usb.ids were on /usr.

[2]:
These are the ones on my desktop that reference /var:
app-admin/system-config-printer-common: /lib/udev/udev-configure-printer
sys-fs/udev: /lib/udev/udev-acl
sci-geosciences/gpsd: /lib/udev/gpsd.hotplug
/lib/udev/rules.d/90-alsa-restore.rules: 
- alsactl restore needs /var/lib/alsa/asound.state




On Thu, Aug 04, 2011 at 12:00:09PM -0400, Rich Freeman wrote:
> On Thu, Aug 4, 2011 at 11:46 AM, Greg KH <gregkh@g.o> wrote:
> > Also, again, this is an upstream issue, based on the packages you have
> > installed, not anything that has changed in the distro itself.  Upstream
> > is also working to resolve the issue already, by mounting /usr from the
> > initramfs, to keep this sane, so people shouldn't really have to worry
> > about this too much, unless they don't use an initramfs...
>
> Agree.  The news was a little shocking to me actually - I had no idea
> people were doing this.  I suspect it just happened as things like
> udev went from enhancements used when you inserted something in a
> pcmcia slot to core features used to get everything from your hard
> drive to your mouse to work.
>
> The more I think about it, the more it seems like we're stuck going
> the initramfs route unless we want to become Gentoo vs the world.
> Dropping support for udev clearly isn't going to be a practical
> option, and the number of changes we'd have to make to get it and its
> dependencies out of /usr is going to be a challenge.
>
> I suspect that in the end we're either going to end up requiring
> initramfs, or we're going to end up implementing what is otherwise in
> dracut in openrc to get those drives mounted much earlier.
>
> Gentoo is a bit unusual in not requiring initramfs in the first place.
>  We can get away it mostly because everybody customizes their
> kernels/grub/etc anyway.  If you're a binary distro and want a one
> bzImage/grub.conf fits all then you need a fancy initramfs to make it
> work.
>
> Rich
>

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee & Infrastructure Lead
E-Mail     : robbat2@g.o
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85