Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[William Hubbs]

This is getting really long, so I'm going to remove more things I'm not
answering directly.

On Sun, Mar 28, 2021 at 07:31:02PM -0400, Joshua Kinard wrote:
> > The problem is, there's a chicken-and-egg problem in the scenario where
> > / and /usr are on separate partitions, and this is why a number of linux
> > distros have moved to requiring an initramfs in this situation.
> > I'm linking systemd's description here, only because it is the best
> > writeup of the issue I've found [1].
> > Anything that is needed in the early boot process requires all of its libraries,
> > dependent libraries, binaries, data files, etc to be on /, and this has
> > become a moving target.
>
> Yeah, I've read systemd's explanation, and generally disagree with it.  They
> created the problem in the first place, then invented their own solution for
> it, and now everyone acts like they're the wise men on the mountain for it.

http://lists.busybox.net/pipermail/busybox/2010-December/074114.html

Rob Landley had a lot to say about how linux should be using initramfs
to handle early boot long before systemd came  along, so this has been
an issue a lot longer than systemd; the systemd guys just amplified it.

> I still don't see the connection to the static *.a libs and whether they're
> on /lib or /usr/lib, though.  Unless we're implying that where the *.a's go,
> so too do the *.so's go, then THAT makes sense because *.so's ARE needed at
> program runtime, whereas *.a's are not.

It is a linker behavior issue we discovered and  worked around.

In the past, Gentoo has gone an extra mile to make some things useable
by people who have separate /usr without an initramfs by moving only
shared libraries to /lib*. We leave everything else that would be in
libdir in the default location which is /usr/lib*. This includes things
like static libs, pkgconfig files etc.

When we started doing this we found that the linker favors libraries in
/usr/lib* over libraries in /lib*, so if you are linking to libfoo and
there's a static version in /usr/lib* and a shared version in /lib* you
will link to the static version. That's what bug 4411 is about. 

gen_usr_ldscript gets around this by creating a linker script at the
original location of the shared library in /usr/lib* when it moves the
shared library to /lib*.
See /usr/lib64/libbz2.so for an example of the generated linker script.

Getting rid of gen_usr_ldscript would move the shared libraries back to
their upstream location (/usr/lib*) and remove the linker scripts. This
would also allow the removal of the usr-ldscript eclass and the
gen_usr_ldscript function from eutils.eclass.

The up side of this is that it allows us to get rid of some of our
custom code in ebuilds, and it is completely transparent to most of our
user base.

The down side is that if you are using separate /usr with no
initramfs, libraries that you were accessing in /lib* during early boot will
moved to /usr/lib* and not be available before /usr is mounted.
This would cause breakage until you reconfigured your system to boot
with an initramfs and mount / and /usr before jumping into the real
system.

> I wonder if we couldn't shovel all static libs off to a dedicated folder
> somewhere, like '/usr/lib/static/<pkg>/*.a', similar to the way debug files
> are now consolidated under '/usr/lib/debug'.  Since they're only needed
> during a specific kind of compilation that we don't support out-of-the-box
> that happens long after the system is fully booted, stuffing them off
> somewhere unimportant would make some sense.  Most modern software should be
> using shared libs by default, and if it ain't, that's either a bug or that
> software is for a very specific function (like a bootloader).

If you started creating separate static libs folders you would need one per
abi, so it would end up being pretty ugly. We don't build that many
static libraries right now, so I'm not sure it is worth moving them to
some other folder.

If we put the shared libs back in the upstream expected location
(/usr/lib*) we would eliminate the linker issue.

*snip*

> > The way I see it, when we start to remove the gen_usr_ldscript calls,
> > people using a sep-usr mount without an initramfs will run into one or
> > both of these issues:
> >
> > - they might have to increase the size of their root partition depending
> >   on what gets added to /lib*
> > - if one package in that list drops gen_usr_ldscript without installing
> >   libraries in /lib*, it will mean they need an initramfs.
>
> I tend to make my root partitions ~4GB, which has often been plenty of room
> for well over 15 years.  But again, location of the *.a static libs is
> irrelevant during system boot.  They are not needed nor referenced when a
> program executes.  A statically-compiled program has all of its dependencies
> lumped inside of it, so you could put it pretty much anywhere on the
> filesystem and run it (ignoring for a moment 'noexec' potentially being
> set).  Or even more to the point, you could e-mail a static binary to
> someone else on a different distro and it'll probably execute just fine.

I think I wasn't very clear. In the second bullet point above I meant
shared libraries, which leads to what you say below.

> The important bit here is certain software packages defaulting their install
> into /usr that are needed to bring the system up (like encryption, lvm,
> etc).  It's when using those setups that you pretty much have to use an
> initramfs, sometimes regardless if you have /usr on another partition or not.
 
 Correct, and this is the issue. Gentoo has done extra work in the past
 to make more things useable  without an initramfs, and a good part of that
 work is in the gen_usr_ldscript function.

If I start removing calls to that function, more shared libraries will
move back to /usr/lib*, breaking systems that use separate /usr without
initramfs.

If I override the build systems of these packages and install all of the
library info in /$(get_libdir), I would have to develop custom code to
move unnecessary things back to /usr/$(get_libdir). This would also be a
bandaid because separate /usr without initramfs would be impossibly
broken if we do the /usr merge eventually.

If I do go forward with removing gen_usr_ldscript, I want to be reasonable
about how I do it, so let me know what your thoughts are about how it should
happen.

Let me know what you think.

William