Re: [gentoo-dev] usr merge

[Alexis Ballier]

On Tuesday, April 5, 2016 2:26:53 PM CEST, Duncan wrote:
> Alexis Ballier posted on Tue, 05 Apr 2016 12:10:51 +0200 as excerpted:
>
>> On Tuesday, April 5, 2016 3:19:59 AM CEST, William Hubbs wrote:
>> [...] ...
>
> As I said in the other thread, I'm running merged /usr and bin/sbin here,
> except that I merged them the other way, with /usr -> . so everything in
> /usr is now in /.
>
> Portage has long "just worked" in that regard, tho I've no idea whether
> the other PMs do.  Portage has enough intelligence to avoid replacing a
> file with a symlink pointing to it (and thus to itself once the
> replacement is done), regardless of which way the directory symlinks
> point.
>
> As such, coreutils "just works".  If the two would end up in the same
> canonical location, the file wins and the symlink isn't installed.

What about the unlikely case with two files ?


> There are a few individual package bugs, including one open right now
> where the gcc-config ebuild does an unconditional rm -f of any old
> versions found in its old sbin location, even when it removes the
> executable it just installed into the bin location, because they're the
> same canonical location.  (Bug number for that and other bugs in the
> reply on the other thread.)  And cmake can get mixed up in some instances
> so a few packages (baloo) have problems with some cmake versions.
>
> But the bugs aren't with portage, they're with the ebuild or the upstream
> sources, and the number of them I've run into in the two years plus I've
> been running merged can fit on one hand.  Certainly, they're few enough
> to deal with on a case-by-case basis.

Yeah, these cases need to be handled on a case by case basis, there's no 
other choice anyway :)

If we want to move on on this, we should definitely track these properly.

[...]
> 1) Unless one is sure of the actual install path used and uses it, equery
> belongs and I assume q and similar tools with the same query, need the
> bare file name, not the full path, because you might use the wrong one.
>
> For instance, /bin/grep, /sbin/grep, /usr/sbin/grep, and /usr/bin/grep,
> are all the same file due to directory level symlinks.  However, if you
> try equery belongs with all four paths, only one of them will return the
> hit you're looking for.
>
> Easily solved by simply doing equery belongs grep (no path), and letting
> equery supply the installed path in its results.  That's actually how I
> find out which path a file was actually installed to, these days, as
> well. =:^)


no real issue here, and anyway, since it parses portage tree & profiles, 
support for guessing usr-merge can be added to restore old behavior

> 2) revdep-rebuild will, in its default configuration, end up processing
> files using all paths.  So grep, to use the same example as above, will
> be processed four times, one each for /bin and /sbin, /usr/bin and /usr/
> sbin.
>
> While it's possible to reconfigure revdep-rebuild to ignore three of
> those paths and only check one of them (and similarly, ignore one of
> /lib64 and /usr/lib64, etc), doing so will result in revdep-rebuild
> complaining about unowned broken binaries if they're installed using a
> different path than the one it processed.
>
> That's not a big problem, because equery belongs <file> (without the
> path) will tell you what owns it as well as the installed path it used,
> and then that package can be remerged manually, if needed.


kind of defeats the point of revdep-rebuild though :)

[...]
>> # Create the /usr merge compatibility symlinks for dir in /bin /sbin; do
>> run_command rm -rf $dir
>>
>> ---> where are the 'ln' and 'rm' taken from after this step ?
>> If this fails here, you're also leaving the system in a broken state.
>>
>> run_command ln -s usr/bin $dir done
>
> In my case I was using the mc binary, which continued to run after the
> transfer, so it wasn't an issue.
>
> But using the individual ln and rm binaries, while they'll still be on
> the path, you may need to run hash -r in the shell so it forgets the old
> location and checks the path again.
>
> Similar thing for the libs, since the lib cache will be screwed after the
> move, until the symlink has been created so the old paths work again, at
> least.  In my case I was using the mc binary which continued to run and
> thus could be used to create the symlink, but for one-shot executables
> like ln, that could be an issue.
>
> One way around the problem would be to create a few static-linked
> executables for the purpose, and ship them in a tarball that's unpacked
> to an unchanging tmp location for the run, so they can be called from
> there to finish up regardless of whether the dynamically linked normal
> executables can still be invoked.
>
> Smart use of the shell's builtin read, echo and redirection could
> probably do some of it too, but can ln be emulated using shell builtins?
>


I'd rather use a binary that ensures everything that needs to be loaded is 
loaded at the beginning and even better if it can enusre system consistency 
and can do rollbacks in case of failure.

A python script would probably work if there is no dynamic import, or a C 
version that doesnt use dlopen() nor execv() & friends. It's probably even 
easier to check that / and /usr are on the same mount point.

Alexis.