Re: [gentoo-portage-dev] [PATCH] Add QA check for unresolved soname dependencies (bug 704320)

[Zac Medico]

On 1/5/20 10:48 PM, Michael 'veremitz' Everitt wrote:
> On 06/01/20 06:38, Zac Medico wrote:
>> Example output for maven-bin-3.6.2 (bug 704618):
>>
>>  * QA Notice: Unresolved soname dependencies:
>>  *
>>  * 	/usr/share/maven-bin-3.6/lib/jansi-native/freebsd32/libjansi.so: libc.so.7 libutil.so.9
>>  * 	/usr/share/maven-bin-3.6/lib/jansi-native/freebsd64/libjansi.so: libc.so.7 libutil.so.9
>>  *
>>
>> This warning comes up when a library or executable has one or
>> more soname dependencies (found in its NEEDED.ELF.2 metadata)
>> that could not be resolved by usual means. If you run ldd
>> on files like these then it will report a "not found" error
>> for each unresolved soname dependency. In order to correct
>> problems with soname dependency resolution, use one or more
>> of the approaches described in the following sections.
>>
>> Content of the NEEDED.ELF.2 metadata file may be useful
>> for debugging purposes. Find the NEEDED.ELF.2 file in the
>> ${D}/../build-info/ directory after the ebuild src_install
>> phase completes, or in the /var/db/pkg/*/*/ directory for an
>> installed package. Each line of the NEEDED.ELF.2 file contains
>> semicolon separated values for a single ELF file. The soname
>> dependencies are found in the DT_NEEDED column:
>>
>>  E_MACHINE;path;DT_SONAME;DT_RUNPATH;DT_NEEDED;multilib category
>>
>> External dependencies
>>
>> For packages that install pre-built binaries, it may be possible
>> to resolve soname dependencies simply by adding dependencies
>> for one or more other packages that are known to provide the
>> needed sonames.
>>
>> Removal of unecessary files
>>
>> For packages that install pre-built binaries, it may be possible
>> to resolve soname dependencies simply by removing unnecessary
>> files which have unresolved soname dependencies. For example,
>> some pre-built binary packages include binaries intended for
>> irrelevant architectures or operating systems, and these files
>> can simply be removed because they are unnecessary.
>>
>> Addition of DT_RUNPATH entries
>>
>> If the relevant dependencies are installed in a location that
>> is not included in the dynamic linker search path, then it's
>> necessary for files to include a DT_RUNPATH entry which refers
>> to the appropriate directory. The special $ORIGIN value can
>> be used to create a relative path reference in DT_RUNPATH,
>> where $ORIGIN is a placeholder for the directory where the
>> file having the DT_RUNPATH entry is located.
>>
>> For pre-built binaries, it may be necessary to fix up
>> DT_RUNPATH using patchelf --set-rpath. For example, use
>> patchelf --set-rpath '$ORIGIN' if a given binary should link
>> to libraries found in the same directory as the binary itself,
>> or use patchelf --set-rpath '$ORIGIN/libs' if a given binary
>> should link to libraries found in a subdirectory named libs
>> found in the same directory as the binary itself.
>>
>> For binaries built from source, a flag like
>> -Wl,-rpath,/path/of/directory/containing/libs will create
>> binaries with the desired DT_RUNPATH entry.
>>
>> Bug: https://bugs.gentoo.org/704320
>> Signed-off-by: Zac Medico <zmedico@g.o>
>> ---
> <snip>
>
> Looks awesome!
>
> One comment: would it be helpful for developers/maintainers to have some
> indication of what's going on here, and what attempts portage is making in
> this regard?

Effectively, it's as though portage runs ldd on all the files installed
by the ebuild, and reports any unresolved soname dependencies. If
portage doesn't agree with ldd then it's a portage bug. If this happens
then I'd expect people to simply file a bug. I would indicate a flaw in
the SonameDepsProcessor logic, an it's a fairly small amount of code:

https://gitweb.gentoo.org/proj/portage.git/tree/lib/portage/util/_dyn_libs/soname_deps.py?h=portage-2.3.84

> Something like a 'verbose' option that would spit out some
> debugging info so that /if/ something goes wrong, or the outcome wasn't
> what was intended, then this can be picked up and corrected somehow else?

Mope, just file a bug and include the exact ebuild that triggered the
unexpected results.

> I think automation is great for picking these issues up, and shining a
> light on where/why something goes wrong,

Yeah, issues like this will go completely unnoticed otherwise (until
someone uses emerge --ignore-soname-deps=n or triggers some undesirable
preserve-libs behavior like bug 265372 or bug 400979).

> but sometimes it needs a few
> minutes of human intervention to achieve the sane outcome in many
> circumstances.

Well, yeah, the whole point of the QA Notice is to solicit some human
intervention in order to resolve the issue. If the issue could be solved
without human intervention then we wouldn't need the QA Notice.

Maybe I should include "file a portage bug" as an approach to take if
none of the previously suggested resolution approaches appears to be
applicable. After all, the SonameDepsProcessor logic is almost certainly
not an entirely perfect model of reality.
-- 
Thanks,
Zac