[gentoo-dev] The gx86 multilib project -- masterplan

[Michał Górny]

Hello,

Following the success of python-r1, the batch of patches I was sending
recently and some random testing, I'd like to introduce my ideas
and plans on how multilib could be introduced to gx86 in a simple
and sane manner.

My major goal with this mail is to summarize the ideas, the problems
and all the work that has been done already.


1. The USE flags and profiles
-----------------------------

The first step in introducing multilib would be to provide the USE
flags necessary to control building for respective ABIs in profiles.
The current proposed patches based on simple amd64 multilib are
available in [1].

From profiles PoV:

1. Each multilib-capable arch set provides USE_EXPAND with relevant
   multilib profiles.

     ABI_X86="32 x32 64"
     ABI_MIPS="o32 n32 n64"

2. All the USE_EXPAND variables are hidden in the base profile, and all
   the flags are masked.

3. Every arch relevant to the particular set of multilib flags unmasks
   and forces the flag for default ABI.

     x86 -> unmasks & forces abi_x86_32
     amd64 -> unmasks & forces abi_x86_64

4. Every multilib profile unmasks the flags for remaining supported
   ABIs and unhides the variable.

     amd64 multilib -> unmasks abi_x86_32, unhides ABI_X86

[1]:http://thread.gmane.org/gmane.linux.gentoo.devel/83341


2. The USE flags and packages
-----------------------------

The packages requesting multilib support either inherit
the multilib-build [2] or its derivative (e.g. autotools-multilib),
or implement the necessary facilities by hand.

The following common rules apply (they are all handled by the eclass):

1. Each multilib-capable ebuilds adds multilib flags for all arches
   to IUSE (IUSE must be constant).

2. The relevant set of flags is enforced through profile flag masking,
   forcing and USE_EXPAND hiding. Therefore, the user only sees
   the flags which are really relevant.

   x86, amd64 no-multilib -> no flags (hidden)
   amd64 multilib -> ABI_X86="32 (64)" (latter forced)

3. The enabled set of ABIs is further filtered through MULTILIB_ABIS
   set by profile. Therefore, even with users mangling the masks they
   won't be able to build something irrelevant to the profile.

4. All the build and libdir details are handled by the multilib.eclass.
   Therefore, all ebuilds should be ready for it already.

5. If no relevant USE flags are selected (bug-case), fallback
   to default ABI is done.

[2]:http://thread.gmane.org/gmane.linux.gentoo.devel/83322


3. Inter-package dependencies
-----------------------------

The inter-package dependencies are done the best way possible --
explicitly ;). The package developers are supposed to know best which
of the dependencies need having same ABIs enabled and which don't.

1. The 'regular' multilib packages are supposed to use
   ${MULTILIB_USEDEP} from multilib-build.eclass on all dependencies
   which need multilib.

   RDEPEND="dev-libs/libfoo[${MULTILIB_USEDEP}]
       dev-libs/libbar[ssl,${MULTILIB_USEDEP}]"

   The dependencies will be required to have the same ABIs enabled
   as the package in question.

2. The specific cases can use the USE flags directly. For example,
   binary x86 packages can depend on abi_x86_32 unconditionally:

   RDEPEND="dev-libs/libfoo[abi_x86_32]"

   Due to proper profile USE flag forcing/masking, that dependency is
   valid both for amd64 multilib & real x86 system.


4. ABI-specific content
-----------------------

The gx86 multilib assumes that the majority of multilib packages will
install only 32-bit libraries and pkg-config files.

1. The installation of 32-bit data in libdir is done implicitly through
   multilib.eclass and econf awareness of ABI variable.

2. Any other content which needs to be ABI-aware (includes, *-config
   programs) need be handled by ebuilds explicitly.

3. Additionally, the autotools-multilib.eclass will ensure that headers
   installed to /usr/include are consistent between ABIs.


5. Solutions to specific problems
---------------------------------

1. x11-proto packages

Those packages install headers to /usr/include and pkg-config files
to /usr/lib64. This supposedly means that the headers could be
ABI-specific; however, so far I haven't seen a single difference.

Possible solutions:

a) check the headers by hand, move pkg-config files to /usr/share,

b) make the proto packages multilib. This will cause identical .pc
   files to be installed to lib32 & lib64 but will also enable eclass
   checks for header consistency.


2. packages which install ABI-dependent headers

e.g. libogg. The issue needs to be fixed in the specific ebuild.

a) fix the headers, e.g.:

  typedef short ogg_int16_t;
  typedef unsigned short ogg_uint16_t;
  typedef int ogg_int32_t;

to:

  #include <stdint.h>

  typedef int16_t ogg_int16_t;
  ...

b) install the header(s) in an alternate location. With packages using
   pkg-config, that would be easy.


3. packages which still reinvent pkg-config with their *-config

e.g. llvm. Really painful to solve; probably will require action both
on llvm ebuild side and consumer side.

a) long-term solution: convince upstream to use pkg-config.

b) short-term: rename the llvm-config script and use the renamed
   version for 32-bit build.


What are your thoughts?

-- 
Best regards,
Michał Górny