[gentoo-dev] [RFC] Perspectives on improving (dis-hacking) python-single-r1

[Michał Górny]

Hello,

Some aspects of the current design of python-single-r1 are gross hack. 
I'd like to discuss potential alternatives to them.


Preamble
========
For the purpose of this mail, let's establish two terms.

'Single' will refer to packages allowing the user to select 1 out of N
supported Python implementations.  This generally means programs linking
to libpython, foreign build systems and end-user apps where installing
multiple copies of the code makes no sense.  Those are the packages
using python-single-r1.

'Multi' will refer to packages allowing the user to select M out of N
supported Python implementations.  This generally means ebuilds for
Python packages (modules, extensions).  Those are the packages using
python-r1 (usually via distutils-r1).

'Single' packages can PYTHON_USEDEP-end either on other 'single'
or on 'multi' packages.  However, 'multi' packages can only
PYTHON_USEDEP-end on other 'multi' packages (because 'single' wouldn't
ever be able to satisfy >1 impl).


Current status
==============
Currently, 'single' packages use two sets of USE flags: PYTHON_TARGETS
and PYTHON_SINGLE_TARGET.  The latter is used to select
the implementation used, and the former is used only for technical
reasons.  Selected PST is required to be also enabled in PT.

All dependencies enforce both PST and PT match.  The PYTHON_USEDEP is
generally a giant hack that's meant to work both for depending
on 'single' and 'multi' packages.  I'm not convinced this was the best
choice anymore.

In single-to-single deps, PYTHON_USEDEP enforces both PST and PT flags
to match.  This means that if two packages have matching PST but for
some reason were built with different PT, users end up having to switch
PT and rebuild the package without any real change.

In single-to-multi deps, PYTHON_USEDEP enforces PT flags.  For this
reason, we need to enforce that selected PST is always present in PT,
and users always have to put both flags in package.use.

There's also a gross hack in PYTHON_USEDEP in 'multi' packages that
means to prevent multi-to-single deps.  However, the dep mismatch it
causes is not very readable.


Alternative 1: PYTHON_TARGETS only
==================================
The first alternative I'd like to explore is removing
PYTHON_SINGLE_TARGET flags.  Why were they added in the first place? 
The primary goal was to be able to set PT to 2.7+3.x without requiring
people to adjust flags for every 'single' package out there.  Plus, it
turned out very convenient for Python team members who want to enable
all PT but obviously can't do the same for PST.

Originally I brought this proposal in context of automated REQUIRED_USE
conflict resolution.  However, that was rejected by the Council.  Still,
it may start making sense again in the near future when we start
removing py2.7 and pypy2.7 support.  If we can limit PT to one
implementation, and handle the remaining packages via IUSE defaults
and package.use, this may just work.

The inconvenience to people enabling multiple PT will remain though.


Alternative 2: new dependency API
=================================
If PST is going to stay, we may look into removing PT from 'single'
packages instead.  The idea is to provide new method of generating
cross-package deps that doesn't require fake flags.

PYTHON_USEDEP would continue working through a transitional period. 
When it's entirely gone, we can remove PYTHON_TARGETS from 'single'
packages.

Single-to-single deps would switch to PYTHON_SINGLE_USEDEP, that only
enforces PST and disregards PT entirely.

Single-to-multi deps would have to use a new generator function,
$(python_gen_multi_dep ...) that would create appropriate USE-mapping
from PST to PT.

Example ebuild would have:

RDEPEND="app-foo/singlepkg[${PYTHON_SINGLE_USEDEP}]
    $(python_gen_multi_dep '
         dev-python/foo[${PYTHON_USEDEP}]
         dev-python/bar[${PYTHON_USEDEP}]
    ')"

This will generate something like:

    pst_python2_7? (
        dev-python/foo[pt_python2_7(-)]
        dev-python/bar[pt_python2_7(-)]
    )
    pst_python3_7? (
        dev-python/foo[pt_python3_7(-)]
        dev-python/bar[pt_python3_7(-)]
    )


Your opinions
=============
So, WDYT?  Do you think this approach is worthwhile?  Do you see other
options?


-- 
Best regards,
Michał Górny