Re: [gentoo-dev] [RFC] python-r1 + distutils-r1, v2 (+ comparison)

[Reinis Danne]

On Mon, Oct 08, 2012 at 05:16:05PM +0200, Michał Górny wrote:
> Hello,
>
> This is the second and hopefully last version of python-r1 +
> distutils-r1 eclasses before committing. I would also like to shortly
> point out the goals and the differences between various python eclasses
> in Gentoo.
>
> Both are attached. For those who prefer hosted form:
> https://bitbucket.org/mgorny/gx86-working-tree/src/5a2d39e69d6d/gx86/eclass/python-r1.eclass
> https://bitbucket.org/mgorny/gx86-working-tree/src/5a2d39e69d6d/gx86/eclass/distutils-r1.eclass
>
> Changes from the previous version:
> - support for DOCS and HTML_DOCS (alike EAPI 4);
> - minimal support for passing arguments to setup.py (through
>   distutils-r1_python_compile & distutils-r1_python_install);
> - EPYTHON values adjusted to match python.eclass;
> - scripts are -${EPYTHON} suffixed -- i.e. foo-python2.7,
>   foo-pypy-c1.8;
> - a wrapper script[1] is used to choose the correct script instead of
>   constant symlink. Thus, EPYTHON & eselect-python are respected,
> - PYTHON_COMPAT can be an array only.
>
> What still needs considering:
> - a nicer way of passing setup.py arguments (mysetuppyargs=()?,
>   esetuppy wrapper?);
> - Prefix support.
>
> [1]:https://bitbucket.org/mgorny/python-exec
>
>
> Now, for the differences part.
>
> Goals
> -----
>
> python.eclass aims to support every single Python package out there,
> including rare corner cases supported in-eclass. It supports both
> packages supporting one and multiple Python implementations. distutils
> support is provided by second eclass, and both provide phase functions
> (python.eclass providing default ph.f. wrappers).
>
> python-distutils-ng aims to support majority of Python packages using
> distutils. With a bit of hackery it can support non-distutils packages,
> but the use is limited to the common cases. In an uncommon case, it's
> not flexible enough. It supports Python packages supporting multiple
> implementations only.
>
> python-r1 aims mostly to provide tools to support majority of Python
> packages. It tries to be simple & flexible, thus allowing handling
> various corner cases without special branches of code in the eclass.
> It doesn't export any phase functions, nor set dependencies by default
> (just provides a variable with them).
>
> Right now, it supports packages supporting multiple implementations
> only; if necessary, the support will be extended -- either through
> additional code if that wouldn't add too much complexity, or through
> additional eclass.
>
> It is accompanied by distutils-r1 which provides a simple overlay for
> the very common case of distutils packages. This eclass exports phase
> functions and sets dependencies implicitly. It also handles renaming
> the distutils-installed scripts and linking the python-exec wrapper.
>
>
> Python targets
> --------------
>
> python.eclass uses implicit Python target specifications. It provides
> no ebuild-transparent way of enabling/disabling them.
>
> python-distutils-ng and python-r1 use USE flags to explicitly list
> enabled Python implementations. Both eclasses use the same values for
> PYTHON_TARGETS USE_EXPAND.
>
>
> Installed scripts
> -----------------
>
> python-distutils-ng rework the installed scripts creating copies for
> Python implementations and changing the shebang. The created copies
> don't differ any other way. The old script name is then symlinked to
> the version for default implementation.
>
> python.eclass & python-r1 instead let distutils rework the scripts
> and just rename them before merging. A wrapper is used to choose
> the correct version, respecting EPYTHON & eselect-python.
>
> python.eclass installs the files to separate installation images
> and merges them. python-r1 installs them to the main image directly,
> renaming the installed scripts between successive implementations.
>
> python.eclass creates a wrapper script for each package. The script is
> written in python. python-r1 instead installs the compiled wrapper
> once (in an ebuild), and symlinks it for the packages. The wrapper is
> written in much simpler C.
>
> The implementation suffixes for all three eclasses are different:
> - python.eclass uses -2.6, -2.7 for Python, -2.7-pypy-1.8,
>   -2.7-pypy-1.9 for pypy and -2.5-jython for jython (enjoy!);
> - p-d-ng uses Python target names (-python2_6, -python2_7, -pypy1_8),
> - p-r1 uses $EPYTHON values (-python2.6, -python2.7, -pypy-c1.8 (sic!)).
>
>
> Dependency on Python
> --------------------
>
> python.eclass has a few variables necessary to set dependencies
> on Python implementation, including internal sub-syntax (and an even
> more complete generator sub-syntax in progress overlay). I suspect that
> most of the possible variants can be achieved using it, just please
> don't make me try learning it all.
>
> Additionally, packages supporting multiple Python implementations are
> required to specify their supported Python versions twice -- with
> PYTHON_DEPEND and RESTRICT_PYTHON_ABIS.
>
> USE flag dependencies are specified using three variables, and I'd like
> to avoid getting anywhere deeper.
>
> p-d-ng has a very simple dependency setup. It has a variable for
> listing supported Python versions and a variable to make Python
> dependencies conditional under the 'python' flag. Lately, it has been
> added a method of listing requested USE flags.
>
> p-d-ng does not provide any support for more complex dependency
> specifications, nor a way to disable adding default Python dependencies.
>
> python-r1 does not append any dependencies by default, and instead
> exports them in a variable. The variable can be used to easily express
> simple cases, and avoids polluting the ebuild in more complex cases. It
> also supports providing USE dependencies for the implementation.
>
> distutils-r1 adds the Python dependencies to RDEPEND & DEPEND;
> additionally, it adds a dependency on python-exec.
>
>
> Dependencies on other packages
> ------------------------------
>
> As python.eclass (as of gx86 state) does not express enabled
> implementations explicitly, it is not possible to request those
> implementations from other packages. The dependencies on other Python
> packages are thus simple.
>
> p-d-ng does not provide any simpler way of writing dependencies
> on other Python packages using the eclass. Thus, the necessary USE
> dependencies have to be written by hand.
>
> python-r1 does provide a simple PYTHON_USEDEP variable which can be
> used in dependency atoms to request the same Python implementations
> being enabled.
>
>
> Phase functions for distutils
> -----------------------------
>
> distutils.eclass allows specifying additional arguments to setup.py.
> src_install() installs default documentation files (*unconditionally*)
> and ${DOCS}, it also does installation image magic.
>
> p-d-ng has mixed unconditional and user-overridable stuff. Copying
> sources is done unconditionally; redoing scripts can be disabled using
> a separate variable. setup.py invocations are overridable.
>
> d-r1 splits each phase into 'global' and 'per-implementation' stages
> which are both overridable. The default stages apply PATCHES (and user
> patches), copy sources, run setup.py, install DOCS & HTML_DOCS and link
> python-exec wrappers.
>
>
>
> Thank you for reading up to this. I'd appreciate any comments, ideas,
> and especially any API suggestions while the eclass still ain't live.

Hi!

Do you have any examples of ebuild conversion for non-distutils
using ebuild? I was reading trough these threads but could find
only examples for distutils-r1.

I have a package which requires for each supported python
version to know python executable, include dir and library to
build python bindings (uses cmake) and location for
site-packages so I can manually install them.

Of these there is function only for executable
(python_get_PYTHON). I hope there is a reasonable way to provide
the rest of them without the need for me to resort on guessing
the locations.


Reinis