[gentoo-dev] python-r1.eclass: the masterplan (dirty RFC)

[Michał Górny]

Hello,

As I was asked to reveal the details of what is my plan for python-r1,
I'd like to submit this short RFC explaining the ideas in more detail.
I'm a bit short on time currently, therefore I will focus only
on drafting the overall steps which I would like to do in the near
future.

I'd like to note that there are a few forks in the road of python-r1,
and I'm quite open to choosing either path satisfying our developer
and user needs better.


General topics:

1. Developing and improving documentation for python-r1;
2. Testing basic python-r1 ideas and gaining feedback;
3. Solving the dependency-compatibility problem with python.eclass;
4. Adding missing features to python-r1;
5. Migrating packages in the tree.


Developing and improving documentation for python-r1
----------------------------------------------------

This is the task I'm putting most effort into recently. The idea is to
provide a good handbook both for users and for developers. So far, all
the basics should be covered already. I have a patch ready to explain
USE_PYTHON as well but I'm delaying the commit until we decide on how
to proceed with python.eclass.

  http://www.gentoo.org/proj/en/Python/python-r1/user-guide.xml
  http://www.gentoo.org/proj/en/Python/python-r1/dev-guide.xml


Testing basic python-r1 ideas and gaining feedback
--------------------------------------------------

The most important point to me right now is gaining more feedback
on the design. So far, most of the feedback was quite positive which
makes me believe the eclasses should be ready soon to be deployed for
a broader use.


Solving the dependency-compatibility problem with python.eclass
---------------------------------------------------------------

This is the problem which blocked the development for now. We have to
decide how to proceed with python.eclass packages depending
on python-r1 packages, and the other way around.

There are two ways of solving this: either providing users
with the necessary information and relying on them not to break their
systems, or modifying python.eclass behavior.


There were considerations that PYTHON_TARGETS variable should be
dropped in favor of modifying the syntax of USE_PYTHON to match
the needs of USE flags. Despite the opinion of my counterpart, I do not
see a benefit here since:

a) PYTHON_TARGETS were announced, documented and deployed already.
A fair number of users set it. Requesting them to rollback the changes
would be at least unfortunate;

b) changing syntax of USE_PYTHON in any way will require users to
modify their make.conf files anyway. There is no sane way of avoiding
this. The solution based on a new variable is safer and cleaner.


Anyway, the simple solution is to preserve the current python.eclass
behavior and put the necessary docs in-place. Most importantly,
the python-r1 User's Guide would document how to proceed with various
values of PYTHON_TARGETS. At some point (when python.eclass
is deprecated), the additional documentation sections would just
be dropped.

Advantages:
+ less work for us,
+ works fine on default installations (just necessary to eselect
  the correct py2/py3 version),
+ can be done sanely.

Disadvantage:
- requires attention from users.

The solution could be partially improved through adding some warnings
in-place. The python-r1 eclass could warn when USE_PYTHON misses
a requested implementation; the other way would be harder to achieve
since portage filters USE_EXPAND variables.


Alternative: modifying python.eclass to respect PYTHON_TARGETS. Since
USE_EXPAND variables are filtered by portage (and in EAPI 5), we can't
really do this without adding PYTHON_TARGETS to packages' IUSE.

So, the step is effectively: make python.eclass export and respect
PYTHON_TARGETS. Deprecate USE_PYTHON; get rid of PYTHON_ABIS.

Advantages:
+ just works™,
+ we could probably get rid of python-updater,
+ we could remove python3 from defaults (the current python.eclass
  behavior is the only reason we keep it there).

Disadvantages:
- needs modifying python.eclass behavior,
- will affect stable packages directly,
- all users will be forced to rebuild all Python packages,
- PYTHON_ABIS will not be respected, so users will be limited
  to implementations listed in profiles),
- potential of unforeseen issues.


I'm fine with either way. However, if the latter is chosen, we'd need
someone who can at least double-check changes to python.eclass and do
a serious bit of testing.


Adding missing features to python-r1
------------------------------------

This is probably the point most of you are interested in. The python-r1
is intended to become a complete replacement for python.eclass,
and thus I'm intending to finally add all features necessary.

The most important points I see here is:


1) splitting common functions into python-utils-r1.

The python-utils-r1 eclass would provide means to work with specific
Python packages like portage. Unlike python-r1, it would not export
IUSE or require any specific USE flags.

I'm not insisting on this nor giving it a very high priority. It is
a straightforward change since python-r1 will inherit the new eclass
anyway.


2) support for packages not supporting multiple Python implementations.

So far, I haven't got much of a feedback in this area. I will probably
proceed with REQUIRED_USE="^^ ( ... )". If we decide to go the 'hard'
way of modifying python.eclass and drop Python 3 from default
PYTHON_TARGETS, this will be painless to most of our users. However, it
will require some of them to explicitly request an appropriate Python
version from the package.

Optionally, I'm considering adding a wrapper eclass for that.
The eclass would just add the REQUIRED_USE and export pkg_setup()
setting up the requested Python implementation (like python.eclass
does).


3) adding major and minor features.

This one's a bit general since those features can be added without
changing the eclass API in general.

a) functions to install and compile Python modules,
b) function to install Python scripts,
c) support for includedir (or cflags?) and other desired info
   in python_export and getters.

If any other features are requested, I will gladly add them.


Migrating packages in the tree
------------------------------

I believe that so far python-r1 has received enough attention to be
considered a good replacement for python.eclass. I'd prefer a smooth
transition over a rough replacement, therefore I believe the correct
way of proceeding now is migrating packages in version or revision
bumps for now.

Firstly, the dependency issue needs to be addressed. Afterwards,
the developers can start migrating their packages on their own will
and preference. When a significant amount of packages is migrated,
the process of deprecating python.eclass should start.

I'm not sure if we should ever completely drop python.eclass. We should
however be able to reach the stage when the tree has no longer any
packages dependent upon python.eclass.


Do you have any other questions? I'd appreciate also opinions
on the optional parts, and the decisions which need to be made.

-- 
Best regards,
Michał Górny