1 |
Hi, everyone. |
2 |
|
3 |
Python namespaces are an ugly hack that Python upstreams do mostly |
4 |
to look more enterprise'y and support installing subpackages of a common |
5 |
parent package in different locations (i.e. zope.*, ...). For more |
6 |
information on them, read PEP 420 [1], Python Packaging Guide [2] |
7 |
and our wiki page [3]. |
8 |
|
9 |
So far we haven't been really providing any special support for |
10 |
namespaces. We were rather relying on things magically working out |
11 |
of the box and applying distutils_install_for_testing whenever they |
12 |
weren't. However, this was really a 'big hammer' solution -- not very |
13 |
precise, not very reliable. |
14 |
|
15 |
I've been working on adding missing test dependencies of yet another |
16 |
zope-related package lately [4] and hit a few more issues with namespace |
17 |
of zope.*. It turned out that due to some random changes in Python 3.5+, |
18 |
'setup.py install --home=...' (used for d_i_f_t) no longer correctly |
19 |
established namespaces for our use and I had to use 'setup.py develop' |
20 |
instead for zope.testrunner [5]. However, even this didn't work for |
21 |
zope.exceptions where I would probably have to also deploy 'setup.py |
22 |
easy_install' or add requirements to setup.py. |
23 |
|
24 |
So I've went ahead and explored other possibilities. I've figured out |
25 |
that at the moment namespace packages work on old Python versions |
26 |
because of *.pth files installed by setuptools along with the packages. |
27 |
However, this hack might be deprecated [6]. Also, it seemed to be |
28 |
the source of problems with Python 3.5+. |
29 |
|
30 |
|
31 |
You can see what I did on the PR [4]. It boils down to: |
32 |
|
33 |
1. Adding dev-python/namespace-zope package (name selected to avoid |
34 |
collisions with potential packages) that installs setuptools-compliant |
35 |
__init__.py. |
36 |
|
37 |
2. Removing *.pth files from dev-python/zope* and making those packages |
38 |
depend on dev-python/namespace-zope instead. |
39 |
|
40 |
This done, I was able to get the tests on the 5 zope packages to pass |
41 |
on all Python implementations without any additional logic |
42 |
(in particular, without a call to d_i_f_t). |
43 |
|
44 |
|
45 |
If you consider this an acceptable solution to the namespace problem, |
46 |
I'd like to set up the following guidelines for dealing with namespaces: |
47 |
|
48 |
1. A single package should be selected to 'hold' the namespace. It can |
49 |
be a common package for the ns (e.g. dev-python/logilab-common) |
50 |
or a dedicated dev-python/namespace-* package. |
51 |
|
52 |
2. This package should install appropriate __init__.py for the |
53 |
namespace. The contents should be selected to match the namespace logic |
54 |
used by the packages (pkgutil vs setuptools). |
55 |
|
56 |
3. All subpackages in the namespace should RDEP (+ DEP) on this package, |
57 |
and not install *-nspkg.pth files. |
58 |
|
59 |
|
60 |
Your thoughts? |
61 |
|
62 |
|
63 |
[1]:https://www.python.org/dev/peps/pep-0420/ |
64 |
[2]:https://packaging.python.org/namespace_packages/ |
65 |
[3]:https://wiki.gentoo.org/wiki/Project:Python/Namespace_packages |
66 |
[4]:https://github.com/gentoo/gentoo/pull/4519 |
67 |
[5]:https://github.com/pypa/setuptools/issues/912 |
68 |
[6]:http://setuptools.readthedocs.io/en/latest/setuptools.html#transitional-note |
69 |
|
70 |
-- |
71 |
Best regards, |
72 |
Michał Górny |