1 |
Hi, everyone. |
2 |
|
3 |
TL;DR I think I came up with a feasible plan towards cleanly removing |
4 |
dev-python/namespace-* packages that aren't technically needed anymore |
5 |
with modern versions of Python. |
6 |
|
7 |
|
8 |
What are namespace packages? Let's take "zope" as an example. |
9 |
Normally, various subpackages like "zope.interface" and |
10 |
"zope.configuration" would have all to be present in a single directory, |
11 |
that is inside the "zope" package. However, if "zope" is made a |
12 |
namespace package, its subpackages can be loaded from different |
13 |
locations. Notably, in order to test zope.configuration prior to |
14 |
installing it, we load it from build tree but its dependency |
15 |
zope.interface from system site-packages. |
16 |
|
17 |
Historically, for this to work, the "zope/__init__.py" had to specify |
18 |
some magic incantations. On Gentoo, we've added dev-python/namespace-* |
19 |
that installed these magical "__init__.py" files and made all |
20 |
subpackages (e.g. dev-python/zope-*) depend on it. |
21 |
|
22 |
However, Python 3.3 introduced PEP 420 implicit namespaces that made |
23 |
magical incantations unnecessary -- if "zope" didn't include |
24 |
"__init__.py", it was implicitly treated as a namespace. Unfortunately, |
25 |
there were some interoperability issues between the two kinds of magical |
26 |
incantations and implicit namespaces. So while we were able to retire |
27 |
pkgutil-style namespaces, setuptools-style namespaces remained. |
28 |
|
29 |
I think we can change that now. |
30 |
|
31 |
|
32 |
I've done some experiments, particularly with dev-python/zope-interface |
33 |
and dev-python/zope-configuration. If nobody finds a problem with this |
34 |
solution, I think we can aim to bump all packages relying on namespaces |
35 |
and retire them in 1-2 months. |
36 |
|
37 |
The rough idea is to remove RDEP on dev-python/namespace-* from |
38 |
the package, and updating python_test to use a temporary pkgutil-style |
39 |
incantation (yes, for setuptools-style packages), e.g.: |
40 |
|
41 |
``` |
42 |
python_compile() { |
43 |
distutils-r1_python_compile |
44 |
find "${BUILD_DIR}" -name '*.pth' -delete || die |
45 |
} |
46 |
|
47 |
python_test() { |
48 |
cd "${BUILD_DIR}/install$(python_get_sitedir)" || die |
49 |
# this is needed to keep the tests working while |
50 |
# dev-python/namespace-zope is still installed |
51 |
cat > zope/__init__.py <<-EOF || die |
52 |
__path__ = __import__('pkgutil').extend_path(__path__, |
53 |
__name__) |
54 |
EOF |
55 |
eunittest |
56 |
rm zope/__init__.py || die |
57 |
} |
58 |
``` |
59 |
|
60 |
With this hack, I've been able to get the tests to work with all |
61 |
the possible scenarios, i.e.: |
62 |
|
63 |
a. with and without the package in question installed, |
64 |
|
65 |
b. with and without dev-python/namespace-* installed. |
66 |
|
67 |
The second point means that we can transition gradually. Once all |
68 |
packages with namespace-* dep are gone, we can lastrite that package |
69 |
and add blockers to trigger cleanup from user systems. We'll have to |
70 |
keep the test hack for some more time but I don't think that's a deal |
71 |
breaker. |
72 |
|
73 |
|
74 |
WDYT? |
75 |
|
76 |
-- |
77 |
Best regards, |
78 |
Michał Górny |