1 |
On 09/04/2022 18.20, Michał Górny wrote: |
2 |
> Hi, everyone. |
3 |
> |
4 |
> TL;DR I think I came up with a feasible plan towards cleanly removing |
5 |
> dev-python/namespace-* packages that aren't technically needed anymore |
6 |
> with modern versions of Python. |
7 |
> |
8 |
> |
9 |
> What are namespace packages? Let's take "zope" as an example. |
10 |
> Normally, various subpackages like "zope.interface" and |
11 |
> "zope.configuration" would have all to be present in a single directory, |
12 |
> that is inside the "zope" package. However, if "zope" is made a |
13 |
> namespace package, its subpackages can be loaded from different |
14 |
> locations. Notably, in order to test zope.configuration prior to |
15 |
> installing it, we load it from build tree but its dependency |
16 |
> zope.interface from system site-packages. |
17 |
> |
18 |
> Historically, for this to work, the "zope/__init__.py" had to specify |
19 |
> some magic incantations. On Gentoo, we've added dev-python/namespace-* |
20 |
> that installed these magical "__init__.py" files and made all |
21 |
> subpackages (e.g. dev-python/zope-*) depend on it. |
22 |
> |
23 |
> However, Python 3.3 introduced PEP 420 implicit namespaces that made |
24 |
> magical incantations unnecessary -- if "zope" didn't include |
25 |
> "__init__.py", it was implicitly treated as a namespace. Unfortunately, |
26 |
> there were some interoperability issues between the two kinds of magical |
27 |
> incantations and implicit namespaces. So while we were able to retire |
28 |
> pkgutil-style namespaces, setuptools-style namespaces remained. |
29 |
> |
30 |
> I think we can change that now. |
31 |
> |
32 |
> |
33 |
> I've done some experiments, particularly with dev-python/zope-interface |
34 |
> and dev-python/zope-configuration. If nobody finds a problem with this |
35 |
> solution, I think we can aim to bump all packages relying on namespaces |
36 |
> and retire them in 1-2 months. |
37 |
> |
38 |
> The rough idea is to remove RDEP on dev-python/namespace-* from |
39 |
> the package, and updating python_test to use a temporary pkgutil-style |
40 |
> incantation (yes, for setuptools-style packages), e.g.: |
41 |
> |
42 |
> ``` |
43 |
> python_compile() { |
44 |
> distutils-r1_python_compile |
45 |
> find "${BUILD_DIR}" -name '*.pth' -delete || die |
46 |
> } |
47 |
> |
48 |
> python_test() { |
49 |
> cd "${BUILD_DIR}/install$(python_get_sitedir)" || die |
50 |
> # this is needed to keep the tests working while |
51 |
> # dev-python/namespace-zope is still installed |
52 |
> cat > zope/__init__.py <<-EOF || die |
53 |
> __path__ = __import__('pkgutil').extend_path(__path__, |
54 |
> __name__) |
55 |
> EOF |
56 |
> eunittest |
57 |
> rm zope/__init__.py || die |
58 |
> } |
59 |
> ``` |
60 |
|
61 |
I quite like that we make the "hack" more local, just in test phase. But |
62 |
I think this code is quite error prone, and an easy mistake can be made |
63 |
if someone copies it incorrectly. |
64 |
|
65 |
Is it possible to add an eclass function to auto create and cleanup this |
66 |
file, and all I need to do is to pass it the correct namespace ("zope" |
67 |
in above example)? |
68 |
|
69 |
Also, by using a common function name, it would be easy to fix it if we |
70 |
did a mistake and find all consumers when we feel safe to cleanup this code. |
71 |
|
72 |
One point of failure for this function is how to do the auto cleanup? I |
73 |
was thinking using environment variables to hold our added magic names, |
74 |
which is created before `python_test` call and cleans after it is done. |
75 |
I really prefer if we could call it during prepare phase (so we can |
76 |
still use auto generated d_e_t calls). |
77 |
|
78 |
> ... |
79 |
> |
80 |
> WDYT? |
81 |
> |
82 |
|
83 |
|
84 |
-- |
85 |
Arthur Zamarin |
86 |
arthurzam@g.o |
87 |
Gentoo Linux developer (Python, GURU, Arch Teams) |