1 |
Hello, fellow developers. |
2 |
|
3 |
It seems that most of the new Python eclass development has been kept |
4 |
in gentoo-python mailing list and most developers hasn't been properly |
5 |
introduced to them and lack necessary information to use them. For that |
6 |
reason, I'd like to quickly summarize what has changed and how Python |
7 |
ebuilds shall be written nowadays. |
8 |
|
9 |
First of all, I would like to apologize for the official eclass |
10 |
guides [1] being a bit out-of-date. I was short on time lately and I |
11 |
tried to use the remaining bits of it to work on ebuilds and eclasses. |
12 |
For the most recent documentation, please use eclass manpages [2]. |
13 |
|
14 |
Secondly, I'd like to make it clear that the old python.eclass is |
15 |
'almost' deprecated. We're in process of converting the in-tree |
16 |
packages to use the new eclasses but that's a lot of work [3]. |
17 |
|
18 |
When committing new packages, you are strongly encouraged to use |
19 |
the new eclasses. If you'd like to convert existing package, feel free |
20 |
to -- but please make sure to do so properly, since the new eclasses |
21 |
aren't supposed to be 1:1-compatible with python.eclass. |
22 |
|
23 |
The new eclasses (except for python-any-r1) require EAPI=5. That's |
24 |
mostly due to USE_EXPAND USE-deps being useless in PMS variant of |
25 |
EAPI<5. That should not be a problem for most of the packages since |
26 |
EAPI=5 portage is stable already and we're expecting users to migrate |
27 |
to an EAPI=5 profile. |
28 |
|
29 |
Now, a few quick details you may want to know about the new eclasses |
30 |
and which may confuse you since it's different to what python.eclass |
31 |
used to do. It's a bit late but I hope it helps you. |
32 |
|
33 |
[1]:http://www.gentoo.org/proj/en/Python/#doc_chap5 |
34 |
[2]:http://devmanual.gentoo.org/eclass-reference/index.html |
35 |
[3]:http://dev.gentoo.org/~mgorny/python-r1/conversion.xhtml |
36 |
|
37 |
|
38 |
The eclasses |
39 |
------------ |
40 |
|
41 |
There are 5 eclasses to choose from. You always inherit only one |
42 |
of the eclasses, the 'parent' ones will be pulled in for you. |
43 |
|
44 |
1) distutils-r1.eclass -- for *all* packages which use the distutils |
45 |
build system. Replacement for distutils.eclass. |
46 |
|
47 |
2) python-r1.eclass -- for packages which are being installed for |
48 |
multiple Python implementations at the same time. Replacement for |
49 |
python.eclass with SUPPORT_PYTHON_ABIS=1. |
50 |
|
51 |
3) python-single-r1.eclass -- for packages which can be built for |
52 |
one Python implementation at a time only. Replacement for python.eclass |
53 |
without SUPPORT_PYTHON_ABIS. |
54 |
|
55 |
4) python-any-r1.eclass -- for packages which need only simple, |
56 |
build-time dependency on Python. Doesn't export USE flags and only uses |
57 |
'|| ( python:X.Y python:X.Z )' kind of deps. Usually replaces |
58 |
non-eclass deps on Python. |
59 |
|
60 |
5) python-utils-r1.eclass -- helper functions. Inheriting it directly |
61 |
sounds a bit fragile. |
62 |
|
63 |
So, if you package uses distutils (calls setup.py), you always inherit |
64 |
distutils-r1 only. Otherwise, you inherit 2-4 of your choice. |
65 |
|
66 |
|
67 |
|
68 |
Variable and function substitutions |
69 |
----------------------------------- |
70 |
|
71 |
PYTHON_DEPEND, RESTRICT_PYTHON_ABIS -> PYTHON_COMPAT (an array!) |
72 |
SUPPORT_PYTHON_ABIS -> depends on eclass |
73 |
PYTHON_USE_WITH* -> hand-written USE-dep in PYTHON_REQ_USE |
74 |
|
75 |
DISTUTILS_SRC_TEST -> manual IUSE, DEPEND, python_test() |
76 |
PYTHON_CFLAGS -> has to be done by hand |
77 |
|
78 |
PYTHON_MODNAME -> removed, not necessary anymore |
79 |
DISTUTILS_USE_SEPARATE_SOURCE_DIRECTORIES -> DISTUTILS_IN_SOURCE_BUILD |
80 |
DISTUTILS_GLOBAL_OPTIONS -> mydistutilsargs (an array) |
81 |
DOCS -> an array now, replaces defaults (instead of appending) |
82 |
added HTML_DOCS, PATCHES |
83 |
|
84 |
$(PYTHON) -> ${PYTHON} (path) or ${EPYTHON} (basename) |
85 |
${PYTHON_ABI} -> use ${EPYTHON} instead |
86 |
|
87 |
[outside of distutils-r1, for distutils-r1 see below] |
88 |
python_execute_function -> python_foreach_impl |
89 |
python_execute_function -f -> python_export_best; ... |
90 |
python_mod_{optimize,cleanup} -> removed |
91 |
|
92 |
$(python_get_sitedir) and friends -> now contain ${EPREFIX} as well |
93 |
insinto $(python_get_...) -> python_domodule, python_doinclude, ... |
94 |
|
95 |
python_convert_shebangs: |
96 |
- on already-installed script -> python_replicate_script, |
97 |
- on to-be-installed scripts -> python_foreach_impl python_doscript ..., |
98 |
- in python-single-r1 -> python_fix_shebang "${D}"... |
99 |
|
100 |
Former two create impl-variants and wrapper. The latter converts only. |
101 |
|
102 |
I think that's all common cases. |
103 |
|
104 |
|
105 |
PYTHON_COMPAT |
106 |
------------- |
107 |
|
108 |
Eclasses 1-4 use PYTHON_COMPAT to denote supported Python |
109 |
implementations. You have to list all the implementations there. |
110 |
There's no 'but my package supports even Python 4!' |
111 |
|
112 |
Please test packages before adding something there. Grep bugzie |
113 |
for bugs, run test suites, *check dependencies*. If in doubt, add only |
114 |
python2.7 and 3.2 (if both work). We'll add other versions if somebody |
115 |
needs them. |
116 |
|
117 |
|
118 |
|
119 |
distutils-r1 |
120 |
------------ |
121 |
|
122 |
Most importantly, please don't migrate packages 1:1 from the old |
123 |
eclass. The new eclass design is based on python-distutils-ng. Feel |
124 |
free to look at some of dev-python/ packages to get a feeling how to |
125 |
solve some of the common issues. |
126 |
|
127 |
The packages which can provide solutions for most of the common |
128 |
problems (by having those problems) include: |
129 |
|
130 |
- dev-python/matplotlib -- py2-only deps in py2+3 package (please |
131 |
always open bugs for those), weird config command, |
132 |
|
133 |
- dev-python/numpy -- a lot of horrible, random stuff, |
134 |
|
135 |
- dev-python/pygments -- nosetests and 2to3 conversion, |
136 |
|
137 |
- dev-python/unittest2 -- separate sources for py2 & py3, tests having |
138 |
problems with parallel builds. |
139 |
|
140 |
A few quick notes: |
141 |
|
142 |
1) src_prepare() to src_install() are replaced each with two sub-phases: |
143 |
|
144 |
- python_prepare_all() to python_install_all() -- which are run once |
145 |
for the whole build process. There you put the common stuff like |
146 |
applying patches, building and installing docs. |
147 |
|
148 |
python_prepare_all() and python_install_all() (only the two) have |
149 |
default functions so remember to call them |
150 |
(as distutils-r1_python_prepare_all). |
151 |
|
152 |
- python_prepare() to python_install() -- which are run repeatedly for |
153 |
each Python implementation being built. There you put the specific |
154 |
stuff like building and installing the sources, running the tests. |
155 |
|
156 |
python_compile() and python_install() have defaults, and those |
157 |
defaults shall be usually called (distutils-r1_python_compile). Use |
158 |
of python_prepare() should be avoided as that implies in-source |
159 |
build. |
160 |
|
161 |
You declare src_*() phases only in special cases (like disabling |
162 |
parallel build for only one phase, see dev-python/nose). Moving 1:1 |
163 |
distutils.eclass python_execute_function snippets is simply wrong. |
164 |
|
165 |
As in, WRONG: |
166 |
|
167 |
src_test() { |
168 |
testing() { |
169 |
footest |
170 |
} |
171 |
python_foreach_impl testing |
172 |
} |
173 |
|
174 |
CORRECT: |
175 |
|
176 |
python_test() { |
177 |
footest || die "Tests fail with ${EPYTHON}" |
178 |
} |
179 |
|
180 |
|
181 |
2) out-of-source builds + parallel builds usually mean fun but trouble. |
182 |
|
183 |
When working on an ebuild, it's best to set PYTHON_TARGETS=python2_7 |
184 |
temporarily and get it to work like this. Then you try it with all |
185 |
targets enabled and built in parallel. |
186 |
|
187 |
If you can't fix the package to work properly with that kind of build, |
188 |
try DISTUTILS_IN_SOURCE_BUILD=1 first. That usually solves most |
189 |
of the issues, with the cost of keeping multiple source copies. |
190 |
|
191 |
As a last resort, use DISTUTILS_NO_PARALLEL_BUILD=1. You may have to |
192 |
use it to avoid excessive use of memory or disk space. If you have to |
193 |
use it for any other reason, please report a bug upstream. If possible, |
194 |
please try to set it only during specific src_*() phases as it helps |
195 |
a lot in making builds faster. |
196 |
|
197 |
|
198 |
3) non-standard distutils ebuilds |
199 |
|
200 |
The distutils-r1 eclass was designed to handle most common case |
201 |
of distutils use gracefully -- the Python packages. Therefore, it |
202 |
exports phase functions and adds python to DEP and RDEP by default. |
203 |
|
204 |
If you have a package with Python bindings where distutils is only |
205 |
flag-conditional, you want to set DISTUTILS_OPTIONAL=1. Then add |
206 |
${PYTHON_DEPS} to your DEP+RDEP yourself (as with python-r1) and call |
207 |
the phases in 'use python &&'. Or just split the package, it avoids |
208 |
unnecessary rebuilds and makes linking easier/more correct. |
209 |
|
210 |
If you have a distutils package which can be installed for one Python |
211 |
implementation only (like python-single-r1 or distutils.eclass without |
212 |
SUPPORT_PYTHON_ABIS), set DISTUTILS_SINGLE_IMPL=1. This will switch |
213 |
the code to use python-single-r1 instead of python-r1. |
214 |
|
215 |
Both of the above variables have to be set before inheriting |
216 |
distutils-r1. |
217 |
|
218 |
|
219 |
|
220 |
python-r1 and python-single-r1 |
221 |
------------------------------ |
222 |
|
223 |
This is closer to 1:1 from python.eclass. A few notes though: |
224 |
|
225 |
1) use python-r1 *only* if your package and all of its Python deps |
226 |
support being built for multiple Python implementations. Otherwise, |
227 |
use python-single-r1. |
228 |
|
229 |
As in, WRONG: |
230 |
|
231 |
PYTHON_COMPAT=( python2_7 ) # hahaha, tricked him! |
232 |
inherit python-r1 |
233 |
|
234 |
CORRECT: |
235 |
|
236 |
PYTHON_COMPAT=( python{2_6,2_7} ) |
237 |
inherit python-single-r1 |
238 |
|
239 |
|
240 |
2) no Python deps are added by default. Use: |
241 |
|
242 |
RDEPEND="${PYTHON_DEPS}" |
243 |
|
244 |
or: |
245 |
|
246 |
RDEPEND="python? ( ${PYTHON_DEPS} )" |
247 |
|
248 |
|
249 |
3) Python package deps need USE-deps to enforce same implementations |
250 |
being enabled. That is: |
251 |
|
252 |
DEPEND="dev-python/setuptools[${PYTHON_USEDEP}]" |
253 |
|
254 |
If necessary to hack py3 package with py2 deps (please keep a bug open |
255 |
in that case!): |
256 |
|
257 |
RDEPEND="gtk? ( dev-python/pygtk[$(python_gen_usedep python2*)] )" |
258 |
REQUIRED_USE="gtk? ( || ( $(python_gen_useflags python2*) ) )" |
259 |
|
260 |
|
261 |
4) Please try to use out-of-source builds and avoid copying sources |
262 |
unnecessarily. python_foreach_impl() exports BUILD_DIR to help you |
263 |
with that. autotools-utils and cmake-utils respect that. |
264 |
|
265 |
inherit autotools-utils |
266 |
|
267 |
src_configure() { |
268 |
local myeconfargs=( |
269 |
--with-foo |
270 |
#... |
271 |
) |
272 |
|
273 |
# it builds the package in implementation-specific dir! |
274 |
python_foreach_impl autotools-utils_src_configure |
275 |
} |
276 |
|
277 |
src_compile() { |
278 |
python_foreach_impl autotools-utils_src_compile |
279 |
} |
280 |
|
281 |
If you really have to use an in-source build, python_copy_sources() |
282 |
and run_in_build_dir() functions are at your service: |
283 |
|
284 |
src_prepare() { |
285 |
python_copy_sources |
286 |
} |
287 |
|
288 |
src_configure() { |
289 |
python_foreach_impl run_in_build_dir econf ... |
290 |
} |
291 |
|
292 |
|
293 |
5) python-r1 doesn't export pkg_setup(). The implementation data gets |
294 |
set in python_foreach_impl or by python_export_best. |
295 |
|
296 |
python-single-r1 exports pkg_setup(). It finds out which implementation |
297 |
was selected and exports all data for it. |
298 |
|
299 |
|
300 |
6) The generic idea of 'converting shebangs' is discouraged. |
301 |
|
302 |
Packages supporting multiple implementations are supposed to create |
303 |
impl-variants of *all* the scripts. This is only way to enforce that |
304 |
PYTHON_COMPAT is respected properly. |
305 |
|
306 |
In distutils-r1 this is done automatically. In python-r1 packages, |
307 |
you can either: |
308 |
|
309 |
a) install scripts to temporary locations and use python_doscript() |
310 |
to install them. Note that this is done for a single implementation |
311 |
only (so you put it in python_foreach_impl). |
312 |
|
313 |
It installs the script as 'foo-${EPYTHON}' and symlinks the wrapper |
314 |
to 'foo'. If your build system tries to write to 'foo' then, this may |
315 |
end up really bad. |
316 |
|
317 |
b) use python_replicate_script() on an installed script. It will copy |
318 |
the script into variants and then symlink the wrapper. All in one run, |
319 |
don't use python_foreach_impl here. |
320 |
|
321 |
In python-single-r1, you can mangle shebangs to match the selected |
322 |
Python implementation. Use python_fix_shebang for that. |
323 |
|
324 |
Please note that the shebang conversion function is very strict |
325 |
in order to avoid mistakenly breaking something. If it fails with your |
326 |
shebang, please let us know. |
327 |
|
328 |
|
329 |
|
330 |
python-any-r1 |
331 |
------------- |
332 |
|
333 |
This eclass is something special. Unlike previous eclasses, it does not |
334 |
set any USE flags, so no rebuilds are forced when user changes enabled |
335 |
Python implementations. It can be used to handle build-time deps |
336 |
on Python. |
337 |
|
338 |
Alike with other eclasses, you need to set PYTHON_COMPAT and use |
339 |
PYTHON_DEPS to get proper dependencies. |
340 |
|
341 |
# Build system needs python2.6+ |
342 |
PYTHON_COMPAT=( python{2_6,2_7} ) |
343 |
inherit python-any-r1 |
344 |
|
345 |
DEPEND="${PYTHON_DEPS}" |
346 |
|
347 |
It adds '|| ( python:{2_7,2_6} )'. pkg_setup() finds out best installed |
348 |
one and uses it. |
349 |
|
350 |
-- |
351 |
Best regards, |
352 |
Michał Górny |