Gentoo Archives: gentoo-dev

From: Francesco Riosa <vivo75@×××××.com>
To: gentoo development <gentoo-dev@l.g.o>
Subject: [gentoo-dev] RFC: ${PYTHON_COMPAT_ADD} in addition to ${PYTHON_COMPAT_OVERRIDE}
Date: Mon, 15 Jan 2018 15:27:38
Message-Id: c9cd2bea-74b1-0c2a-2633-bd114140c5d5@gmail.com
1 In late 2015 ${PYTHON_COMPAT_OVERRIDE} has been standardized and added
2 to all python eclasses, it's useful for developers that want test and
3 mark the package for newer versions of python.
4
5 However (unless I'm missing something) PYTHON_COMPAT_OVERRIDE is not
6 usable if:
7 - the user want only python 2.7 and 3.6 (latest) installed
8   no python 3.5
9 - don't want to mess dependencies (the warnings in the eclass are scary)
10
11 So, what can be done to let the user choose it's preferred python
12 version(s) without having to build it's own overlay?
13
14 One solution is to change ebuilds in tree to include a user variable in
15 the PYTHON* arrays, example:
16
17 -PYTHON_COMPAT=( python3_{4,5} )
18 +PYTHON_COMPAT=( python3_{4,5} ${PYTHON_COMPAT_ADD} )
19
20 if someone want to bet that packages are ok with 3.6/latest (even before
21 a developer tested it) then add PYTHON_COMPAT_ADD=python3_6 to
22 /etc/portage/make.conf and run egencache.
23
24 Indeed biggest problem in changing $PYTHON* variables is that metadata
25 also change and cache is invalidated.
26
27 However if the problem is known (*), and if the change to metadata is
28 stable per "system"/"gentoo repo clone", then the solution to the
29 problem is easy; just run $(egencache --update -j$(nproc) --repo gentoo)
30 after each sync.
31 The most important thing is that this solution is scriptable and need no
32 human intervention.
33
34 Notice also that it's easy to have an array with duplicate values
35 PYTHON_COMPAT=( python3_6 python3_6 ) but at a first glance
36 _python_set_impls() is resilient to this.
37
38 (*) In a perfect world, where global variables that can change metadata
39 are known {egencache,emerge} can be made conscious and warn if the cache
40 is invalid, but that's out of scope at the moment.
41
42 The procedure to find out which packages are enabled for python 3.5 but
43 not for python 3.6 is not totally straightforward but this should be the
44 list of ~amd64 packages interested:
45
46
47 app-admin/glance                    [15.0.0]
48 app-admin/setools                   [4.1.1]
49 app-arch/bloscpack                  [0.11.0]
50 app-arch/patool                     [1.12]
51 app-arch/vimball                    [0.5.1]
52 app-backup/attic                    [0.16]
53 app-backup/untangle-https-backup    [0.0.7]
54 app-benchmarks/bootchart2           [0.14.7-r1]
55 app-emulation/libguestfs            [1.36.5]
56 app-emulation/sen                   [0.5.1]
57 app-i18n/transifex-client           [0.12.4]
58 app-misc/asciinema                  [1.4.0]
59 app-misc/gramps                     [4.2.5]
60 app-misc/icdiff                     [1.9.1]
61 app-misc/jira-cli                   [2.1.5]
62 app-misc/lirc                       [0.10.0_rc2]
63 app-misc/solaar                     [0.9.2-r3]
64 app-misc/terminal-colors            [2.2]
65 app-mobilephone/obexftp             [0.24-r1]
66 app-office/libreoffice-bin          [5.4.2.2-r1]
67 app-pda/libimobiledevice            [1.2.0]
68 app-portage/elogviewer              [2.7]
69 app-portage/gentoolkit-dev          [0.3.1]
70 app-portage/gs-elpa                 [0.1.3-r1]
71 app-portage/g-sorcery               [0.2.1-r1]
72 app-portage/gs-pypi                 [0.2.1-r1]
73 app-portage/pqlop                   [0.02-r1]
74 app-text/grip                       [4.2.0]
75 app-text/gtranslator                [2.91.7-r1]
76 app-text/landslide                  [1.1.3]
77 app-text/pelican                    [3.7.1]
78 app-text/pytextile                  [2.3.3]
79 app-text/sigil                      [0.9.8]
80 app-text/wiki2beamer                [0.9.5-r1]
81 app-vim/jedi                        [0.8_p20171015]
82 app-vim/pyclewn                     [2.1-r1]
83 dev-games/vamos                     [0.8.2]
84 dev-lang/hy                         [0.13.1]
85 dev-libs/gom                        [0.3.2]
86 dev-libs/Ice                        [3.6.3-r1]
87 dev-libs/libcec                     [4.0.2-r1]
88 dev-libs/libsolv                    [0.6.22]
89 dev-libs/link-grammar               [5.3.11]
90 dev-perl/Inline-Python              [0.560.0]
91 dev-python/abstract_rendering       [0.5.1]
92 dev-python/algopy                   [0.5.3]
93 dev-python/amqplib                  [1.0.2-r1]
94 dev-python/ansi2html                [1.3.0]
95 dev-python/aodhclient               [0.8.0]
96 dev-python/argcomplete              [1.9.2]
97 dev-python/asdf                     [1.2.1]
98 dev-python/astlib                   [0.8.0]
99 dev-python/astrodendro              [0.2.0]
100 dev-python/astroml                  [0.3]
101 dev-python/astroml-addons           [0.2.2]
102 dev-python/astroscrappy             [1.0.5]
103 dev-python/awscli                   [1.14.16]
104 dev-python/backports-abc            [0.5]
105 dev-python/beaker                   [1.8.1]
106 dev-python/behave                   [1.2.5-r1]
107 dev-python/bibtexparser             [0.6.2]
108 dev-python/breathe                  [4.6.0]
109 dev-python/castellan                [0.12.2]
110 dev-python/ccdproc                  [1.2.0]
111 dev-python/cgkit                    [2.0.0-r1]
112 dev-python/cgroup-utils             [0.6]
113 dev-python/chump                    [1.5.2]
114 dev-python/cliff                    [2.8.0]
115 dev-python/cliff-tablib             [1.1]
116 dev-python/Coffin                   [2.0.1]
117 dev-python/construct                [2.8.17]
118 dev-python/cursive                  [0.1.2]
119 dev-python/cvxopt                   [1.1.9]
120 dev-python/d2to1                    [0.2.12_p1]
121 dev-python/demjson                  [2.2.4]
122 dev-python/django-compressor        [1.5]
123 dev-python/django-sortedm2m         [1.3.2]
124 dev-python/django-tables2           [1.1.2]
125 dev-python/dnslib                   [0.9.6]
126 dev-python/dogpile-core             [0.4.1]
127 dev-python/ed25519ll                [0.6]
128 dev-python/enzyme                   [0.4.1-r2]
129 dev-python/fedmsg                   [0.18.3]
130 dev-python/feedgenerator            [1.9]
131 dev-python/flask-admin              [1.5.0]
132 dev-python/flask-appconfig          [0.11.1-r1]
133 dev-python/flask-babelex            [0.9.3]
134 dev-python/flask-bootstrap          [3.3.7.1]
135 dev-python/flask-debug              [0.4.3]
136 dev-python/flask-mongoengine        [0.9.3]
137 dev-python/flask-nav                [0.6]
138 dev-python/flask-pymongo            [0.4.1]
139 dev-python/flask-whooshalchemy      [0.8]
140 dev-python/flipflop                 [1.0]
141 dev-python/formencode               [1.3.0-r2]
142 dev-python/fuzzywuzzy               [0.12.0]
143 dev-python/gcs-oauth2-boto-plugin   [1.14]
144 dev-python/geoalchemy2              [0.3]
145 dev-python/glance_store             [0.22.0]
146 dev-python/guessit                  [2.1.4]
147 dev-python/gwcs                     [0.7]
148 dev-python/hglib                    [2.1]
149 dev-python/hiredis                  [0.2.0]
150 dev-python/httreplay                [0.2.0]
151 dev-python/husl                     [4.0.3]
152 dev-python/hvac                     [0.2.17]
153 dev-python/id3-py                   [1.2-r1]
154 dev-python/ImageHash                [3.7]
155 dev-python/imageio                  [2.1.1]
156 dev-python/influxdb                 [2.12.0]
157 dev-python/intelhex                 [2.0]
158 dev-python/ipy                      [0.83]
159 dev-python/irc                      [15.0.6]
160 dev-python/iso3166                  [0.8]
161 dev-python/iso_639                  [0.4.5]
162 dev-python/jaraco-classes           [1.4]
163 dev-python/jaraco-collections       [1.5]
164 dev-python/jaraco-functools         [1.15.1]
165 dev-python/jaraco-itertools         [2.0]
166 dev-python/jaraco-logging           [1.5]
167 dev-python/jaraco-stream            [1.1.1]
168 dev-python/jaraco-text              [1.8]
169 dev-python/jellyfish                [0.5.6]
170 dev-python/jenkinsapi               [0.2.29]
171 dev-python/jpype                    [0.6.2]
172 dev-python/jsonpickle               [0.9.3]
173 dev-python/jupyter                  [1.0.0-r1]
174 dev-python/jupyter_console          [5.1.0]
175 dev-python/keystonemiddleware       [4.17.0]
176 dev-python/kitchen                  [1.2.4]
177 dev-python/Kivy                     [1.10.0]
178 dev-python/ldappool                 [2.1.0]
179 dev-python/libcloud                 [1.2.1]
180 dev-python/libvirt-python           [3.10.0]
181 dev-python/libzilla                 [1.4]
182 dev-python/matplotlib2tikz          [0.6.14]
183 dev-python/maybe                    [0.4.0]
184 dev-python/meld3                    [1.0.2-r1]
185 dev-python/meteor-ejson             [1.1.0]
186 dev-python/microversion-parse       [0.1.4]
187 dev-python/mmh3                     [2.3.1]
188 dev-python/moviepy                  [0.2.2]
189 dev-python/mysql-connector-python   [2.1.4]
190 dev-python/nagiosplugin             [1.2.4]
191 dev-python/nbdime                   [0.2.0]
192 dev-python/netcdf4-python           [1.2.2]
193 dev-python/neutron-lib              [1.9.1]
194 dev-python/nose2                    [0.6.5]
195 dev-python/Numdifftools             [0.9.17]
196 dev-python/odfpy                    [1.3.2]
197 dev-python/openstacksdk             [0.9.17]
198 dev-python/os-brick                 [1.15.5]
199 dev-python/osc-lib                  [1.7.0]
200 dev-python/oslo-cache               [1.25.1]
201 dev-python/oslo-context             [2.17.1]
202 dev-python/oslo-db                  [4.25.1]
203 dev-python/oslo-log                 [3.30.2]
204 dev-python/oslo-messaging           [5.30.2]
205 dev-python/oslo-middleware          [3.30.1]
206 dev-python/oslo-privsep             [1.22.1]
207 dev-python/oslo-reports             [1.22.1]
208 dev-python/oslo-rootwrap            [5.9.2]
209 dev-python/oslo-service             [1.25.1]
210 dev-python/oslo-versionedobjects    [1.26.1]
211 dev-python/oslo-vmware              [2.23.1]
212 dev-python/osprofiler               [1.11.0]
213 dev-python/os-testr                 [0.8.0]
214 dev-python/os-traits                [0.3.3]
215 dev-python/os-vif                   [1.7.0]
216 dev-python/os-win                   [2.2.0]
217 dev-python/os-xenapi                [0.2.0]
218 dev-python/ovs                      [2.7.2]
219 dev-python/ovsdbapp                 [0.4.0]
220 dev-python/owslib                   [0.16.0]
221 dev-python/pelican-minify           [0.9]
222 dev-python/pid                      [2.1.1]
223 dev-python/pika                     [0.10.0]
224 dev-python/pika-pool                [0.1.3]
225 dev-python/pilkit                   [2.0]
226 dev-python/pillowfight              [0.2]
227 dev-python/plotly                   [1.9.6]
228 dev-python/prov                     [1.5.1]
229 dev-python/pycadf                   [2.6.0]
230 dev-python/pycanberra               [0_pre20130515]
231 dev-python/pycipher                 [0.2-r1]
232 dev-python/PyDbLite                 [3.0.4]
233 dev-python/pydot-ng                 [1.0.0]
234 dev-python/pydotplus                [2.0.2]
235 dev-python/pyds9                    [1.8.1]
236 dev-python/pyee                     [1.0.2]
237 dev-python/pyfeyn                   [1.0.0]
238 dev-python/pyfits                   [3.4-r1]
239 dev-python/pyflann                  [1.9.1]
240 dev-python/pygal                    [2.1.1]
241 dev-python/pygccxml                 [1.7.5]
242 dev-python/pyghmi                   [1.0.22]
243 dev-python/pygpu                    [0.6.7]
244 dev-python/pygsl                    [2.1.1]
245 dev-python/pykwalify                [1.5.2]
246 dev-python/pylibacl                 [0.5.0-r1]
247 dev-python/pyminuit                 [1.2.1-r1]
248 dev-python/pynzb                    [0.1.0-r1]
249 dev-python/pyopencl                 [2017.2]
250 dev-python/pypowervm                [1.1.6]
251 dev-python/pyproj                   [1.9.5.1]
252 dev-python/pyrqlite                 [2.0]
253 dev-python/pyrtf                    [0.45-r2]
254 dev-python/pysaml2                  [4.5.0]
255 dev-python/pyscaffold               [2.4.4]
256 dev-python/pyshark                  [0.3.7.2]
257 dev-python/pytest-qt                [2.3.0]
258 dev-python/pytest-raisesregexp      [2.1]
259 dev-python/python-barbicanclient    [4.5.2]
260 dev-python/python-ceilometerclient  [2.6.2]
261 dev-python/python-cinderclient      [3.1.0]
262 dev-python/python-ddp               [0.1.5]
263 dev-python/python-designateclient   [2.7.0]
264 dev-python/python-djvulibre         [0.8]
265 dev-python/python-gammu             [2.5]
266 dev-python/python-gnupg             [0.4.0]
267 dev-python/python-heatclient        [1.11.1]
268 dev-python/python-ironicclient      [1.17.0]
269 dev-python/python-magnumclient      [2.3.1]
270 dev-python/python-manilaclient      [1.14.0]
271 dev-python/python-meteor            [0.1.6]
272 dev-python/python-mistralclient     [3.1.4]
273 dev-python/python-monascaclient     [1.7.0]
274 dev-python/python-mpd               [0.5.5]
275 dev-python/python-neutronclient     [6.5.0]
276 dev-python/python-novaclient        [9.1.1]
277 dev-python/python-openstackclient   [3.12.0]
278 dev-python/python_orocos_kdl        [1.3.1-r1]
279 dev-python/python-otrs              [0.3.0]
280 dev-python/python-rethinkdb         [2.3.0]
281 dev-python/python-saharaclient      [1.1.0]
282 dev-python/python-scsi              [0_pre160211]
283 dev-python/python-senlinclient      [1.2.0]
284 dev-python/python-swiftclient       [3.4.0]
285 dev-python/python-troveclient       [2.5.0]
286 dev-python/python-zaqarclient       [1.2.0]
287 dev-python/python-zunclient         [0.4.1]
288 dev-python/pyuv                     [1.2.0]
289 dev-python/pyzor                    [1.0.0]
290 dev-python/rackspace-auth-openstack [1.3]
291 dev-python/rackspace-monitoring     [0.8.0]
292 dev-python/rackspace-novaclient     [2.1]
293 dev-python/raven                    [5.31.0]
294 dev-python/redis-py-cluster         [1.3.4]
295 dev-python/rencode                  [1.0.5]
296 dev-python/requests-cache           [0.4.12]
297 dev-python/retry-decorator          [1.0.0]
298 dev-python/rnc2rng                  [2.5]
299 dev-python/rply                     [0.7.4]
300 dev-python/rtslib-fb                [2.1.62]
301 dev-python/ryu                      [4.14]
302 dev-python/seaborn                  [0.7.1]
303 dev-python/semantic_version         [2.6.0-r1]
304 dev-python/sepolgen                 [2.6]
305 dev-python/slackclient              [1.1.0]
306 dev-python/sleekxmpp                [1.3.1-r1]
307 dev-python/spectral-cube            [0.4.0]
308 dev-python/specutils                [0.2.2]
309 dev-python/sqlalchemy-rqlite        [1.0]
310 dev-python/stormpath                [2.4.5]
311 dev-python/structlog                [16.1.0]
312 dev-python/stsci-distutils          [0.3.7]
313 dev-python/stsci-sphinxext          [1.2.1]
314 dev-python/tablib                   [0.11.2]
315 dev-python/tagpy                    [2013.1]
316 dev-python/taskflow                 [2.14.1]
317 dev-python/tempest-lib              [1.0.0]
318 dev-python/tenacity                 [4.4.0]
319 dev-python/testfixtures             [4.9.1]
320 dev-python/tinydb                   [3.1.3]
321 dev-python/tooz                     [1.58.0]
322 dev-python/torment                  [3.0.3]
323 dev-python/transmissionrpc          [0.11]
324 dev-python/trollius                 [2.1]
325 dev-python/tvdb_api                 [1.10_pre20150406-r1]
326 dev-python/twitter                  [1.17.1]
327 dev-python/uncompyle6               [2.10.1]
328 dev-python/urdf_parser_py           [0.3.3]
329 dev-python/utmp                     [0.4]
330 dev-python/vcstools                 [0.1.40]
331 dev-python/versiontools             [1.9.1-r1]
332 dev-python/w3lib                    [1.18.0]
333 dev-python/webassets                [0.12]
334 dev-python/ws4py                    [0.3.4]
335 dev-python/wsgiintercept            [1.3.1]
336 dev-python/wtf-peewee               [0.2.6]
337 dev-python/xonsh                    [0.4.7]
338 dev-python/yapsy                    [1.11.223-r1]
339 dev-python/yaql                     [1.1.3]
340 dev-python/zake                     [0.2.1]
341 dev-util/cppcheck                   [1.81]
342 dev-util/cram                       [0.7]
343 dev-util/devhelp                    [3.24.0]
344 dev-util/distro-info                [0.14]
345 dev-util/dogtail                    [0.9.10]
346 dev-util/fatrace                    [0.12]
347 dev-util/gcovr                      [3.3]
348 dev-util/howdoi                     [1.1.9]
349 dev-util/rosinstall                 [0.7.8]
350 dev-util/spec-cleaner               [0.8.9]
351 dev-util/wstool                     [0.1.17]
352 dev-vcs/git-cola                    [2.11]
353 dev-vcs/gitg                        [3.26.0]
354 games-misc/ponysay                  [3.0.2]
355 games-util/nml                      [0.4.4-r1]
356 gnome-base/gnome-shell              [3.24.3]
357 gnome-extra/chrome-gnome-shell      [9]
358 gnome-extra/cinnamon                [3.6.6-r2]
359 gnome-extra/gnome-builder           [3.24.2]
360 media-gfx/fontforge                 [20170731-r3]
361 media-gfx/sigal                     [1.3.0]
362 media-libs/pyliblo                  [0.10.0]
363 media-sound/gnome-music             [3.24.2]
364 media-sound/pithos                  [1.3.1]
365 media-tv/tvnamer                    [2.4]
366 media-video/pitivi                  [0.98.1]
367 media-video/subliminal              [2.0.5-r1]
368 net-analyzer/carl                   [0.9-r2]
369 net-analyzer/flent                  [1.0.1]
370 net-analyzer/rrdtool                [1.7.0]
371 net-analyzer/speedtest-cli          [1.0.2]
372 net-irc/irker                       [2.18]
373 net-irc/limnoria                    [20171025]
374 net-libs/libsignon-glib             [1.14]
375 net-misc/electron-cash              [3.0]
376 net-misc/electrum                   [3.0.5]
377 net-misc/electrum-ltc               [3.0_pre20171218]
378 net-misc/gns3-gui                   [2.0.3-r1]
379 net-misc/gns3-server                [2.0.3-r1]
380 net-misc/openvswitch                [2.8.1]
381 net-misc/trackma                    [0.7.4]
382 net-misc/whatportis                 [0.6]
383 net-misc/you-get                    [0.4.536]
384 net-news/liferea                    [1.12_rc3-r1]
385 net-wireless/urh                    [1.6.4.1]
386 sci-astronomy/pyephem               [3.7.6.0]
387 sci-biology/biopython               [1.68]
388 sci-libs/clblas                     [2.10]
389 sci-libs/grib_api                   [1.14.5]
390 sci-libs/shogun                     [5.0.0]
391 sci-mathematics/dunshire            [0.1.1]
392 sci-mathematics/pymc                [2.3.6]
393 sci-mathematics/relational          [2.5]
394 sci-visualization/fityk             [1.3.1]
395 sys-apps/policycoreutils            [2.7]
396 sys-apps/selinux-python             [2.7]
397 sys-auth/keystone                   [12.0.0]
398 sys-block/targetcli-fb              [2.1.45]
399 sys-cluster/cinder                  [11.0.2]
400 sys-cluster/heat                    [9.0.2]
401 sys-cluster/nova                    [16.0.4]
402 sys-cluster/sanlock                 [3.4.0-r1]
403 sys-fs/bcache-tools                 [1.0.8_p20140220-r1]
404 sys-kernel/kergen                   [0.1.4]
405 sys-libs/libsemanage                [2.7]
406 sys-power/acpilight                 [1.0-r1]
407 www-apps/nikola                     [7.8.8]
408 www-apps/radicale                   [1.1.1]
409 www-misc/urlwatch                   [2.6]
410 x11-misc/menumaker                  [0.99.10]
411 x11-wm/qtile                        [0.10.6]
412 dev-db/pgadmin4                     [2.1]
413 dev-libs/protobuf                   [3.5.1.1]
414 dev-python/flask-login              [0.3.2-r1]
415 dev-python/flask-mail               [0.9.1-r1]
416 dev-python/flask-migrate            [2.1.1]
417 dev-python/flask-principal          [0.4.0-r1]
418 dev-python/flask-security           [1.7.5-r1]
419 dev-python/flask-uploads            [0.2.0-r1]
420 dev-python/funcsigs                 [1.0.2-r1]
421 dev-python/translate-toolkit        [2.0.0]
422 kde-apps/lokalize                   [17.12.1]
423 media-gfx/hugin                     [2017.0]
424 media-libs/gexiv2                   [0.10.6]
425 net-irc/hexchat                     [2.12.4-r2]
426 sys-fs/zfs                          [0.7.5-r1]
427 sys-process/glances                 [2.11.1]
428 dev-python/batinfo                  [0.3]
429 dev-python/flask-babel              [0.11.2]
430 dev-python/flask-script             [2.0.6]
431 dev-python/flask-testing            [0.6.2]
432 dev-python/flask-wtf                [0.14.2]
433 media-libs/opencv                   [3.3.0-r4]
434 dev-python/ip-associations-python-novaclient-ext      [0.2]
435 dev-python/os-diskconfig-python-novaclient-ext        [0.1.3]
436 dev-python/os-networksv2-python-novaclient-ext        [0.26]
437 dev-python/os-virtual-interfacesv2-python-novaclient-ext [0.20]
438 dev-python/rax-default-network-flags-python-novaclient-ext [0.4.0]
439 dev-python/rax-scheduled-images-python-novaclient-ext [0.3.1]
440 net-analyzer/nagios-check_openvpn-simple              [0.0.1-r1]

Replies