[gentoo-portage-dev] The build system (and install layout) of Portage

[Michał Górny]

Hi, everyone.

You've probably had the opportunity to hear that a lot has changed
in Python packaging since Portage's setup.py was written in 2014.  There
were some minor changes to keep it working since but it's time to
reconsider.

Long story short, distutils is strongly deprecated, setuptools
deprecated most of the customizations (and we're relying heavily
on customizations), PEP 517 doesn't cover our use cases exactly...
and it's quite likely that sooner or later our build system will fall
apart.  On top of that, setuptools is going through a stage of "let's
vendor a new dependency every week", so it doesn't look like a feasible
long-term solution.  A large part of the problem is that Portage is
heavily relying on non-Pythonic idioms for installing stuff.

At this point, I'd love to use flit for Portage, as that's a nice build
system with upstream that understands packagers.  I mean, if someone
tells me they want to avoid vendoring stuff *and* creating cyclic
dependencies (all other Python build systems vendor tons of stuff right
now), then I can't but appreciate that very much.  That said, I don't
think flit is powerful enough to handle all the intricacies Portage
requires, starting with the fancy way it's installing scripts to many
different directories.

An alternative is to go back to using (at least partially) Makefiles or
Meson.  However, that would have the important drawback that we'd lose
the ability to install Portage as a regular Python package (e.g. inside
a virtualenv).

At this point, I've pretty much lost all motivation to work on it. 
Nevertheless, it's something that needs to be done eventually.  Does
anyone have some idea, motivation and will to transition Portage to
another build system?

-- 
Best regards,
Michał Górny