[gentoo-alt] idea: use portage to build a scientific Python installation

[Martin Luessi]

Hi all,

I recently had the idea that it might be possible to use portage to
build a scientific Python installation and wanted to float the idea by
you to see what you think.

First, let me explain the reason for why anyone would want to do so.
For work, I use Python extensively for scientific computing. However,
I do not have administrator rights on my workstation and the
distribution we use (CentOS) does not have the latest Python packages
that are needed for scientific computing. In addition, even if CentOS
had the packages, it wouldn't be feasible to constantly ask the
sysadmins to install/update packages. One solution is to use a
scientific Python distribution from a commercial vendor, e.g., Canopy
from Enthought or Anaconda from Continuum Analytics. While these
distributions work quite well, they are expensive for non-academic
users and they are not very flexible, i.e., it can be difficult to
install packages that are not in the package repository provided by
the vendor, especially if the packages need additional dependencies. I
also have a gentoo-prefix setup on my workstation. However, the whole
prefix directory is very large as it makes minimal assumptions about
the libraries provided by the host system. The size is a problem when
using it over NFS e.g. on a cluster. Also, I have found that it is
difficult to get X11 applications working as the gentoo-prefix will
install its own X server etc.

This made me wonder whether portage could be used to build a
scientific Python installation. My idea is instead of making very
minimal assumptions about the libraries provided by the host system
(as done in a normal prefix install), one could generate a world file
listing all the libraries provided by the host system and freeze their
versions using package-mask. Like that, programs and libraries in the
prefix would link to libraries on the host system whenever possible,
which would make the prefix smaller. By having a gentoo based
scientific Python installation, one could take advantage of all the
packages provided by gentoo-science and it would make it easy to
install Python packages that depend on non-Python libraries.
What do you guys think, is this feasible?

Best,

Martin