Re: [gentoo-science] Usage of blas and lapack with 64 bit integers

[Honza Macháček]

On 10/22/20 1:06 PM, Aisha Tammy wrote:
> > ...Currently I use in my work mainly sci-chemistry/nwchem, now at the > 7.0.0 version. It uses blas and lapack, and prefers compilation with > 64
> bit integers (ILP64) data model, although an utility to convert it > for LP64
> model with 32 bit integers is provided. I'm attaching my > ebuild that I've
> finally used to install and start using the new > NWChem version. I'm also
> adding all the patches the ebuild references > in case you wanted to test the
> ebuild. I'm affraid the way I made it > use the ILP64 version of
> sci-libs/openblas-0.3.10 is rather ugly, and > some more elegant, more
> general, more systematic way should be > available...
>
> Yes, you can try to install the OpenBLAS package from ::gentoo with
> the 'index-64bit' use flag enabled, which will install a libopenblas64.so
> which should have what you want.
>
> I am working on cleanin up these packages, but this should work for your
> particular problem.

   I'm sorry I wrote too much of mostly irrelevant talking, and failed to 
describe my problem properly.

   Yes, I've found out that to get sci-libs/openblas compiled in the ILP64 model 
I have to set the “index-64bit” USE flag instead of the former “int64”. I don't 
understand why the flag was renamed, and I don't like it (I'll write more on the 
topic further below), but I can live with it as long as I know what I have to 
set to get what I need.

   The worst problem I see in the effect of the flag: sci-libs/openblas-0.3.10 
compiles in the LP64 (32 bit integer) model, and installs the library with all 
the bells and whistles. In addition, if the “index-64bit” is set, the ILP64 (64 
bit integer) model version is compiled, and the resulting library file installed 
— completely bare, with no pkg-config file, no eselect configuration. As a 
result eselect knows nothing of the ILP64 version being installed, and packages 
that want to use it cannot get the necessary configuration information from 
pkg-config, at least not directly; you can see in my sci-chemistry/nwchem ebuild 
that I let it query pkg-config for the library directories of the LP64 openblas 
version, and add the name of the ILP64 library, manually hardcoding in the 
ebuild the knowledge got from reading the sci-libs/openblas ebuild of the name 
used for the ILP64 library and of the fact it is installed into the same 
directory as the LP64 version. This is non-elegant and error-prone in case 
changes are made to the sci-libs/openblas ebuild in future version changing the 
name used for the ILP64 library or the directory it's installed into.

   Maybe I should ask questions and protest the current state of afaires in the 
Gentoo bugzilla when the sci-libs/openblas ebuild resides in the main tree but I 
think I've seen phasing out the eselect alternatives concept and moving the 
ebuild to the main repository mentioned a few times in this list, so I think 
people responsible for the changes, and knowing their rationale are somewhere 
around.

   The problem with eselect may span more packages, at least sci-libs/mkl, the 
only linear algebra library besides sci-libs/openblas that supports the ILP64 
model, as far as I know. In the past, in the eselect alternatives ecosystem, 
ILP64 model libraries were installed as a separate category of linear algebra 
libraries. Now sci-libs/openblas with the “index-64bit” flag set installs both 
LP64 and ILP64 libraries, only does not inform pkg-config nor eselect of the 
ILP64 version existence. On the other hand sci-libs/mkl isntalls, as far as I 
understand its ebuild, always only one version, and the “int64” flag selects 
whether it should use the LP64 }32 bit integer) or ILP64 (64 bit integer) model. 
I don't use sci-libs/mkl and I have not tested its behaviour but I suspect that 
after installing it with “int64”, selecting it with eselect will break all the 
programs using linear algebra libraries and expecting 32 bit integers.

   By the way, there are newer versions of sci-libs/mkl in the science overlay 
than in the main tree but they still depend on the old removed eselect 
alternatives ecosystem. That may add to the relevance of my questions and 
comments on mostly main tree packages to the science overlay project, as it 
suggests that phasing out the old eselect alternatives ecosystem, partial to the 
science overlay, is still a work in progress, and while it includes moving 
packages from the overlay to the main tree, recreating the functionality based 
in the past on that ecosystem is a job of the overlay developers.

   I have not experimented with setting the default integer size to 64 bits for 
my whole system, so I don't know if such an approach is viable. I believe that 
having LP64 and ILP64 mode packages coexisting side by side is desirable, and 
that eselect should recognize LP64 and ILP64 libraries as separate categories, 
and offer switching among different available implementations inside these 
categories separately, just like it used to do in the eselect alternatives 
times, never replace an LP64 model library with an ILP64 model one or vice 
versa. Having all the ebuilds of the libraries that support installation in both 
model install them both side by side when ILP64 model is selected would likely 
be ideal, whenever possible.

   Besides I think that the flag selecting the ILP64 model should retain its old 
name “int64”. It's shorter than “index-64bit”, and at the same time more 
descriptive, less confusing. Moreover it has a tradition. While in the science 
overlay it's currently used only by my old sci-chemistry/nwchem and by 
virtual/scalapack, where it selects as the scalapack implementation 
sci-libs/mkl[int64], in the main tree it's used by:

sci-libs/hypre

sci-libs/mkl

sci-libs/parmetis

sci-libs/pastix

sci-libs/scotch

sci-libs/superlu_mt

   In all those packages the “int64” flag makes the ebuild set the integer size 
to 64 bits, only in the sci-libs/scotch-6.0.4-r2 it sets only some array indices 
to 64 bits (-DIDXSIZE64), leaving all the other integers at the system default 
size — and it may be wrong, and the setting for all integers to be 64 bits wide 
(-DINTSIZE64) should likely be added, as sci-libs/pastix[int64] depends on 
sci-libs/scotch[int64], and sets the size of all integers to 64 bits, so 
sci-libs/scotch[int64] installed by the current ebuild may break it.

   The “index-64bit” flag is unique to the sci-libs/openblas ebuild. There is 
another solitaire flag “64bit-index” unique to the sci-libs/blis ebuild — and 
its effect is to set the integer size to 64 bits.

   I don't know where the “index” part came from but it's definitely messy.

   That's, I'm afraid, more than enough commenting and criticising from me. I'm 
getting back to my usual hibernation. After some time, rather long then short, I 
may wake up again, and file some bugs on the Gentoo bugzilla, publish my 
personal overlay, quit using Gentoo, or whatever.

   All the best for now.

         Honza Macháček