Re: [gentoo-science] Usage of blas and lapack with 64 bit integers

[Aisha Tammy]

On 10/23/20 8:14 AM, Honza Macháček wrote:

> On 10/22/20 1:06 PM, Aisha Tammy wrote:
>> > ...Currently I use in my work mainly sci-chemistry/nwchem, now at the > 7.0.0 version. It uses blas and lapack, and prefers compilation with > 64 bit integers (ILP64) data model, although an utility to convert it > for LP64 model with 32 bit integers is provided. I'm attaching my > ebuild that I've finally used to install and start using the new > NWChem version. I'm also adding all the patches the ebuild references > in case you wanted to test the ebuild. I'm affraid the way I made it > use the ILP64 version of sci-libs/openblas-0.3.10 is rather ugly, and > some more elegant, more general, more systematic way should be > available...
>>
>> Yes, you can try to install the OpenBLAS package from ::gentoo with
>> the 'index-64bit' use flag enabled, which will install a libopenblas64.so
>> which should have what you want.
>>
>> I am working on cleanin up these packages, but this should work for your
>> particular problem.
>
>    I'm sorry I wrote too much of mostly irrelevant talking, and failed to describe my problem properly.
>
>    Yes, I've found out that to get sci-libs/openblas compiled in the ILP64 model I have to set the “index-64bit” USE flag instead of the former “int64”. I don't understand why the flag was renamed, and I don't like it (I'll write more on the topic further below), but I can live with it as long as I know what I have to set to get what I need.
>
>    The worst problem I see in the effect of the flag: sci-libs/openblas-0.3.10 compiles in the LP64 (32 bit integer) model, and installs the library with all the bells and whistles. In addition, if the “index-64bit” is set, the ILP64 (64 bit integer) model version is compiled, and the resulting library file installed — completely bare, with no pkg-config file, no eselect configuration. As a result eselect knows nothing of the ILP64 version being installed, and packages that want to use it cannot get the necessary configuration information from pkg-config, at least not directly; you can see in my sci-chemistry/nwchem ebuild that I let it query pkg-config for the library directories of the LP64 openblas version, and add the name of the ILP64 library, manually hardcoding in the ebuild the knowledge got from reading the sci-libs/openblas ebuild of the name used for the ILP64 library and of the fact it is installed into the same directory as the LP64 version. This is non-elegant
> and error-prone in case changes are made to the sci-libs/openblas ebuild in future version changing the name used for the ILP64 library or the directory it's installed into.
>
>    Maybe I should ask questions and protest the current state of afaires in the Gentoo bugzilla when the sci-libs/openblas ebuild resides in the main tree but I think I've seen phasing out the eselect alternatives concept and moving the ebuild to the main repository mentioned a few times in this list, so I think people responsible for the changes, and knowing their rationale are somewhere around.
>
>    The problem with eselect may span more packages, at least sci-libs/mkl, the only linear algebra library besides sci-libs/openblas that supports the ILP64 model, as far as I know. In the past, in the eselect alternatives ecosystem, ILP64 model libraries were installed as a separate category of linear algebra libraries. Now sci-libs/openblas with the “index-64bit” flag set installs both LP64 and ILP64 libraries, only does not inform pkg-config nor eselect of the ILP64 version existence. On the other hand sci-libs/mkl isntalls, as far as I understand its ebuild, always only one version, and the “int64” flag selects whether it should use the LP64 }32 bit integer) or ILP64 (64 bit integer) model. I don't use sci-libs/mkl and I have not tested its behaviour but I suspect that after installing it with “int64”, selecting it with eselect will break all the programs using linear algebra libraries and expecting 32 bit integers.
>
>    By the way, there are newer versions of sci-libs/mkl in the science overlay than in the main tree but they still depend on the old removed eselect alternatives ecosystem. That may add to the relevance of my questions and comments on mostly main tree packages to the science overlay project, as it suggests that phasing out the old eselect alternatives ecosystem, partial to the science overlay, is still a work in progress, and while it includes moving packages from the overlay to the main tree, recreating the functionality based in the past on that ecosystem is a job of the overlay developers.
>
>    I have not experimented with setting the default integer size to 64 bits for my whole system, so I don't know if such an approach is viable. I believe that having LP64 and ILP64 mode packages coexisting side by side is desirable, and that eselect should recognize LP64 and ILP64 libraries as separate categories, and offer switching among different available implementations inside these categories separately, just like it used to do in the eselect alternatives times, never replace an LP64 model library with an ILP64 model one or vice versa. Having all the ebuilds of the libraries that support installation in both model install them both side by side when ILP64 model is selected would likely be ideal, whenever possible.
>
>    Besides I think that the flag selecting the ILP64 model should retain its old name “int64”. It's shorter than “index-64bit”, and at the same time more descriptive, less confusing. Moreover it has a tradition. While in the science overlay it's currently used only by my old sci-chemistry/nwchem and by virtual/scalapack, where it selects as the scalapack implementation sci-libs/mkl[int64], in the main tree it's used by:
>
> sci-libs/hypre
>
> sci-libs/mkl
>
> sci-libs/parmetis
>
> sci-libs/pastix
>
> sci-libs/scotch
>
> sci-libs/superlu_mt
>
>    In all those packages the “int64” flag makes the ebuild set the integer size to 64 bits, only in the sci-libs/scotch-6.0.4-r2 it sets only some array indices to 64 bits (-DIDXSIZE64), leaving all the other integers at the system default size — and it may be wrong, and the setting for all integers to be 64 bits wide (-DINTSIZE64) should likely be added, as sci-libs/pastix[int64] depends on sci-libs/scotch[int64], and sets the size of all integers to 64 bits, so sci-libs/scotch[int64] installed by the current ebuild may break it.
>
>    The “index-64bit” flag is unique to the sci-libs/openblas ebuild. There is another solitaire flag “64bit-index” unique to the sci-libs/blis ebuild — and its effect is to set the integer size to 64 bits.
>
>    I don't know where the “index” part came from but it's definitely messy.
>
>    That's, I'm afraid, more than enough commenting and criticising from me. I'm getting back to my usual hibernation. After some time, rather long then short, I may wake up again, and file some bugs on the Gentoo bugzilla, publish my personal overlay, quit using Gentoo, or whatever.
>
>    All the best for now.
>
>          Honza Macháček
>


These are all things that I working on fixing.

If you are really interested in following the changes I am doing, you can

take a peek at my WIP overlay (I only uploaded this a day ago)

https://github.com/bsd-ac/gentoo-blas-lapack



Current changes I have made

- split openblas/blis and others into two providers for int32 and int64

   so that they can work simultaneously in parallel without breaking

- make provider libraries load by chain loading from dummy libs

  (this strictly adheres to standard unsuspecting users might be left broken)

- make a single eslect-library module for managing blas/lapack/blas64/lapack64

   (it also handles all other library implementations)



All packages in gentoo and science should be fine with this.



This may also fix your problems, or you can try and give suggestions to improve.



Aisha