Re: [gentoo-science] On BLAS and LAPACK int64 ABI

[Mo Zhou]

Hi Gentoo Science team,

In the sci overlay there is a long existing feature "int64" for the
BLAS/LAPACK packages, with which the type of array index in the
API/ABI will be changed into int64_t so that BLAS/LAPACK libraries
could be able to handle super large arrays. The reason of existence
of such feature stems from a Fortran compiler feature which could
change the default INTEGER size (e.g. 32bit, 64bit).

Recently the BLAS/LAPACK runtime switching mechanism has been merged
into ::gentoo, but you may find no counterpart to the "int64" feature
existing in ::sci. According to previous discussion, we could indeed
add a new alternative group called, for example, BLAS64 and LAPACK64.
However, it must be pointed out that, even if the "BLAS64" thing is
well supported by commonly used optimized implementations such as
BLIS, OpenBLAS, and MKL, the 64-bit-indexing feature is not well
supported by the reference CBLAS. Even worse, there is no common
standard for such "BLAS64" and "LAPACK64" libraries in terms of
ABI and SONAME. That means every move on the "BLAS64/LAPACK64"
runtime switching mechanism will be highly experimental and
may easily be broken by the authorities who define standards.

Before I started the GSoC project for Gentoo I had already started
working on adding BLAS64/LAPACK64 alternatives. We Debian science
team thought it's fine to experiment even if there is no standard
at all. Exactly due to this reason, the 64-bit-index feature wasn't
put on the original GSoC schedule. I thought we could first
experiment it on Debian.

However, now we have to face the problem again because some packages
such as Julia require the 64-bit-index ABI. (ah, yes, I was aware of
this since I also maintain Julia for Debian). Combining the fact that
there is no standard (even de-facto standard), and the demand of
some packages such as Julia, Benda and I reached an agreement that
we could simply add 64-bit-index support for OpenBLAS and postpone
the runtime switching feature for BLAS64/LAPACK64.

Julia calls BLAS64 and LAPACK64 through FFI. Theoretically Julia
could use any BLAS/LAPACK implementations but actually Julia's
LinearAlgebra standard library is not quite ready for such a
switching feature. Hence Debian's Julia package is actually
specifically linked against libopenblas.so (32-bit) and it is
irrelevant to the runtime switching feature. Afterall there is
no problem by forcing Julia users to use OpenBLAS.

In a word, what Benda and I plan to do are:

* add index-64bit USE flag to openblas, which compiles an extra
  libopenblas64.so for Julia's use.
* postpone the plan for BLAS64/LAPACK64 runtime switching, and
  experiment on Debian first.

That said, if anyone believes the BLAS64/LAPACK64 feature is sort
of urgent need, we can discuss further and revise the plan again.

Thanks.

See also:
[1] the 64-bit-indexing feature has been postponed by
    reference lapack upstream:
https://github.com/Reference-LAPACK/lapack/issues/184


On 2019-07-11 13:59, Benda Xu wrote:
> Hi,
>
> This discussion emerged from https://github.com/gentoo/sci/pull/876.  I
> am taking it here for better future reference.
>
> cdluminate:
>> It would be problematic to provide int64 ABI/API through libblas.so or
>> liblapack.so since that would lead to a mess with the int32
>> ABI/API. Adding the int64 USE flags to existing packages will make it
>> hard to write ebuilds for users who want e.g. both int32 and int64
>> openblas at the same time. My personal recommendation is to provide a
>> new set of packages, say eselect-{blas64,lapack64} + sci-libs/lapack64,
>> which are clearly isolated with the int32 ones. These new packages
>> should change SONAMEs into say libblas64.so and liblapack64.so. This is
>> the same as what I'm doing for Debian's 64bit-indexing BLAS/LAPACK
>> packages.
>
> heroxbd:
>> We can make int64 as a use flag and by enabling it we install 2 sets of
>> libraries: libblas.so and libblas64.so. What do you think?
>
> kiwifb:
>> I know why you want that @heroxbd - you want to avoid ebuild
>> duplication. On the face of it it is less ebuilds to maintain in
>> exchange of more complex ebuilds. However I think it would still be
>> wise to have at least separate virtuals - that means dependencies on
>> int32 and int64 are completely separate and allow a different set of
>> implementations without impinging on each other.
>
>> Whether to go for doubling the ebuilds or making them more complex is
>> up to the maintainer. And we have to deal with the headers in any
>> case.
>
> cdluminate:
>> I agree with @kiwifb . At least we need
>> virtual/{blas64,cblas64,lapack64,lapacke64}. For the implementations
>> we could introduce two new USE flags: int32 and int64, where each of
>> them could trigger a separate full build, and int32 is enabled by
>> default. Same for the eselect packages.
>
> I agree with int32 and int64 USE flags. but int32 and int64 is too
> simple and misleading, how about index_int32 and index_int64?  They are
> named after python_targets_python3_7.
>
> Virtual/blas could do the same: virtual/blas[index_int32,index_int64].
>
> Yours,
> Benda