1 |
From memory, as far as I know only octave can only be configured to |
2 |
use int64 (or at least it could). |
3 |
As you can imagine supporting int64 requires specific support |
4 |
on the part of the application using the library. There has been |
5 |
a few entries in bugzilla about people not understand why R broke |
6 |
after linking it to int64 blas/lapack in the early days. |
7 |
|
8 |
Apart from Julia and a few commercial code there are a few custom |
9 |
scientific applications here and there using int64. Very few widely |
10 |
distributed packages. |
11 |
|
12 |
In that light, I think the proposal is quite sensible. People |
13 |
who want to use int64 are likely to use either MKL or openblas, |
14 |
may be BLIS [especially handy for C++ but then you probably don’t |
15 |
care too much about the BLAS compatibility layer]. |
16 |
|
17 |
François |
18 |
|
19 |
> On 17/07/2019, at 10:39 PM, Mo Zhou <lumin@××××××.org> wrote: |
20 |
> |
21 |
> Hi Gentoo Science team, |
22 |
> |
23 |
> In the sci overlay there is a long existing feature "int64" for the |
24 |
> BLAS/LAPACK packages, with which the type of array index in the |
25 |
> API/ABI will be changed into int64_t so that BLAS/LAPACK libraries |
26 |
> could be able to handle super large arrays. The reason of existence |
27 |
> of such feature stems from a Fortran compiler feature which could |
28 |
> change the default INTEGER size (e.g. 32bit, 64bit). |
29 |
> |
30 |
> Recently the BLAS/LAPACK runtime switching mechanism has been merged |
31 |
> into ::gentoo, but you may find no counterpart to the "int64" feature |
32 |
> existing in ::sci. According to previous discussion, we could indeed |
33 |
> add a new alternative group called, for example, BLAS64 and LAPACK64. |
34 |
> However, it must be pointed out that, even if the "BLAS64" thing is |
35 |
> well supported by commonly used optimized implementations such as |
36 |
> BLIS, OpenBLAS, and MKL, the 64-bit-indexing feature is not well |
37 |
> supported by the reference CBLAS. Even worse, there is no common |
38 |
> standard for such "BLAS64" and "LAPACK64" libraries in terms of |
39 |
> ABI and SONAME. That means every move on the "BLAS64/LAPACK64" |
40 |
> runtime switching mechanism will be highly experimental and |
41 |
> may easily be broken by the authorities who define standards. |
42 |
> |
43 |
> Before I started the GSoC project for Gentoo I had already started |
44 |
> working on adding BLAS64/LAPACK64 alternatives. We Debian science |
45 |
> team thought it's fine to experiment even if there is no standard |
46 |
> at all. Exactly due to this reason, the 64-bit-index feature wasn't |
47 |
> put on the original GSoC schedule. I thought we could first |
48 |
> experiment it on Debian. |
49 |
> |
50 |
> However, now we have to face the problem again because some packages |
51 |
> such as Julia require the 64-bit-index ABI. (ah, yes, I was aware of |
52 |
> this since I also maintain Julia for Debian). Combining the fact that |
53 |
> there is no standard (even de-facto standard), and the demand of |
54 |
> some packages such as Julia, Benda and I reached an agreement that |
55 |
> we could simply add 64-bit-index support for OpenBLAS and postpone |
56 |
> the runtime switching feature for BLAS64/LAPACK64. |
57 |
> |
58 |
> Julia calls BLAS64 and LAPACK64 through FFI. Theoretically Julia |
59 |
> could use any BLAS/LAPACK implementations but actually Julia's |
60 |
> LinearAlgebra standard library is not quite ready for such a |
61 |
> switching feature. Hence Debian's Julia package is actually |
62 |
> specifically linked against libopenblas.so (32-bit) and it is |
63 |
> irrelevant to the runtime switching feature. Afterall there is |
64 |
> no problem by forcing Julia users to use OpenBLAS. |
65 |
> |
66 |
> In a word, what Benda and I plan to do are: |
67 |
> |
68 |
> * add index-64bit USE flag to openblas, which compiles an extra |
69 |
> libopenblas64.so for Julia's use. |
70 |
> * postpone the plan for BLAS64/LAPACK64 runtime switching, and |
71 |
> experiment on Debian first. |
72 |
> |
73 |
> That said, if anyone believes the BLAS64/LAPACK64 feature is sort |
74 |
> of urgent need, we can discuss further and revise the plan again. |
75 |
> |
76 |
> Thanks. |
77 |
> |
78 |
> See also: |
79 |
> [1] the 64-bit-indexing feature has been postponed by |
80 |
> reference lapack upstream: |
81 |
> https://github.com/Reference-LAPACK/lapack/issues/184 |
82 |
> |
83 |
> |
84 |
> On 2019-07-11 13:59, Benda Xu wrote: |
85 |
>> Hi, |
86 |
>> |
87 |
>> This discussion emerged from https://github.com/gentoo/sci/pull/876. I |
88 |
>> am taking it here for better future reference. |
89 |
>> |
90 |
>> cdluminate: |
91 |
>>> It would be problematic to provide int64 ABI/API through libblas.so or |
92 |
>>> liblapack.so since that would lead to a mess with the int32 |
93 |
>>> ABI/API. Adding the int64 USE flags to existing packages will make it |
94 |
>>> hard to write ebuilds for users who want e.g. both int32 and int64 |
95 |
>>> openblas at the same time. My personal recommendation is to provide a |
96 |
>>> new set of packages, say eselect-{blas64,lapack64} + sci-libs/lapack64, |
97 |
>>> which are clearly isolated with the int32 ones. These new packages |
98 |
>>> should change SONAMEs into say libblas64.so and liblapack64.so. This is |
99 |
>>> the same as what I'm doing for Debian's 64bit-indexing BLAS/LAPACK |
100 |
>>> packages. |
101 |
>> |
102 |
>> heroxbd: |
103 |
>>> We can make int64 as a use flag and by enabling it we install 2 sets of |
104 |
>>> libraries: libblas.so and libblas64.so. What do you think? |
105 |
>> |
106 |
>> kiwifb: |
107 |
>>> I know why you want that @heroxbd - you want to avoid ebuild |
108 |
>>> duplication. On the face of it it is less ebuilds to maintain in |
109 |
>>> exchange of more complex ebuilds. However I think it would still be |
110 |
>>> wise to have at least separate virtuals - that means dependencies on |
111 |
>>> int32 and int64 are completely separate and allow a different set of |
112 |
>>> implementations without impinging on each other. |
113 |
>> |
114 |
>>> Whether to go for doubling the ebuilds or making them more complex is |
115 |
>>> up to the maintainer. And we have to deal with the headers in any |
116 |
>>> case. |
117 |
>> |
118 |
>> cdluminate: |
119 |
>>> I agree with @kiwifb . At least we need |
120 |
>>> virtual/{blas64,cblas64,lapack64,lapacke64}. For the implementations |
121 |
>>> we could introduce two new USE flags: int32 and int64, where each of |
122 |
>>> them could trigger a separate full build, and int32 is enabled by |
123 |
>>> default. Same for the eselect packages. |
124 |
>> |
125 |
>> I agree with int32 and int64 USE flags. but int32 and int64 is too |
126 |
>> simple and misleading, how about index_int32 and index_int64? They are |
127 |
>> named after python_targets_python3_7. |
128 |
>> |
129 |
>> Virtual/blas could do the same: virtual/blas[index_int32,index_int64]. |
130 |
>> |
131 |
>> Yours, |
132 |
>> Benda |