[gentoo-soc] Benchmarking suite - Report 10

[Andrea Arteaga]

This is the report of the project "Automated benchmark suite for
numerical libraries in Gentoo" for the week 25 July-1 August.
This is also available on my blog [1].

== Project description ==

The project aims to develop a simple yet powerful automated system of
benchmarking for numerical libraries. The Gentoo software system
provides many implementations of widely used standards such as BLAS,
CBLAS, LAPACK, ScaLAPACK and some other numerical library such as
FFTW, MKL. The developed tools will aid the system maintener to choose
the best suited implementation with respect to the machine hardware
and to test the samer implementation or different ones with different
compilers, compiler versions and compile flags.

== Progrees during the week ==

This week has been devoted to the following features:

* Management of the dependencies
* Improvement of the PBLAS/ScaLAPACK tests and investigation of some
BTL-related problem -- see below
* 3-Dimensional FFTW tests

Before this week's work, only packages without unsatisfied
dependencies on the main system could be tested. For example, if one
did not have cmake installed in the system, he could not test eigen,
as it requires cmake to be installed in order to compile. This is not
a problem anymore. The dependencies are emerged into the "sandbox"
root along with the package to be tested, and the environment is
adjusted in order to make use of the software installed. The delay of
this report is due to this work, which has required more work as
expected -- a small rewriting of the preliminary part of the script
has been done --; now the report can be complete.

Some investigation has been done regarding the input matrices and
vectors for the tests. In order to be fair, the tests should produce
the same inputfor the libraries, as different input could require
different work by the libraries. The linear congruential random number
generatorhas been written, along with the new-style matrix generator.
This will not completely solve the problem, but it represents a step
in the right direction. Now the PBLAS/ScaLAPACK tests use this method,
and the results are satisfactory.

The work with FFTW is finished for now. 1-dimensional, 2-dimensional
and 3-dimensional complex discrete Fourier transformations are tested
in both forward and backward versions and with the "estimate" and
"measure" strategies. If the time will allow it, I will also add sinus
and cosinus tranformations.

== Plan for the next week ==

During the following week I will implement the tests for METIS. This
is a set of executables and libraries that compute graph and mesh
partitioning -- along with other numerical functions --, which are
widely used in the field of numerical simulations and other scientific
computations.

The project has intensively gained features in the last weeks. After
the METIS tests implementation, the quality of the whole project will
be tested and improved. A considerable part of the time has been
devoted to the BTL library, and more work will be performed in order
to submit the new code upstream. Some work has to be also done in
order to make the script more user-friendly; this means managing the
exceptions that occur sometimes, avoid conflicts between tests already
done and the currect ones, reduce the probability of errors. Much
documentation has to be written. The ebuild will be splitted in two
part, a Gentoo-specific one and a general one containing the modified
BTL.

All this work will eventually provide a finished and usable project,
which is my primary objective for the programme. I do not want to
provide an incomplete or undocumented software; I rather prefer
pospone the inclusion of new features after the programme end, if the
time is not enough.

Best regards
Andrea Arteaga

[1] http://wp.me/pzWEm-8A