[gentoo-user] Re: File system testing

[James]

Hervé Guillemet <herve <at> guillemet.org> writes:

>
> Le 16/09/2014 21:07, James a écrit :
> >
> > By now many are familiar with my keen interest in clustering gentoo
> > systems. So, what most cluster technologies use is a distributed file
> > system on top of the local (HD/SDD) file system.


> Have you found this document :
> http://hal.inria.fr/hal-00789086/PDF/a_survey_of_dfs.pdf

Hello Herve,

Yes, I read the document and it is a good introduction to some
of my issues on which file system(s) to use for clustering. But, it's 
more of a survey than a comparison/benchmark study, which would be
really beneficial. 

DFS are moving so fast now, and their setups and features are
rarely a one to one match. For example, (currently) the best load balancing
you find, is actually in the apps that run above the cluster software. [1]
Some of the performance/resource-utilizations of the files systems/resources
 are determined by real-time analytics with graphical displays. I'm
not sure that load balancing even belongs in a DFS, yet in the paper
you reference, it was prominently discussed. Things are moving so
fast there in the distributed-*/cluster/cluster-tools/cluster-apps
space, one really need a system set up to apply almost daily patches
for testing. I never realize just how much reading is necessary just
to understand the current landscape in clustering.

I'm trying to figure out an echo_system where gentoo folks can experiment
wtih mesos clustering for scientific applications. After that, the
more general case should be mature enough for general purpose applications.
I'm avoiding the clustered web arena, as that is just too much for
me to digest; so somebody else could champion that part of all of
those Apache-cluster technologies.

Thanks for the document link!

James

[1]