| 1 |
Andrew Savchenko <bircoph <at> gentoo.org> writes: |
| 2 |
|
| 3 |
|
| 4 |
> While it is good to have another solution available, I don't see |
| 5 |
> any real benefits of FhgFS/BeeGFS compared to Lustre these days. |
| 6 |
> At the time where FhgFS was created, Lustre indeed was unable to |
| 7 |
> use multiple metadata servers, so this was a bottleneck. But now |
| 8 |
> Lustre also supports distributed metadata, so they should on par in |
| 9 |
> this matter. |
| 10 |
|
| 11 |
Interesting thesis. I only have anecdotal information, from those |
| 12 |
I've encountered who are willing to converse, privately. Many more sites |
| 13 |
exist than are publicized as I think most (scientific) groups have a keen |
| 14 |
interest in distributed processing, in an open source semantic. |
| 15 |
I did notice the '9999' version of lustre in portage (science overlay), but |
| 16 |
reading elsewhere I did not know it was still being actively developed? |
| 17 |
|
| 18 |
> On the other hand, Lustre has much larger community (e.g. see |
| 19 |
> TOP-500 list) and is much better tested (and even under such |
| 20 |
> conditions it has problems in some corner cases). Thus I see no |
| 21 |
> advantage in FhgFS for HPC setups. |
| 22 |
|
| 23 |
Strangely, the folks I have chatted up do not publish their test results |
| 24 |
as that would be quite a large undertaking to assure critics that the |
| 25 |
tests are fair and equivalent, with the only thing different being the |
| 26 |
local and cluster file systems. Lustre seems to have a bad rap, but that |
| 27 |
may be due to folks testing much earlier versions. I'm no authority on the |
| 28 |
subject; just trying to ferret out pathways for robust cluster computing |
| 29 |
on gentoo; although containers are useful, my focus is on the |
| 30 |
leanest/fastest bare metal HPC Opensource approach. to clusters on gentoo. |
| 31 |
|
| 32 |
|
| 33 |
> Of course world of parallel distributed file systems is very |
| 34 |
> versatile, so for different tasks/workloads different file systems |
| 35 |
> are the most suitable, but for typical IB-based HPC storage I see |
| 36 |
> no better solution than Lustre at this moment. |
| 37 |
|
| 38 |
|
| 39 |
YES. But also these test/benchmarks should include Cephfs, gluster, and |
| 40 |
tachyon if not many others. [1] Perhaps we should encourage some of our |
| 41 |
gentoo-devs, to put up a wiki for gentoo-HPC, with at least a working |
| 42 |
framework of packages suggested, including all the DFS tricks therein ? |
| 43 |
Me, I'm just stumbling my way around to try to figure out a resonable |
| 44 |
pathway to HPC on gentoo. |
| 45 |
|
| 46 |
I thought that systemd was going to dominate these cluster-container wars |
| 47 |
until I started reading up on Docker's acquisition of the main dev at Alpine |
| 48 |
linux and the rapid movement of Docker to 'subsume' Alpine linux as it's |
| 49 |
distro for releases [2]. Alpine leverages OpenRC and eudev and Docker is |
| 50 |
preparing for battle with other container offerings, commercially, so this |
| 51 |
does suggest that the performance battle with clusters is now openly |
| 52 |
challenging the systemd proponents for performance bragging rights. Combined |
| 53 |
with the question of the DFS, it does lsuggest some publish test comparing |
| 54 |
these different approaches would be of keen interest to a wide audience. |
| 55 |
|
| 56 |
The only test code I am aware of for HPC on gentoo is sys-cluster/hpl |
| 57 |
and I'm not sure how well that will exercise the DFS performance questions. |
| 58 |
|
| 59 |
|
| 60 |
> Best regards, |
| 61 |
> Andrew Savchenko |
| 62 |
|
| 63 |
James |
| 64 |
|
| 65 |
|
| 66 |
[1] |
| 67 |
http://www.datanami.com/2016/02/23/meet-alluxio-the-distributed-file-system-formerly-known-as-tachyon/ |
| 68 |
|
| 69 |
[2] https://www.brianchristner.io/docker-is-moving-to-alpine-linux/ |
Archives