Re: [gentoo-user] File system testing

[J. Roeleveld]

On Tuesday, September 16, 2014 07:07:38 PM James wrote:
> Hello,
>
> By now many are familiar with my keen interest in clustering gentoo
> systems. So, what most cluster technologies use is a distributed file
> system on top of the local (HD/SDD) file system. Naturally not
> all file systems, particularly the distributed file systems, have
> straightforward instructions. Also, an device file system, such as
> XFS and a distibuted (on top of the device file system) combination
> may not work very well when paired. So a variety of testing is
> something I'm researching. Eliminiation of either file system
> listed below, due to Gentoo User Experience is most welcome information,
> as well as tips and tricks to setting up any file system.
>
>
> Distributed File Systems (DFS):
> HDFS (poor performance)
> Lustre
> Ceph
> XtreemFS
> GlusterFS
> MooseFS
> FhGFS (BeeGFS) soon to be entirely open sourced?
> Any other distributed file systems I should consider using?
>
> Local (Device) File Systems LFS:
> btrfs
> zfs
> ext4
> xfs
>
> Obviously I do not what to test all combinations of DFS/LocalFS
> so your comments are extremely welcome as is any and all
> related information.
>
> James

James,

Is my understanding correct that the top list all require one of the bottom 
list?
Eg. the "clustering" FSs only ensure the files on the LFSs are 
duplicated/spread over the various nodes?

I would normally expect the clustering FS to be either the full layer or a 
clustered block-device where an FS can be placed on top.

Otherwise it seems more like a network filesystem with caching options (See 
AFS).

I am also interested in these filesystems, but for a slightly different 
scenario:

- 2 servers in remote locations (different offices)
- 1 of these has all the files stored (server A) at the main office
- The other (server B - remote office) needs to "offer" all files from serverA

When server B needs to supply a file, it needs to check if the local copy is 
still the "valid" version. If yes, supply the local copy, otherwise download 
from server A. When a file is changed, server A needs to be updated.
While server B is sharing a file, the file needs to be locked on server A 
preventing simultaneous updates.

I prefer not to supply the same amount of storage at server B as server A has.
The remote location generally only needs access to 5% of the total amount of 
files stored on server A. But not always the same 5%.

Does anyone know of a filesystem that can handle this?

--
Joost