Re: [gentoo-user] Re: File system testing

[Rich Freeman]

The HTML...it hurts my eyes...  :)

On Thu, Sep 18, 2014 at 4:24 AM, J. Roeleveld <joost@××××××××.org> wrote:
>
> On Wednesday, September 17, 2014 08:56:28 PM James wrote:
>
>> Alec Ten Harmsel <alec <at> alectenharmsel.com> writes:
>
>> > As far as HDFS goes, I would only set that up if you will use it for
>> > Hadoop or related tools. It's highly specific, and the performance is
>> > not good unless you're doing a massively parallel read (what it was
>> > designed for). I can elaborate why if anyone is actually interested.
>

FYI - one very big limitation of hdfs is its minimum filesize is
something huge like 1MB or something like that.  Hadoop was designed
to take a REALLY big input file and chunk it up.  If you use hdfs to
store something like /usr/portage it will turn into the sort of
monstrosity that you'd actually need a cluster to store.

>
> My current understanding is:
>
> - ZFS is production ready, but due to licensing issues, not included in the
> kernel
>
> - BTRFS is included, but not yet production ready with all planned features
>

Your understanding of their maturity is fairly accurate.  They also
aren't 100% moving in the same direction - btrfs aims more to be a
general-purpose filesystem replacement especially for smaller systems,
and zfs is more focused on the enterprise, so it lacks features like
raid reshaping (who needs to add 1 disk to a raid5 when you can just
add 5 more disks to your 30 disk storage system).

I think btrfs has a bit more hope of being an ext4 replacement some
day for both this reason and the licensing issue.  That in no way
detracts from the usefulness of zfs, especially for larger deployments
where the few areas where btrfs is more flexible would probably be
looked at as gimmicks (kind of like being able to build your whole OS
from source :) ).

> For me, Raid6-like functionality is an absolute requirement and latest I
> know is that that isn't implemented in BTRFS yet. Does anyone know when that
> will be implemented and reliable? Eg. what time-frame are we talking about?
>

I suspect we're talking months before it is really implemented, and
much longer before it is reliable.  Right now btrfs can write raid6,
but it can't really read it.  That is, it operates just fine until you
actually lose a disk containing something other than parity, and then
it loses access to the data.  This code is only in the kernel for
development purposes and nobody advocates using it for production.
Most of the code in btrfs which is reliable has been around for years,
like raid1 support, and obviously it will be years until the raid5/6
code reaches that point.  I am using btrfs mainly because once that
day comes it will be much easier to migrate to it from btrfs raid1
than from zfs (which has no mechanism for migrating raid levels
in-place (that is, within an existing vdev) - you would need to add
new drives to the pool, migrate the data, and remove the old drives
from the pool, which is nice if you have a big stack of drives and
spare sata ports lying around like you would in a SAN).

--
Rich