1 |
On Fri, Jun 21, 2013 at 3:31 AM, Duncan <1i5t5.duncan@×××.net> wrote: |
2 |
> So with 4k block sizes on a 5-device raid6, you'd have 20k stripes, 12k |
3 |
> in data across three devices, and 8k of parity across the other two |
4 |
> devices. |
5 |
|
6 |
With mdadm on a 5-device raid6 with 512K chunks you have 1.5M in a |
7 |
stripe, not 20k. If you modify one block it needs to read all 1.5M, |
8 |
or it needs to read at least the old chunk on the single drive to be |
9 |
modified and both old parity chunks (which on such a small array is 3 |
10 |
disks either way). |
11 |
|
12 |
> Forth, back to the parity. Remember, raid5/6 has all that parity that it |
13 |
> writes out (but basically never reads in normal mode, only when degraded, |
14 |
> in ordered to reconstruct the data from the missing device(s)), but |
15 |
> doesn't actually use it for integrity checking. |
16 |
|
17 |
I wasn't aware of this - I can't believe it isn't even an option |
18 |
either. Note to self - start doing weekly scrubs... |
19 |
|
20 |
> The single down side to raid1 as opposed to raid5/6 is the loss of the |
21 |
> extra space made available by the data striping, 3*single-device-space in |
22 |
> the case of 5-way raid6 (or 4-way raid5) vs. 1*single-device-space in the |
23 |
> case of raid1. Otherwise, no contest, hands down, raid1 over raid6. |
24 |
|
25 |
This is a HUGE downside. The only downside to raid1 over not having |
26 |
raid at all is that your disk space cost doubles. raid5/6 is |
27 |
considerably cheaper in that regard. In a 5-disk raid5 the cost of |
28 |
redundancy is only 25% more, vs a 100% additional cost for raid1. To |
29 |
accomplish the same space as a 5-disk raid5 you'd need 8 disks. Sure, |
30 |
read performance would be vastly superior, but if you're going to |
31 |
spend $300 more on hard drives and whatever it takes to get so many |
32 |
SATA ports on your system you could instead add an extra 32GB of RAM |
33 |
or put your OS on a mirrored SSD. I suspect that both of those |
34 |
options on a typical workload are going to make a far bigger |
35 |
improvement in performance. |
36 |
|
37 |
Which is better really depends on your workload. In my case much of |
38 |
my raid space is used my mythtv, or for storage of stuff I only |
39 |
occasionally use. In these use cases the performance of the raid5 is |
40 |
more than adequate, and I'd rather be able to keep shows around for an |
41 |
extra 6 months in HD than have the DVR respond a millisecond faster |
42 |
when I hit play. If you really have sustained random access of the |
43 |
bulk of your data than a raid1 would make much more sense. |
44 |
|
45 |
> So several points on btrfs: |
46 |
> |
47 |
> 1) It's still in heavy development. |
48 |
|
49 |
That is what is keeping me away. I won't touch it until I can use it |
50 |
with raid5, and the first common containing that hit the kernel weeks |
51 |
ago I think (and it has known gaps). Until it is stable I'm sticking |
52 |
with my current setup. |
53 |
|
54 |
> 2) RAID levels work QUITE a bit differently on btrfs. In particular, |
55 |
> what btrfs calls raid1 mode (with the same applying to raid10) is simply |
56 |
> two-way-mirroring, NO MATTER THE NUMBER OF DEVICES. There's no multi-way |
57 |
> mirroring yet available |
58 |
|
59 |
Odd, for some reason I thought it let you specify arbitrary numbers of |
60 |
copies, but looking around I think you're right. It does store two |
61 |
copies of metadata regardless of the number of drives unless you |
62 |
override this. |
63 |
|
64 |
However, if one considered raid1 expensive, having multiple layers of |
65 |
redundancy is REALLY expensive if you aren't using Reed Solomon and |
66 |
many data disks. |
67 |
|
68 |
From my standpoint I don't think raid1 is the best use of money in |
69 |
most cases, either for performance OR for data security. If you want |
70 |
performance the money is probably better spent on other components. |
71 |
If you want data security the money is probably better spent on |
72 |
offline backups. However, this very-much depends on how the disks |
73 |
will be used - there are certainly cases where raid1 is your best |
74 |
option. |
75 |
|
76 |
Rich |